Proxmox 6.1 and 6.2 PCIe Passthrough 

You're welcome!

Very glad to have pointed you in the right direction. That’s definitely an interesting setup. Passing the controller through I’ve heard of. But not someone passing through the boot drive as well. Good to know

Glad I could help! I love BookStack. MediaWiki has it's place but BookStack is just so easy. :)

Neil ion glad it helped and bookstack is pretty awesome.

Jacob Noori glad it helped and you’re right any older guides don’t really capture this.

Glad it helped. I live Proxmox. But sometimes you gotta work at it LOL.

Nexusband glad it helped. There are lines for AMD in the Ultimate Guide to GPU passthrough and on the Proxmox wiki. But I use intel chips in my R710s so I’ve never had to use anything other than the intel lines.

I'm glad it helped... NVIDIA is a pain. :)

yes it is....now i'm trying to resolve the problem with Code 43. and Nvidia Drivers have successfully installed. Any ideas? :-(

@@yo3isl are your settings correct for the VM so that it doesn't know it's a VM? pause right here and look at the settings for args and CPU... ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-_fkKIMF3HZw.html

config file is ok, problem seems to be from chipset. root@r710:~# dmesg | grep 'remapping' [ 0.854121] DMAR-IR: This system BIOS has enabled interrupt remapping on a chipset that contains an erratum making that feature unstable. To maintain system stability interrupt remapping is being disabled. Please contact your BIOS vendor for an update. Source: www.suse.com/support/kb/doc/?id=000018235

Mihai Stroescu no I get that error as well. Have you done the second part of the video? What version of Proxmox are you on?

Glad it helped you out... :)

That's a great question but it's probably because it doesn't effect everyone the same way. I mean they stated they were changing to that new kernel. So I'm betting we were supposed to understand any impacts to a non standard configuration.... Which I didn't lol. I did find that answer in the Proxmox forums, and I believe the response was from a Proxmox admin/rep. But it did take some digging. I found it also in a Reddit forum post, but only after I had found it myself and made the video...

Done. They're in the description now. Thanks for asking. I should have credited them all along.

I have this one saved as well. One of the best out there. www.reddit.com/r/homelab/comments/b5xpua/the_ultimate_beginners_guide_to_gpu_passthrough/

It’s right at the top of the screen. On the Proxmox machine the VM config files are located at /etc/pve/qemu-server/ and you can use a terminal shell for the node right in the Proxmox webpage.

That's a tough question. I think Proxmox is pretty easy, but to be fair, I've never used unraid.

If you're on 6.2 you don't need the kernel_irqchip=on at all...

Дмитрий Диков I don’t think it’s a matter of the hardware. It’s how iommu groups get handled now. So i believe anyone will need to do it this way.

Should be the same concept but unfortunately I’m not on version 7 just yet.

Sorry but that's definitely outside my wheelhouse. So when you added that line you were able to start the VM correctly and pass through the HBA? That's about all I could help with. I don't use an HBA like that in my systems so I have no experience with it.

@@AllenSampsell yeah, i have a Dell T330 Server with LSI Controller passthrough to a Freenas vm on proxmox, everything works fine until I start putting some load to the freenas (transferring big files) , it just crashed, still investigating what the cause could be. thanks for replying.

Salvador Ortega don’t rule out other distro like Open Media Vault. OMV is Linux based where freenas is BSD I think.

I have not upgraded to 7 yet. Just before it came out I had finished a massive project of creating a cluster and having the VMs loaded from a new NAS I built. So it just hasn't felt worth updating just yet. I wanted to give it more time to shake out the bugs. For speed differences it could boil down to the HBA in IT mode i'm using vs you. Or the storage type you're choosing for your VMs. So many variables, you're just gonna have to test everything. Good luck!

@@AllenSampsell Thank you for the fast response Allen! I am definitely going to keep pushing through. As a software developer, it has been extremely valuable. Even though it may be a little slow at the moment, the pros of being able to spin up different VM's and tear them down far outweighs the cons of it. Depending on the environment, I will also use Ubuntu Server Multipass, but it fills a different gap. Your videos have definitely been helpful. I certainly appreciate the time and effort you put into them. Keep the content coming!

Code 43 can pop up for several reasons so it's hard to say, and I'm not on 7.0 quite yet. Is your last question about a proxmox host device? I would think that's dependent on your hardware. Not sure it matters but I suppose it might for pcie pass-through. I know not much help. Sorry.

​@@AllenSampsell thanks for reply, and for tutorial, finally i managed to passthrough quadro k4000 on proxmox7/win10 by just moving gpu to secondary pcie slot, and put in first slot a cheap gpu to load, and for vm file i removed the whole line of args now the issue still remaining for the first pcie, tried to mount that nvs 310 gpu on a second machine, still get code 43, looks need some params to unload it from host for moment i will test a gtx on first slot to see if there any chance with it

@@AllenSampsell gtx works fine but with args on first pcie slot, nvs still have issue even in other slots

@@Xgamerz_channel ahh so you’re trying to use the GPU that the system is using? I’ve heard it can be done but since I use Dell servers there’s always a built in one for display and then only one slot for a card. So I’ve never tried it. Good luck though. I image you need to blacklist both cards and run Proxmox headless.

“Fix” what? If your question is, does 6.3 behave any different than 6.2? I haven’t run into anything yet.

What’s the error you’re seeing? Does it boot at all?

@@AllenSampsell On the external monitor, it starts the ProxMox boot screen, then cuts off. I can then go into the console (VNC) and see the GPU is being seen, but not being used. Actually, just got it to work. Had to edit the VMID.conf file to make vga: none and that did it. Your info also may have assisted. :)

@@skotsmith4269 glad it’s working now!

Sorry to hear that. Was it working before on that same hardware and the update broke it?

There is an entire procedure for enabling iommu as part of the PCIe passthrough. Have you done all of that? Did this work before and now an update has disabled something? It’s hard to help without further details.

@@AllenSampsell I followed the guide to activate iommu for proxmox and updated the grub and updated the modules and by default dell r720 has sriov activated and virtualization activated as well, the boot error i get is PTE write access fail, and PTE read access failed, befor i had installed proxmox i had vmware esxi 6.7

@@mrj-rock4074 hmm ok. So I would remove the GPU passthrough and verify that the OS will boot without the passthrough. If you’re still getting errors then you can troubleshoot those. But the error I saw that the video is based on was very clearly related to the fix action. I’m not seeing that with your error.

@@AllenSampsell the only way i could get past the boot error was to connect the iso and boot to it then boot to os repair and then it will boot but when i try to select the pci device to give to vm and it tells me iommu is not activated

@@mrj-rock4074 well then yes try moving the unsafe interrupts to the grub like it says in the video. Can’t hurt at this point.

Ahh, well I enable RDP or Remote Desktop in the Windows VM I use and connect with a program that handles either RDP or VNC or SSH. It handles quite a few protocols and it's call mRemoteng mremoteng.org/

psychoboi well I don’t think full-upgrade in Proxmox is something you should do anyway. I would only upgrade via the web GUI if possible. You may want to save your conf files and your VM files to a NAS or other storage and then reinstall Proxmox to 6.2. You’re gonna have a lot of steps but you should then setup using the method for 6.2 in the video.

Config on (bookstack) and lxc already backed up that is no issue. The main issue is black screen on windows login. > Using uefi boot > Nvidia GTX 1070 (slot 5) (no rom file required) > Hp Dl380 g9 Everything works on v6.1, and I didn't change qemu version (no flags added) (a reddit guide is good enough) but when I installed 6.2 (which is latest) all things f-up (only windows). Hackintosh and Linux still working fine as they should be. I followed some guide which enable msi on GPU the issue is I never able to boot on safe mode so recovery mode with cmd registery editor but I didn't save the change bsod/bootloop occur. I try to explain sorry for long message but reddit didn't helped me they said stay downgraded I think only you can help me

psychoboi for the Windows VM, when you run it under 6.2 what error are you seeing? My VM just wouldn’t start and gave me an iommu group error. Which is why I made the video. If you’re having a different issue I might not be able to help

@@AllenSampsell I solved it finally > Install 6.1 GPU passthrough and with drivers and enable MSI > then edit grub and modules from your video and then full-upgrade. > I learnt if I install custom nvidia drivers it works like a charm thanks for helping me If you tell me that will great help you said "back to remoter" what is that ? Can you tell me about that (hardware/software) thanks

psychoboi glad I could help. And what do you mean by custom Nvidia drivers? The software that I use for RDP/VNC//SSH is called mremoteng. mremoteng.org It’s a great piece of software.

Fixed! So turns out it was not enabled in my BIOS/UEFI settings.

@@Deefourteen thanks for the update. And yeah that’s a different error than I was talking about. My first question would have been Have you verified the state of iommu? So your response/fix makes a lot of sense.

My Bookstack? No. But one of the best sources of that info is here www.reddit.com/r/homelab/comments/b5xpua/the_ultimate_beginners_guide_to_gpu_passthrough/

You mean the guest VM? So that you don't get Code 43? I'll see what I can do...

@@AllenSampsell Code 43 will appear in which location, cause I don't have any display for the vm neither monitor nor console

@@AllenSampsell I own a NVIDIA 2080 Super and running it on a AMD platform, after some research and dumping the rom of the graphics card. I found that in a reference link on the pci passthrough page of promox wiki, it said OVMF does not work for type 0 card. And the rom dumping result shows that the card is type 0 which is legacy BIOS but some website said this card does support UEFI. I am so confused now.

留在彼此的身邊 code 43 is what shows up inside the Windows VM when you load Nvidia drivers and their software detects that’s it is a VM. It then disables the GPU with a code 43. Lots of the steps in guides for GPU passthrough are trying to get you up to and around that issue.

留在彼此的身邊 it’s a pretty advanced card I would be surprised if it can’t be done in OVMF where it needs to be. What errors are you seeing? Also, you might take this discussion over to my forum vs doing it here on RU-vid. www.allenscloud.com/forum/

lol my mic has foam on it. What I need is to have my server rack not be in the same room right next to my desk where I record. But thanks for the feedback