GPU Passthrough Update on Proxmox 5.4-6

Подписаться 3,4 тыс.

Просмотров 17 тыс.

50% 1

Even though this is an "Update" video on how to add a GPU to your Dell R710 in Proxmox... I also provide information on the creation of a Windows 10 VM and USB passthrough. Some info applies to more than just an R710.
Graphics Cards that should fit in an R710 without mod... (SHOULD):
PNY NVIDIA GeForce GT 710
amzn.to/2W6BfZ1
EVGA GeForce GT 730 2GB
amzn.to/2YOJ2Yo
Great new guide on Reddit:
/ the_ultimate_beginners...
Stable virtio drivers:
fedorapeople.org/groups/virt/...
Link to big RAM hugepages fix:
forum.proxmox.com/threads/vm-...

Наука

Опубликовано:

29 июл 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 73

@EricDenny 5 лет назад

Great help! Helped me get past similar issue on my R910.

@Kamel419 4 года назад

exceptional explanations, thanks so much for going into depth on this topic.

@ClearlyCero 4 года назад

Really helpful, thanks!

@mattb9876 5 лет назад

BTW, I forgot to say this earlier... Great vid! So much great updated info in this vid that's absolutely necessary and sounds much easier to do with the Proxmox update. Only a week ago I finally got my GPU passthrough working with my PNY GT710 - that took almost two weeks to figure out using Proxmox v5.4-5. Today my new Zotac GT710 arrived along with two 4x PCIE v2.0 USB 3.0 cards! I'm very excited to see if I can get all this working together much faster than I did the original GPU passthrough! LOL I'm also adding a KVM and will be attempting to do a dual passthrough for an extended screen in Win10, one, with sound, to my main big receiver and tv, with the other through the KVM to share my monitor with a second device. Hopefully this won't be as painful! (sulking now and going to go update Proxmox to the new version.... sigh...)

@AllenSampsell 5 лет назад

Matt B thanks I love doing this stuff on my own. I just share because that’s fun too. I learn so much from RU-vid and other sources why not try to help others... And the upgrade should be easy. You’ve switched to the non-enterprise repo right? It’s as easy as hitting the update then upgrade buttons.

@mattb9876 5 лет назад

@@AllenSampsell Yep! I have the non enterprise repo set and the "reminder" screen shushed. I don't have auto-updstes set up yet because first I want to get a feeling for the update processes, then I'll set it and forget it. When I finished the passthrough issue, I started studying the boot processes, so I've only been actively breaking my Proxmox settings anyway while poking and prodding at the MBR, GRUB, and kernel modules, trying to get a better understanding of the PCIE mapping and associated issues. I figured if I let it upgrade, it might fix something I broke on purpose!

@maltoNitho 5 лет назад

Woot! TYVM.

@illanesp 3 года назад

HOw can I do this with my creative sound card or muy USB DAC??

@AllenSampsell 3 года назад

With your VM turned off go to the Hardware tab and hit Add and select PCI Device. Select the PCI card you want to pass through and see if it boots. Unfortunately that's the only advice I can give you. I haven't passed through any other cards before.Never had the need.

@MikeSmitran 3 года назад

Great tut :) Thanks! how big should the psu be to support those two cards in one case?

@AllenSampsell 3 года назад

That depends on what two graphics cards you use. If use use a simple one for Proxmox. And a beefy card for GPU passthrough then you can probably get away with a 500 to to 650. If you use two beefy graphics you might need to go bigger. If you’re using a Dell R710 like I am you’ll be limited by space and start up voltages. But in a normal PC you’re going to be ok.

@MikeSmitran 3 года назад

@@AllenSampsell Thanks Allen! Yes two beefy ones :) with 850

@bijeshbalakrishnan9118 3 года назад

i have a dell R510 with Dell PERC H800 SAS RAID Controller VVGYD PowerVault 1gb Cache and Battery and hp storage array D2700. i installed proxmox and it can see the storage array. I tried to install freenas but the vm cannot see my array disks. can you help

@AllenSampsell 3 года назад

If I had to guess the RAID card you’re using doesn’t support IT mode. ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-PeFJtjVvGyc.html

@tonyrking 5 лет назад

Just a quick and most likely stupid question.. why would you want to use the GPU on the host? I'm trying to see how useful it would be to passthough the GPU to the Guest? :/

@AllenSampsell 5 лет назад

Well it depends on the configuration. If you only have one GPU in your computer you kind of need the host to see it. And then you need a second GPU to passthrough to VMs. On a Dell R710 it has a graphics card built in. And whatever GPU you want to passthrough needs to be blacklisted from the host so it can't see/use it.

@DeniskaNYC 4 года назад

Great video, thank you. Any plans to make Update video for Proxmox 6?

@AllenSampsell 4 года назад

DennyNY yes but I’ll need to test it in 6 first to see how much has changed. I plan to do an upgrade to PVE 6 video first and a few others.

@DeniskaNYC 4 года назад

@@AllenSampsell I'm new to proxmox, but have same setup (R710 + GT710). Follow your video I manage install and passthrough GPU w/o any problem. I have Proxmox 6 installed - no problem with RAM memory anymore. I increase up to 32 Gb and have no problem. Only thing what strange , when VM is running it shows what 95.92% of RAM in use but in Windows only 10%

@AllenSampsell 4 года назад

DennyNY did you set the RAM for the Windows VM to use ballooning? I’m pretty sure I advise against it as it causes problems. So to the Proxmox system that amount of RAM is now “in use” or dedicated to that VM. But inside the VM windows knows how much it’s actually using. So I believe this is normal.

@DeniskaNYC 4 года назад

@@AllenSampsell no, I follow your advise no ballooning was used. I also just try add 22 cores w/o any config modification. So far work flawlessly. For connection to VM I use NoMachine app. Like it so far.

@mattb9876 5 лет назад

No!!!! LOL I spent over a week reading, researching, testing, crying, etc.... And got my PNY GT710 working great as a passthrough only last week.... and suddenly there's a new update to Proxmox! I did this using Proxmox version 5.4-5 and NOW they update the very thing I struggled with AND someone released a new, better, and updated GPU walkthrough! I'm almost mad now. I feel like I was part of a mystery comedy and spent all my time trying to kick a door down instead of just checking to see if it was unlocked! LOL OTOH I really learned a lot through the pain of exploring and testing so much GPU passthrough issues. I'm running an r710 with Xeon 5680's (3.33GHz with 12 cores each), 128GB RAM, 128GB SSD boot drive, and six 2TB SATAs (wish they were true SAS) on an h700 RAID controller. I'll post a couple more comments in a few minutes about some of my slight variations from what you've done in your set-up

@AllenSampsell 5 лет назад

Matt B I feel your pain... I had this working and then suddenly it wasn’t. That R710 I’m doing this on is a dual 6 core 3.33ghz machine with 48 gigs of RAM. A 120gig SSD boot drive. And a smattering of SAS and SATA drives of varying sizes and speeds. It’s also now running a H700. That was one of the “upgrades”. I just should have gone with another H200 in IT mode. Lesson learned though.

@mattb9876 5 лет назад

Read my second comment (of three... I had a lot to say!) for thoughts which pertain to RAID controllers and indirectly to IT mode and non-hardware RAID. I had so much to say, I had to split it all up 😁

@squalazzo 5 лет назад

i've a Dell 7010 USFF, so no space for addon GPU, can i passthrough the integrated (i5 cpu) gpu? I tried with previous proxmox version without success, 2 months ago...

@AllenSampsell 5 лет назад

You're running Proxmox on that? Wow, but unfortunately I think the answer would be no... If it's integrated and Proxmox is using it for for displaying the VNC console I think you're out of luck. That form factor is challenging for any kind of project, even something like pfSense. You'd probably have to use USB NICs to expand it. And I'm not sure how stable that would be.

@squalazzo 5 лет назад

@@AllenSampsell aside the gpu as said, everything else was working flawlessly, 8gb ram and 240gb ssd +1tb hybrid hdd

@AllenSampsell 5 лет назад

@@squalazzo I'm glad it's working out for you. What kind of VMs are you running on it? Heavy workload? I image your power bill is much better than mine... :)

@squalazzo 5 лет назад

@@AllenSampsell i tested 2 at same time, my goal was to have a W10 vm with gpu pt so to use as a normal desktop, and an ubuntu one with hassio and various add-ons... But as no gpu pt I reverted to just W10 and using an udoo x86 for hassio... The 7010 was about 30W, using a power metering socket

@AllenSampsell 5 лет назад

@@squalazzo Well with those requirements I suggest running Windows 10 baremetal and using Docker for hassio if that's the Home Assistant program I've seen... you could even try a Docker container of Domoticz... :)

@Rickety3263 4 года назад

I have an Intel 4-port ethernet nic. After modifying the kernel commandline and updating /etc/modules, I passed it thru! Each copper port comes up as a separate address to pass thru, which works, BUT... I canNOT pass one copper port to guest A and another copper port to guest B and have them run at the same time. Is this expected behavior?? Is there a solution or work around so 2 VMs can be assigned different parts of the same card?

@AllenSampsell 4 года назад

Shawn Gollatz well I don’t know why you had to do anything at the command line but I have a USB card in my R710 and each port shows up separately. I’ve tested that passing one through to a VM works but I’ve not tested a different port to two different simultaneous VMs.

@Rickety3263 4 года назад

Allen Sampsell I had to activate IOMMU and followed the documentation and it worked. I might be pushing the limits of a type one hypervisor by splitting things “that” much. Would be great though! Im testing on a cluster of Dell 7010 SFF’s before I drop a few grand on machines with more horsepower

@AllenSampsell 4 года назад

Shawn Gollatz ahh gotcha. I just assumed that was on already and you had to do something special just for that card.

@merlingt1 4 года назад

When you remote desktop, do you see an entry in device manager -> display adapters, for "remote display ...."? When I install my Nvidia drivers successfully (no error codes) this option disappears and I can no longer RDP into Windows and I just a get a black screen and eventually kicks me off (although I can still RDP with noVNC). I am at wits ends right now with my GTX 750 Ti.

@AllenSampsell 4 года назад

merlingt1 if you can still use noVNC from Proxmox then you haven’t turned on the check box in the PCI passthrough hardware settings for your GPU. Once you do that the only way into the VM is using RDP or another VNC server you install in the VM. It also removes the Generic Display Adapter that noVNC uses.

@merlingt1 4 года назад

@@AllenSampsell I ended up starting fresh with a new VM. I have now checked all the boxes of the GPU: "All functions", "Primary GPU", "Rom-Bar", and "PCI-Express" and disabled noVNC through Hardware->Display->None. I was able to remote desktop and install the Nvidia drivers. Under device manager->Display adapter, I see "Remote display..." and my Nvidia GTX750Ti. However, it is still not using the GPU. If I run dxdiag and click display it tells me that it is using the remote display adapter, and if I run the GPU UserBenchmark, it fails when it gets to the directx GPU test and tells me that it cannot run in a remote display connection. PS. I can also get into the Nvidia Control Panel and everything appears as expected but the display is very slow and it appears to just be using the remote display adapter.

@AllenSampsell 4 года назад

merlingt1 well yes. You’re using RDP so you are in fact using the remote display adapter. Are you using a dummy plug to emulate a monitor?

@merlingt1 4 года назад

@@AllenSampsell Yup, well I have it connected to an actual monitor for testing and I can see the Windows splash screen when the VM is on. Still slow as molasses despite everything appearing installed correct, no errors, etc. I am going to give this one last crack tomorrow and if I cannot get it working I am just going to dump Proxmox for something else. Thanks for your help.

@merlingt1 4 года назад

This seems like a not uncommon problem...this person is having the exact same problem as me. www.reddit.com/r/homelab/comments/guhtqw/proxmox_with_nvidia_gpu_passthrough_poor/

@mattb9876 5 лет назад

A couple things different in my GPU passthrough for Win 10 and another random thing... The most important difference I noticed are the HDD settings. Your suggestion of the SCSI HDD is correct and the Cache = "Write back" is correct if there is NO hardware RAID being used for the VM drive. However, if a hardware RAID controller is being used, the best setting is Cache = "Write through" along with, in Advanced options, both "No backup" and "Skip replication" checked. The reason for this is because Proxmox runs everything on zfs file system. The zfs file system is the up-and-coming OS I very reluctantly agree will (and should) soon be used almost exclusively for large data systems. But I digress, another topic for another day... If you're running a system with a hardware RAID controller, the controller uses its own cache and controls the writing and verification. By selecting "Write through", a write is sent to the hardware RAID controller, cached as necessary, then the RAID controller confirms it received the data, completing the "write through to the card" which will then be independently responsible for ensuring the actual writing of the data. If set to "Write back", your system will wait until the RAID controller has reported finished writing and confirming the writes, not just caching them. Due to the hardware cache on the RAID card, the data isn't lost even if the system fails and any pending writes will be done when up and running again. On a system without large amounts of writing, there probably won't be much difference. The "No backup" and ""Skip replication" checks are used for the zfs filesystem. For those not familiar with zfs, in a simplified nutshell, it's a software RAID. You can use these settings without changes, but if you already have hardware RAID, the zfs software RAID becomes redundant and uses more CPU cycles and a not-insignificant amount of extra RAM. As we start surpassing our "safe RAID storage" limits, I think this kind of system dragging filesystem will unfortunately become necessary. I had issues with getting my PCIE ports mapped properly due to BIOS updates which implement "expensive" software lockdowns and validations to protect the CPUs from malware like Meltdown and Spectre. In order to remap PCIEs, one of the settings that had to be changed disables that protection. I don't like that I had to bypass that protection in order to map my PCIEs differently, but I don't know any other way. I'd really like to know if there is a way to use the remapping while staying safe. Perhaps there's a way to unlock, remap, then relock? Maybe I've misunderstood the process?

@AllenSampsell 5 лет назад

Matt B Matt B so you should check that again, Proxmox itself is using LVM and you have the option of using ZFS if you want to for storage. Which I do not for various reasons. I have a mirrored ZFS drive setup for testing but not for serious use. While I am using the H700 in that machine I use Raid 0 on all the drives. So I’m not sure Write Though would make a difference but I’m certainly going to test it now. So thanks for that. And as for the remapping I don’t think I’ve updated to the BIOS update that implemented those fixes but even if I did I’m not that worried about them. Maybe I should be but this is just my home lab and not a business.

@mattb9876 5 лет назад

@@AllenSampsell I'll definitely go look again at filesystem types again. It's very possible my brain simply became addled during the time I spent researching the mapping issues! 👽 Like you, this is my home hobby kit too, so I'm very open to critique and corrections! Any time I'm wrong and corrected, I know I've just learned something new and it makes me the better for it! During daylight hours I'm a DBA, so this is very different for me and a new kind of challenge that I'm enjoying. I've been very interested in and slowly learning about IoT devices and methods for about a year now and decided to upgrade my hardware about 2 months ago. That's when I got my r710 and chose Proxmox to handle all the crazy virtualized scenarios I've thought about throwing at the physical IoT devices I'm playing with. I have licenses for a couple other Win Server editions too (2012 and 2016 I think?) and I assume they'll have the same issues as Win10 which is why I started with that. I eventually plan to have at least a single instance of two full OSs (not just containers), probably a Windows and a Linux, running full time at the same time for my testing and use the two GPUs for display. That's where I'll start being able to incorporate sensors and various inputs into some databases and hope I can find new ways to use that info. Although I'm usually really good with networking, I'm still struggling a bit with the way Proxmox handles its networking, so that will be my next deep-learning objective.

@AllenSampsell 5 лет назад

@@mattb9876 I stand corrected... I was thinking of the local-lvm and not the "local" which it turns out actually is ZFS. I completely missed the distinction between the two in the documentation. But I've never actually used "local" to store anything other than ISO files, so I've just never run into an issue with it. I always use small drives for boot/local and my storage drives are mapped using Directory method using ext4. Again thanks for making me actually look. :) And I'm jealous of your DBA knowledge. I work, well lets just say project management with a tiny bit of sysadmin stuff thrown in. I do all this at home for fun and learning.

@mattb9876 5 лет назад

@@AllenSampsell We do it at home because that's where we have the freedom to choose what we do! DBA work is probably the easiest in the whole industry but it can still have really cool challenges. I taught myself the basics of SQL and moved into a DBA position about 5 yrs later. If you like it, do it! Being a PM can be really good too though, but from my experience, PMs tend to always be stuck in the middle. A good way to move beyond that is by learning some SQL and consider analysis.

@AllenSampsell 5 лет назад

Matt B ohh I’ve dabbled in SQL and I’m not really a PM it’s just one aspect of what I do. My title is SATCOM Systems Analyst it’s just hard to explain what I do lol.

@voodooutt 4 года назад

ok, :) i wanna know how you got your proxmox GUI into dark mode.

@AllenSampsell 4 года назад

github.com/Weilbyte/PVEDiscordDark and using the Stylus plugin

@voodooutt 4 года назад

@@AllenSampsell Thanks, Dark mode FTW - I just did the server side installation with the shell.

@AllenSampsell 4 года назад

voodooutt never tried the server side. Hope it works out for you. Wit the Stylus browser extension it’s pretty easy to turn it off if I want to. And it applies itself to all my PVEs, which I like.

@voodooutt 4 года назад

@@AllenSampsell It did work, no problems. I did not use Stylus because I use the new chromium Edge browser.

@voodooutt 4 года назад

I am trying to find the vendor ID of the graphics I'm trying to pass through: A10-7850K with Radeon R7 Series. I can find some specifics on the processor here ( www.amd.com/en/products/specifications/processors ) but having a difficult time with how to exactly get this code right, as I'm using the AMD R7 and not an nvidia processor: args: -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=NV43FIX,kvm=off' I am asking because I can see the card in the VM but I'm getting the dreaded Code 43 inside the VM.

@AllenSampsell 4 года назад

That hv_vendor_id=NV43FIX is not something you need for an AMD graphics card. The NV43FIX is for Nvidia getting a code 43. So you can take that out. Try leaving the machine as Q35 and adding the arg kernel_irqchip=on

@voodooutt 4 года назад

@@AllenSampsell add as 'kernel_irqchip=on' or without '

@voodooutt 4 года назад

either way, does not help with the code 43.

@AllenSampsell 4 года назад

@@voodooutt edit the .conf file for your VM and add a line that looks like this... args: kernel_irqchip=on if there are more than one arg you might need those ' ' I'm not sure. I just blew out my Proxmox test box where I actually just last week setup Passthough with an AMD R7. Did it with both Windows 10 and Ubuntu 20.

@AllenSampsell 4 года назад

voodooutt ok. Wanted to rule it out as I was using that arg last week when setting up my test VMs. I got an R7 passed through to both Windows 10 and Ubuntu and had Plex using it for hardware encoding. This is in an R710 though. Not sure what hardware you’re running or what guide you’re using. I may need to do a video on non nvidia GPU passthrough.

@jovanyeo2941 5 лет назад

hi Allen, Did you managed to get GTX 1070 passthrough?

@jovanyeo2941 5 лет назад

currently running on promoxo 6.0 beta with a VM, running on Windows 10 1903 and installed with a zotac GTX1070. Motherboard: Supermicro SCA X11

@AllenSampsell 5 лет назад

No, but I’ve only run Proxmox on an R710 server that can’t naturally support the power requirements for a 1070. So I’m sure it can be done with the right setup. Also, I don’t own a 1070 to test with :)

@jovanyeo2941 5 лет назад

Thanks for yr prompt response.. I cant get works. Only managed to get it works on AMD gpu.. I have tired the following: and it cant works 1) sk1080 patches 2) edited the gpu bios 3) follow the guide u have recommended 4) signed the certificate None of the above works. Hope u can share , if you have more idea.

@AllenSampsell 5 лет назад

Jovan Yeo so what error are you seeing maybe someone else reading these comments can help as I probably can’t.

@jovanyeo2941 5 лет назад

@@AllenSampsell depends , if I do sk1080patches I will get error 52 else is error 43.

@mr-biz 4 года назад

Thank you so much for this video. I have been pulling my hair out for days trying to get GPU passthrough PROXMOX. With Proxmox 6.1 there were some changes made to the q35 machine to improve pci passthrough (see - forum.proxmox.com/threads/fix-for-pci-passthrough-and-issues-with-proxmox-6-gui-regex-that-needs-resolving-after-upgrade.56507/). Consequently, I tried from shell: "qm set -args '-machine type=q35,kernel_irqchip=on -cpu host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=NV43FIX,kvm=off'" . I was also able to get 2 cores x 4 threads = 8 plus 32GB of Ram straight off the bat.

@AllenSampsell 4 года назад

I should have paid closer attention to the comments.... I didn't figure this out until a few days ago and just posted a video including this information yesterday. I appreciate you passing this on to everyone else who may have benefited before me. :)

@charliebrownau 5 лет назад

Gday. Please upload this to bitchute

@AllenSampsell 5 лет назад

Why would I want to do that? You having issues seeing it on RU-vid? Bitchute seems to be a haven for hate filled content and I'm not sure I want to be associated with them.