AMD releases open-source GIM driver aimed at GPU virtualization, support for mainstream Radeon GPUs coming later

I've always wondered when we'll see them sold on mobile-on-desktop boards for a penny: I guess it's sooner now rather than later.

Phoenix based Mini-ITX boards with a nice 8-core 5.1GHz Ryzen 7 are currently €300 with a similarly potent 780m iGPU (not that either is really great at gaming).

And for €500 you'll get a 16 core Dragon Range, just in case you'd prefer a beefier dGPU.

Those probably aren't making big bucks for AMD either, but at least they aren't selling them at a huge loss, unlike these Intel parts, which have cost a ton to make, but can't deliver any meaningful value advantage.

Still, scrapping them would be even more expensive, so I am keeping a lookout on AliExpress.

abufrejoval said:

I'm not sure I follow.

PCI pass-through and GPU virtualization are entirely different things. PCI pass-through just passes a physical port into a VM. GPU virtualization is when you do GPU HW acceleration inside VMs by using a host GPU. Currently this is only possible with expensive data center GPUs because there is no hypervisor driver for consumers GPUs that supports this.

We have hundreds of VDI desktops with virtualizated GPUs by putting a bunch of A100s in our APEX stack.

palladin9479 said:

PCI pass-through and GPU virtualization are entirely different things. PCI pass-through just passes a physical port into a VM. GPU virtualization is when you do GPU HW acceleration inside VMs by using a host GPU. Currently this is only possible with expensive data center GPUs because there is no hypervisor driver for consumers GPUs that supports this.

We have hundreds of VDI desktops with virtualizated GPUs by putting a bunch of A100s in our APEX stack.

Perhaps you can enlighten us on how this works in your environment and what exactly the VMs get to see in terms of virtual hardware?

The ability to partition a GPU into smaller chunks has been around for a long time and across vendors. Even Intel iGPUs on "desktop" Xeons supported this, at least according to press releases of the time. And those partitioned vGPUs would then be passed through as distinct GPUs (smaller) GPUs to the VMs.

I actually still have a Haswell Xeon somewhere, which should be able to do that, but with an iGPU so weak I never felt like sharing it: it got a GTX 980ti instead, which had all Quadro features fused off or disabled in drivers.

That's the classic domain of Nvidia's Quadro series cards and good enough for some security sensitive CAD work, rather common in the defense sector from what I heard.

To my understanding this partitioning is what Nvidia also does for their cloud gaming with GeFroce Now.

Of course you could also add an abstraction layer which uses GPU resources on the host to accelerate GPU operations inside a VM for VDI purposes. As far as I know this has been done at one time on Microsoft Servers for RDP sessions under the name RemoteFX.

I remember playing around with that and it was complex to set up and yielded modest gains at Direct X10 abstractions, and targeting AutoCAD or similar, nothing involving modern facilities like programmable shaders.

I've also used that approach on Linux using VirtGL and run some graphics benchmarks (Univention Singularity among them) on V100 GPUs sitting in a data center 600km apart from the notebook it rendered on via VNC: since the V100 is a compute device, it was actually the only way to get graphics out of it and performanc wasn't bad.... with exclusive access for a single client.

When it comes to CUDA workloads or Direct X12 or Vulkan games with programmable shaders and massive textures, to the best of my knowledge they can't deal with dynamic or overcommitted resources, they'll ask about the number of GPU cores and their share of VRAM available on startup and expect to have those resources permanently available until they finish. They organize their wave fronts all around those GPU cores and if some of those cores were to go missing or get pre-empted for somebody elses application, that could bring entire wave fronts to a stall.

And the memory pressure on VRAM would be even worse: you can't just swap gigabytes of textures (or machine learning model data) over a PCIe bus and get acceptable performance levels.

If I'm wrong I'm happy to learn about it but I just can't see several thousand GPU cores task switch dynamically, while the near lock-step wave front nature of GPGPU code simply can't deal with those stepping out of line.

abufrejoval said:

Perhaps you can enlighten us on how this works in your environment and what exactly the VMs get to see in terms of virtual hardware?

In the case of the A40s each VM is a Windows desktop complete with full DX11 3D acceleration. This is used to provide acceleration to a host of applications that are designed to run on real GPUs.

https://www.nvidia.com/en-us/drivers/vgpu-software-driver/
Inside vmware we installed the nvidia vGPU drivers then configured Horizon to spin the desktops with the appropriate Windows vGPU driver.

Currently the only vGPU drivers that exist are for data center GPUs. Installing a commodity 4080 or 9070 XT into a server won't matter because there is no hypervisor vGPU driver nor a guest vGPU driver that could use it. AMD releasing theirs open source means the FOSS community can find ways to incorporate commodity AMD GPUs into it allowing for cheap desktop virtualization.

I had to look up our system RQ and I believe we are using A40 cards. They are the server headless add in GPUs. Here's is the documentation for driver support.

https://docs.nvidia.com/vgpu/13.0/grid-vgpu-release-notes-vmware-vsphere/index.html

I carefully explained why this isn't nearly as huge a thing as many might be led to believe and you refuse that by arguing that that can't be true, because you're doing it.

But you fail to realize that your use case isn't what a TH audience would expect, while I patiently try to explain that the use cases they are more likely to expect, won't be supported the way they do, because it's impossible to do so.

palladin9479 said:

In the case of the A40s each VM is a Windows desktop complete with full DX11 3D acceleration. This is used to provide acceleration to a host of applications that are designed to run on real GPUs.

DX11 should be a first hint, because that's not DX12 or CUDA.

palladin9479 said:

https://www.nvidia.com/en-us/drivers/vgpu-software-driver/
Inside vmware we installed the nvidia vGPU drivers then configured Horizon to spin the desktops with the appropriate Windows vGPU driver.

Currently the only vGPU drivers that exist are for data center GPUs. Installing a commodity 4080 or 9070 XT into a server won't matter because there is no hypervisor vGPU driver nor a guest vGPU driver that could use it. AMD releasing theirs open source means the FOSS community can find ways to incorporate commodity AMD GPUs into it allowing for cheap desktop virtualization.

I had to look up our system RQ and I believe we are using A40 cards. They are the server headless add in GPUs. Here's is the documentation for driver support.

https://docs.nvidia.com/vgpu/13.0/grid-vgpu-release-notes-vmware-vsphere/index.html

So I've looked up your references and they pretty much confirm my theories.

You can use three basic ways to virtualize GPUs:
1. partition with pass-through of the slices
2. time-slice
3. new accelerated abstraction

Option #3 is what is historically perhaps the oldest variant, where Hypervisor vendors implement a new abstract 3D capable vGPU, which is using host GPU facilities to accelerate 3D primitives inside the VMs. I believe Nvida called that vSGA, KVM has a Vir-GL, VMware has its own 3D capable VGA, Microsoft has RemoteFX, and there is the VirtualGL I already mentioned.

I'm sure there is more but the main problem there remains that those abstractions can't keep up with the evolution of graphics hardware or overcome the base issues of context switch overhead and VRAM pressure. They lag far behind in API levels and performance, while in theory they can evenly distribute the resources of the host GPU among all VMs.

Option #2 and option #1 are the classic alternatives on how you slice and dice any multi-core computing resource, for a single core #2 is the only option and used in CPUs pretty much from the start. But as I tried to point out, the overhead of time-slicing is dictated by the context size, a few CPU registers are fine, the register files of thousands of SIMD GPU cores measure in megabytes and context switch overhead grew with the power of the GPUs: what appeared acceptable at first, might even still work for vector type CAD work, that doesn't involve complex shaders or GPGPU code may no longer be practical for modern API interactive (game) use. And then you'd still have to partition the VRAM, because even if paging of VRAM is technically supported, the performance penalty would be prohibitive for anything except some batch type HPC.

And that leads to option #1 as the most typical implementation in cases like cloud gaming.

But that isn't "huge", actually what each VM would get can only be smaller slices. And that won't be very popular when most people here actually yearn for the bigger GPU.

Long story short: when a vendor like AMD is giving something away for free, it's mostly because they can't really monetize it for much as it has low value. If there is a chance to make a buck they are much inclined to take it, even if sometimes they'll forgo a penny if they can make the competition lose a dime with it.

But here it's mostly marketing shenanigans and trying to raise misleading expectations, I don't want people to fall for.
If you didn't, that's great. But I had the impression that you were actually helping AMD in pushing a fairy tale, even if not intentional.

palladin9479 said:

PCI pass-through and GPU virtualization are entirely different things. PCI pass-through just passes a physical port into a VM. GPU virtualization is when you do GPU HW acceleration inside VMs by using a host GPU. Currently this is only possible with expensive data center GPUs because there is no hypervisor driver for consumers GPUs that supports this.

palladin9479 said:

Currently the only vGPU drivers that exist are for data center GPUs. Installing a commodity 4080 or 9070 XT into a server won't matter because there is no hypervisor vGPU driver nor a guest vGPU driver that could use it. AMD releasing theirs open source means the FOSS community can find ways to incorporate commodity AMD GPUs into it allowing for cheap desktop virtualization.

PCIe passthrough gets you GPU HW acceleration within the VM, on regular hardware with regular drivers. E.g. for my RX 6600, I pass it through to the VM, then install the regular AMD desktop graphics drivers from within the guest OS. Running ubuntu with the standard amdgpu driver on the host.