4 ms·

I really wish that everybody would finally admit that the "fixed function pipeline" is dead and shift the development work to making the use of the GPU compute

by bsder 4y ago

I really wish that everybody would finally admit that the "fixed function pipeline" is dead and shift the development work to making the use of the GPU compute units mainstream. Unreal Engine 5 ("nanite") shows that the high end game engines are already stepping around a lot of the fixed function pipeline. It's just going to get worse. The problem is that the GPU compute systems throw away decades of CPU processor development. This is good in some ways. However, it's very bad in others. GPUs don't yet have a well-defined memory model (to be fair, even CPUs are still a bit sketchy in places)--so you can't guarantee atomicity or progress or fairness. GPUs don't have the equivalent of an MMU--so they can't be partitioned or protected. etc. The problem is that switching to a "compute" model and making that work well would commodify the GPU--and that's the last thing that AMD and NVIDIA want.

moonchild 4y ago
> I really wish that everybody would finally admit that the "fixed function pipeline" is dead There is a great deal of space between 'fixed function pipeline' (which no one is using) and 'all compute all the time' (which pretty much no one is using for real-time graphics either). > GPUs don't yet have a well-defined memory model Nvidia has a memory model. (I have heard--from an nvidia employee--that amd plays a bit more fast and loose.) https://www.youtube.com/watch?v=VogqOscJYvk https://www.youtube.com/watch?v=VogqOscJYvk > GPUs don't have the equivalent of an MMU--so they can't be partitioned or protected. etc. You do not need an MMU to partition or to protect. (Not to say that GPU security isn't a shitshow.) > The problem is that switching to a "compute" model and making that work well would commodify the GPU--and that's the last thing that AMD and NVIDIA want. https://s22.q4cdn.com/364334381/files/doc_financials/2023/Q223/Rev_by_Mkt_Qtrly_Trend_Q223.pdf https://s22.q4cdn.com/364334381/files/doc_financials/2023/Q2... look at where nvidia's revenue is coming from, and tell me again that they're not already there.
- slabity 4y ago
  > There is a great deal of space between 'fixed function pipeline' (which no one is using) and 'all compute all the time' (which pretty much no one is using for real-time graphics). Assuming "fixed function pipeline" means the currently standard programmable pipeline that most people use (and not the extremely old fixed-function pipeline), is there really much of a difference from 100% compute pipelines? If compute shaders had the ability to manually invoke the hardware rasterizer, what functional difference would there be? The only other major difference I can think of is how the non-compute pipelines can optimize keeping data in registers/cache between different shader steps that can't really be done with compute shaders.
  - Const-me 4y ago
    > non-compute pipelines can optimize keeping data in registers/cache between different shader steps that can't really be done with compute shaders That’s exactly the problem addressed by mesh shaders. Each thread group of the mesh shader outputs a batch of triangles to be rasterized. These batches have a hard limit of 32kb of output data per batch. I think such a small size limit is there so the vertex + primitive data stays in the faster memory, inside the GPU chip.
    - slabity 4y ago
      I agree, but that only reinforces the point that there doesn't even seem to be much of a difference between the older graphical pipeline and a 100% compute pipeline. If graphics drivers could just allow compute shaders to manually call the rasterizer and give functionality to control what data stays in memory in different parts of a pipeline/command-buffer, then you'd be able to implement both the older graphical pipeline and the newer task/mesh shader pipeline entirely in compute shaders. I'm not entirely experienced in how graphics hardware works at a lower level though, so I'm sure I must be missing something important.
johntb86 4y ago
> GPUs don't have the equivalent of an MMU--so they can't be partitioned or protected. etc. All modern GPUs (last 5-10 years at least) have MMUs.
- rjsw 4y ago
  I got the feeling that the GP post meant a MMU that protected GPU threads from each other, not just an IOMMU between GPU and CPU.
  - deleted 4y ago
    [deleted]
  - johntb86 4y ago
    GPUs can do that. For example, Intel GPUs since Ivy Bridge have had a PPGTT (Per Process Graphics Translation Tables) to protect the memory of each process from other processes. It can also be used at a finer-grained level if there are multiple VkDevices in the same process. You need that sort of MMU to have any sort of secure process isolation without doing expensive command buffer validation on the CPU.
raphlinus 4y ago
> GPUs don't yet have a well-defined memory model Vulkan has a perfectly fine memory model, and it's non-optional in Vulkan 1.3. The rest of GPU land of course has some catching up to do. It guarantees atomicity but does not guarantee progress (there's active work on characterizing that). And fairness is probably not something you want, as it almost always comes at a performance cost.
yazzku 4y ago
The only true statement above is that UE5 and other modern engines are moving quite a bit of stuff to compute. I'm not sure where you're getting the other stuff from, but it's all provably false, except for maybe the last bit, which hinges on speculation.
- bsder 4y ago
  You can feel free to disagree with my opinions, but my statements are certainly not provably false. Nanite IS stepping around the fixed function pipeline because the triangles they draw are on the order or less of the size of a pixel. They only allow the fixed function pipeline to render the triangles when they are big (ie. almost never). Current GPU memory models don't give you enough to guarantee progress. Most models seem to do that well enough, but there are pathological cases that aren't that uncommon. Both of these are backed up from Nanite presentations. So, if you think those are false, know that you are disagreeing with Unreal who has put it on the record. GPUs don't have MMUs. They have memory interfaces, but they don't have the parts that allow you to protect memory, handle system vs user protections, and manage load. This is why things like Shadertoy can bring your computer to a grinding halt or even cause you to crash. About the only thing "false" was the fact that the graphics cards don't have a defined memory model. As was pointed out, Vulkan 1.3 does define a memory model. However, the manufacturers have yet to catch up nor are they compliant with that memory model. Perchance you would like to revise your statement?
  - yazzku 4y ago
    Yes, let's revise it. > I really wish that everybody would finally admit that the "fixed function pipeline" is dead and shift the development work to making the use of the GPU compute units mainstream. > Unreal Engine 5 ("nanite") shows that the high end game engines are already stepping around a lot of the fixed function pipeline. It's just going to get worse. Nanite does everything you stated here and thereafter in your reply, but Nanite does not speak for high-end game engines from other companies, which do use ffp units like the rasterizer. Yes, those guys are also moving quite a bit of stuff to compute as I already stated, but to say that the ffp is dead at present is overblown. Nanite does everything you mentioned and for the reasons you mentioned; what I pointed out was not true was the generalization of that to other modern and respectable engines. > GPUs don't yet have a well-defined memory model (to be fair, even CPUs are still a bit sketchy in places)--so you can't guarantee atomicity or progress or fairness. This we already cleared up. > GPUs don't have the equivalent of an MMU--so they can't be partitioned or protected. etc. GPUs do have MMUs, like others have stated, that handle things like virtual address translation. See, for example: https://learn.microsoft.com/en-us/windows-hardware/drivers/display/gpummu-model https://learn.microsoft.com/en-us/windows-hardware/drivers/d... Yes, you are probably right that they don't have complete isolation -- has anybody developed side-channel attacks for this yet? -- but that doesn't mean they lack an MMU. Also, the part about handling load is a bit nuanced. Drivers typically put timeouts on calls, for example, that'll kill the application when it induces a hang (or something long enough to be flagged as a hang.) I've run into this on mobile devices in particular. So yeah, Shadertoy does grind down low-end devices and the result is not exactly pretty, but that does not mean there are absolutely no load protections in place (there are). > The problem is that switching to a "compute" model and making that work well would commodify the GPU--and that's the last thing that AMD and NVIDIA want. Speculation.
fulafel 4y ago
Here's MS driver model explaining that on a "GpuMmu" model driver, "Each process has separate CPU and GPU virtual address spaces that use distinct page tables": https://learn.microsoft.com/en-us/windows-hardware/drivers/display/gpummu-model https://learn.microsoft.com/en-us/windows-hardware/drivers/d... Also the virtualization support advertised by NV and AMD uses this. Allowing for GPU partitioning, used eg in VDI setups. Eg NVidia seems to even have multiple generations of HW modes for this, one timeslicing based one (but I guess still HW enforced memory protection) and one core-splitting based one called MIG. And here's AMD in 2016: https://www.amd.com/system/files/documents/amd-mxgpu-white-paper.pdf https://www.amd.com/system/files/documents/amd-mxgpu-white-p... As usual with GPUs the problems seem to be more in the drivers and rest of sw stack, bugs, difficult programmability, proprietary nature, fragmentation, etc than strictly HW features.
- bsder 4y ago
  Interesting. I did not know this. Probably because this seems to be confined to the data center boards. MIG for NVIDIA seems confined to the data center chips: https://developer.nvidia.com/blog/dividing-nvidia-a30-gpus-and-conquering-multiple-workloads/ https://developer.nvidia.com/blog/dividing-nvidia-a30-gpus-a... For AMD, from your reference, this is Firepro only? Any other VDI references seemed to be data center only as well.

by bsder 4y ago

moonchild 4y ago

> I really wish that everybody would finally admit that the "fixed function pipeline" is dead There is a great deal of space between 'fixed function pipeline' (which no one is using) and 'all compute all the time' (which pretty much no one is using for real-time graphics either). > GPUs don't yet have a well-defined memory model Nvidia has a memory model. (I have heard--from an nvidia employee--that amd plays a bit more fast and loose.) https://www.youtube.com/watch?v=VogqOscJYvk https://www.youtube.com/watch?v=VogqOscJYvk > GPUs don't have the equivalent of an MMU--so they can't be partitioned or protected. etc. You do not need an MMU to partition or to protect. (Not to say that GPU security isn't a shitshow.) > The problem is that switching to a "compute" model and making that work well would commodify the GPU--and that's the last thing that AMD and NVIDIA want. https://s22.q4cdn.com/364334381/files/doc_financials/2023/Q223/Rev_by_Mkt_Qtrly_Trend_Q223.pdf https://s22.q4cdn.com/364334381/files/doc_financials/2023/Q2... look at where nvidia's revenue is coming from, and tell me again that they're not already there.

slabity 4y ago
> There is a great deal of space between 'fixed function pipeline' (which no one is using) and 'all compute all the time' (which pretty much no one is using for real-time graphics). Assuming "fixed function pipeline" means the currently standard programmable pipeline that most people use (and not the extremely old fixed-function pipeline), is there really much of a difference from 100% compute pipelines? If compute shaders had the ability to manually invoke the hardware rasterizer, what functional difference would there be? The only other major difference I can think of is how the non-compute pipelines can optimize keeping data in registers/cache between different shader steps that can't really be done with compute shaders.
- Const-me 4y ago
  > non-compute pipelines can optimize keeping data in registers/cache between different shader steps that can't really be done with compute shaders That’s exactly the problem addressed by mesh shaders. Each thread group of the mesh shader outputs a batch of triangles to be rasterized. These batches have a hard limit of 32kb of output data per batch. I think such a small size limit is there so the vertex + primitive data stays in the faster memory, inside the GPU chip.
  - slabity 4y ago
    I agree, but that only reinforces the point that there doesn't even seem to be much of a difference between the older graphical pipeline and a 100% compute pipeline. If graphics drivers could just allow compute shaders to manually call the rasterizer and give functionality to control what data stays in memory in different parts of a pipeline/command-buffer, then you'd be able to implement both the older graphical pipeline and the newer task/mesh shader pipeline entirely in compute shaders. I'm not entirely experienced in how graphics hardware works at a lower level though, so I'm sure I must be missing something important.

johntb86 4y ago

> GPUs don't have the equivalent of an MMU--so they can't be partitioned or protected. etc. All modern GPUs (last 5-10 years at least) have MMUs.

rjsw 4y ago
I got the feeling that the GP post meant a MMU that protected GPU threads from each other, not just an IOMMU between GPU and CPU.
- deleted 4y ago
  [deleted]
- johntb86 4y ago
  GPUs can do that. For example, Intel GPUs since Ivy Bridge have had a PPGTT (Per Process Graphics Translation Tables) to protect the memory of each process from other processes. It can also be used at a finer-grained level if there are multiple VkDevices in the same process. You need that sort of MMU to have any sort of secure process isolation without doing expensive command buffer validation on the CPU.

raphlinus 4y ago

> GPUs don't yet have a well-defined memory model Vulkan has a perfectly fine memory model, and it's non-optional in Vulkan 1.3. The rest of GPU land of course has some catching up to do. It guarantees atomicity but does not guarantee progress (there's active work on characterizing that). And fairness is probably not something you want, as it almost always comes at a performance cost.

yazzku 4y ago

The only true statement above is that UE5 and other modern engines are moving quite a bit of stuff to compute. I'm not sure where you're getting the other stuff from, but it's all provably false, except for maybe the last bit, which hinges on speculation.

bsder 4y ago
You can feel free to disagree with my opinions, but my statements are certainly not provably false. Nanite IS stepping around the fixed function pipeline because the triangles they draw are on the order or less of the size of a pixel. They only allow the fixed function pipeline to render the triangles when they are big (ie. almost never). Current GPU memory models don't give you enough to guarantee progress. Most models seem to do that well enough, but there are pathological cases that aren't that uncommon. Both of these are backed up from Nanite presentations. So, if you think those are false, know that you are disagreeing with Unreal who has put it on the record. GPUs don't have MMUs. They have memory interfaces, but they don't have the parts that allow you to protect memory, handle system vs user protections, and manage load. This is why things like Shadertoy can bring your computer to a grinding halt or even cause you to crash. About the only thing "false" was the fact that the graphics cards don't have a defined memory model. As was pointed out, Vulkan 1.3 does define a memory model. However, the manufacturers have yet to catch up nor are they compliant with that memory model. Perchance you would like to revise your statement?
- yazzku 4y ago
  Yes, let's revise it. > I really wish that everybody would finally admit that the "fixed function pipeline" is dead and shift the development work to making the use of the GPU compute units mainstream. > Unreal Engine 5 ("nanite") shows that the high end game engines are already stepping around a lot of the fixed function pipeline. It's just going to get worse. Nanite does everything you stated here and thereafter in your reply, but Nanite does not speak for high-end game engines from other companies, which do use ffp units like the rasterizer. Yes, those guys are also moving quite a bit of stuff to compute as I already stated, but to say that the ffp is dead at present is overblown. Nanite does everything you mentioned and for the reasons you mentioned; what I pointed out was not true was the generalization of that to other modern and respectable engines. > GPUs don't yet have a well-defined memory model (to be fair, even CPUs are still a bit sketchy in places)--so you can't guarantee atomicity or progress or fairness. This we already cleared up. > GPUs don't have the equivalent of an MMU--so they can't be partitioned or protected. etc. GPUs do have MMUs, like others have stated, that handle things like virtual address translation. See, for example: https://learn.microsoft.com/en-us/windows-hardware/drivers/display/gpummu-model https://learn.microsoft.com/en-us/windows-hardware/drivers/d... Yes, you are probably right that they don't have complete isolation -- has anybody developed side-channel attacks for this yet? -- but that doesn't mean they lack an MMU. Also, the part about handling load is a bit nuanced. Drivers typically put timeouts on calls, for example, that'll kill the application when it induces a hang (or something long enough to be flagged as a hang.) I've run into this on mobile devices in particular. So yeah, Shadertoy does grind down low-end devices and the result is not exactly pretty, but that does not mean there are absolutely no load protections in place (there are). > The problem is that switching to a "compute" model and making that work well would commodify the GPU--and that's the last thing that AMD and NVIDIA want. Speculation.

fulafel 4y ago

Here's MS driver model explaining that on a "GpuMmu" model driver, "Each process has separate CPU and GPU virtual address spaces that use distinct page tables": https://learn.microsoft.com/en-us/windows-hardware/drivers/display/gpummu-model https://learn.microsoft.com/en-us/windows-hardware/drivers/d... Also the virtualization support advertised by NV and AMD uses this. Allowing for GPU partitioning, used eg in VDI setups. Eg NVidia seems to even have multiple generations of HW modes for this, one timeslicing based one (but I guess still HW enforced memory protection) and one core-splitting based one called MIG. And here's AMD in 2016: https://www.amd.com/system/files/documents/amd-mxgpu-white-paper.pdf https://www.amd.com/system/files/documents/amd-mxgpu-white-p... As usual with GPUs the problems seem to be more in the drivers and rest of sw stack, bugs, difficult programmability, proprietary nature, fragmentation, etc than strictly HW features.

bsder 4y ago
Interesting. I did not know this. Probably because this seems to be confined to the data center boards. MIG for NVIDIA seems confined to the data center chips: https://developer.nvidia.com/blog/dividing-nvidia-a30-gpus-and-conquering-multiple-workloads/ https://developer.nvidia.com/blog/dividing-nvidia-a30-gpus-a... For AMD, from your reference, this is Firepro only? Any other VDI references seemed to be data center only as well.