4 ms·

I wonder if at some point G(raphics)PUs and G(^HMachine-Learning)PUs will diverge to the point that we can start calling them different things. I don't particul

I wonder if at some point G(raphics)PUs and G(^HMachine-Learning)PUs will diverge to the point that we can start calling them different things. I don't particularly buy into the idea that 3D rendering is just an application of a bunch of compute units with a rasterizer at the end. There's certainly some alpha in graphics-dedicated hardware. Hey! Maybe we can call them "video cards" again!

wmf 3y ago
AMD diverged RDNA (graphics) and CDNA (HPC/AI) already. Nvidia removed the graphics-specific hardware from Hopper although it appears to be a small fraction of the chip.
hackerlight 3y ago
Then when neural rendering becomes a thing in games we'll have come full circle
- WithinReason 3y ago
  They can coexist, you can rasterize a surface and then neural-render its appearance
Galanwe 3y ago
I think the terms GPUs (for "regular pipeline" work - direct, opengl) and GP-GPU (for "custom pipeline" work - cuda, custom built) were used when I was working in the area some 15y ago.
ribit 3y ago
Hardware rasterizers are likely to disappear before long. We still use them because they are energy and area-efficient when working with large triangles. But already achieving high primitive count is difficult with hardware rasterizers, which is why some modern rendering approaches rasterize using computer shaders (e.g. UE3 Nanite). And of course, there is ray tracing which will replace the rasterization pipelines within the next decade or so. As to ML, it absolutely needs different hardware than your traditional GPUs. The reason we associate the two is because GPUs are parallel processors and thus naturally suit the task. But as we move forward things like in-memory processing will become much more important. High-performance, energy efficient matrix multiplication also requires very different hardware layouts than your standard wide GPU SIMDs.
- thechao 3y ago
  That’s the dream of HW GPU designers everywhere. I’ve got a bunch of dead Cell and Larrabee chips laying around upstairs that’d like to disagree, though. Coverage rasterization is just one tiny piece of the GPU story; the really hard parts (software-wise) are things like varying interpolation, primitive synthesis (and varying synthesis), change-of-basis, and specific sampling and blending modes. It turns out that once you add in the HW to make “a few edge cases” faster, you end up with all of the HW you need for a complete GPU.
  - ribit 3y ago
    Sure, there are things that will always benefit from some ASIC magic (like, I don't see texturing going away any time soon). And we even get new specialized hardware, such as hardware RT. But advances in both performance and rendering approaches make some of the old fixed-function unnecessary. For example, Apple already dropped most fixed function related to blending and MS — since they serialize the fragments before shading they don't get any race conditions. And from what I understand they don't have any interpolation hardware either, it's all done in the shader. Of course, they pay for all this convenience with additional fixed-function complexity of the TBDR binner, but I think this illustrates how an algorithmic change can make certain things unnecessary. Tomorrow's real-time rendering is all about RT, point clouds, microprimitives, and all that interesting stuff, the fixed function rasterizer simply won't be as useful anymore.
  - flopsamjetsam 3y ago
    What is "change-of-basis"? My Google-fu failed me.
te_chris 3y ago
Apple’s been doing this for a while with their separate GPUs and Neural Engines, no?
- ribit 3y ago
  Apple has three hardware units for machine learning (if you disregard the regular CPU FP hardware): the neural engine (for energy efficient inference for some types of models), the AMX unit (for low-latency matmul and long vector), and the GPU (which does contain some hardware for making matmul more efficient). Apple is a bit of a unique case since all processors share the same memory hierarchy, so they can afford to keep things separately. But I wouldn’t say that Apple has anything comparable to Nvidia’s Tensor Cores: AMX probably comes closest but it’s integrated with the CPU L2 and the total throughout is much lower. Unless you want FP64, that’s is very good compared to mainstream Tensor Cores.
latchkey 3y ago
AI Accelerator (AIA) is a new term I'm seeing a lot recently.

wmf 3y ago

AMD diverged RDNA (graphics) and CDNA (HPC/AI) already. Nvidia removed the graphics-specific hardware from Hopper although it appears to be a small fraction of the chip.

hackerlight 3y ago

Then when neural rendering becomes a thing in games we'll have come full circle

WithinReason 3y ago
They can coexist, you can rasterize a surface and then neural-render its appearance

Galanwe 3y ago

I think the terms GPUs (for "regular pipeline" work - direct, opengl) and GP-GPU (for "custom pipeline" work - cuda, custom built) were used when I was working in the area some 15y ago.

ribit 3y ago

Hardware rasterizers are likely to disappear before long. We still use them because they are energy and area-efficient when working with large triangles. But already achieving high primitive count is difficult with hardware rasterizers, which is why some modern rendering approaches rasterize using computer shaders (e.g. UE3 Nanite). And of course, there is ray tracing which will replace the rasterization pipelines within the next decade or so. As to ML, it absolutely needs different hardware than your traditional GPUs. The reason we associate the two is because GPUs are parallel processors and thus naturally suit the task. But as we move forward things like in-memory processing will become much more important. High-performance, energy efficient matrix multiplication also requires very different hardware layouts than your standard wide GPU SIMDs.

thechao 3y ago
That’s the dream of HW GPU designers everywhere. I’ve got a bunch of dead Cell and Larrabee chips laying around upstairs that’d like to disagree, though. Coverage rasterization is just one tiny piece of the GPU story; the really hard parts (software-wise) are things like varying interpolation, primitive synthesis (and varying synthesis), change-of-basis, and specific sampling and blending modes. It turns out that once you add in the HW to make “a few edge cases” faster, you end up with all of the HW you need for a complete GPU.
- ribit 3y ago
  Sure, there are things that will always benefit from some ASIC magic (like, I don't see texturing going away any time soon). And we even get new specialized hardware, such as hardware RT. But advances in both performance and rendering approaches make some of the old fixed-function unnecessary. For example, Apple already dropped most fixed function related to blending and MS — since they serialize the fragments before shading they don't get any race conditions. And from what I understand they don't have any interpolation hardware either, it's all done in the shader. Of course, they pay for all this convenience with additional fixed-function complexity of the TBDR binner, but I think this illustrates how an algorithmic change can make certain things unnecessary. Tomorrow's real-time rendering is all about RT, point clouds, microprimitives, and all that interesting stuff, the fixed function rasterizer simply won't be as useful anymore.
- flopsamjetsam 3y ago
  What is "change-of-basis"? My Google-fu failed me.

te_chris 3y ago

Apple’s been doing this for a while with their separate GPUs and Neural Engines, no?

ribit 3y ago
Apple has three hardware units for machine learning (if you disregard the regular CPU FP hardware): the neural engine (for energy efficient inference for some types of models), the AMX unit (for low-latency matmul and long vector), and the GPU (which does contain some hardware for making matmul more efficient). Apple is a bit of a unique case since all processors share the same memory hierarchy, so they can afford to keep things separately. But I wouldn’t say that Apple has anything comparable to Nvidia’s Tensor Cores: AMX probably comes closest but it’s integrated with the CPU L2 and the total throughout is much lower. Unless you want FP64, that’s is very good compared to mainstream Tensor Cores.

latchkey 3y ago

AI Accelerator (AIA) is a new term I'm seeing a lot recently.