5 ms·

I'm curious about the addition of Tensor Cores. It sounds like both the regular Shader Units and Tensor Cores can carry out GEMM, just that the latter are more

I'm curious about the addition of Tensor Cores. It sounds like both the regular Shader Units and Tensor Cores can carry out GEMM, just that the latter are more efficient at it. I'm guessing that for ML applications, both the Tensor Cores and Shader Units are used. Are tensor cores idle during 3D gaming? How do they differ from the regular Shader Units? Why are they more efficient?

frogblast 6y ago
The tensor cores are basically more efficient because they don't implement a 'read from register file; execute one instruction; write to register file'-style architecture: They are basically a big array of ALUs hard-wired in a grid to do a matrix multiply: The connectivity between ALUs is baked into silicon: no general purpose register file to hold the intermediate results, just wires directly from the output of one ALU into the input of the next (well, and registers for pipelining). No overhead for register files, instruction decoding, instruction scheduling, load/store units, cache, etc. (load/store is handled by the usual SM instruction set). This is basically the same concept you find in any "AI acceleration" hardware (Google's TPUs, various neural-thingys you find in Phones, etc). As for using in games, this hardware is generally idle, except for a few cases (typically because Nvidia did the work to light them up): - upscaling from lower resolution rendered products to higher resolution (Nvidia markets this under the name "Deep Learning Super Sampling"), which is basically taking a derivation Temporal Anti-aliasing and mapping it onto the tensor cores to exploit that hardware - Doing CNN-like denoising and reconstruction of a noisy ray traced image.
hesk 6y ago
> Are tensor cores idle during 3D gaming? The tensor cores are used for real-time raytracing, ML-based anti-aliasing during gaming, and other features. The Turing whitepaper has a lot of information on that. https://www.nvidia.com/content/dam/en-zz/Solutions/design-visualization/technologies/turing-architecture/NVIDIA-Turing-Architecture-Whitepaper.pdf https://www.nvidia.com/content/dam/en-zz/Solutions/design-vi... A more generic answer to this question is that everything NVIDIA puts in their GPUs is for the gaming market. Other use cases are incidental.
- Scene_Cast2 6y ago
  I think the ray tracing cores may be separate, they accelerate BVH traversal and intersection.
  - mastax 6y ago
    The tensor cores are used to denoise the raytraced result.
- Causality1 6y ago
  Between the 30% framerate hit and the (to me) barely-perceptible visual difference, real-time raytracing still feels like a gimmick.
  - mrguyorama 6y ago
    The new Minecraft stuff is the very first time I've seen raytracing produce graphics that make my brain conceive this could be a "realistically lit" scene, and it doesn't even take advantage of things like significant amounts of reflection and re-reflection. HOWEVER, I see zero benefit to the actual game from that.
    - kevingadd 6y ago
      In the case of Minecraft no new graphics feature is really going to benefit the "actual game", but players who obsess over their created spaces might really appreciate having better lighting that they can control through things like reflectivity. It's something you'd have to poll the average teen-who-still-plays-Minecraft about though.
    - Causality1 6y ago
      Have you tried the many advanced shader packs that've been available for years and don't need raytracing? https://youtu.be/JJeb1eEZYhI https://youtu.be/JJeb1eEZYhI
  - kevingadd 6y ago
    If you haven't seen any screenshots of it at work in Remedy's Control, it's pretty nice there. Nothing groundbreaking but it's the first time I went 'oh!' and realized that I was seeing stuff that would've been invisible without raytracing. It's used for shadowing, reflections on glass, and a bunch of other stuff that ties the scene together and ensures things are where you expect them to be. I expect we'll see some more compelling uses of it in the next few years, given that it took until the last year or so to see games start using it in a way that wasn't just tacked on. (It makes sense that Control was one of the first titles to use it extensively, since they were publishing talks about GPU raytracing well before the hardware was in anyone's hands)
corysama 6y ago
> Are tensor cores idle during 3D gaming? Nv is making a use for them via https://en.wikipedia.org/wiki/Deep_Learning_Super_Sampling https://en.wikipedia.org/wiki/Deep_Learning_Super_Sampling > How do they differ from the regular Shader Units? Why are they more efficient? They basically implement a special-purpose, single-instruction 4x4 matrix multiply operation with 16-bit floats. https://www.anandtech.com/show/12673/titan-v-deep-learning-deep-dive/3 https://www.anandtech.com/show/12673/titan-v-deep-learning-d...

frogblast 6y ago

The tensor cores are basically more efficient because they don't implement a 'read from register file; execute one instruction; write to register file'-style architecture: They are basically a big array of ALUs hard-wired in a grid to do a matrix multiply: The connectivity between ALUs is baked into silicon: no general purpose register file to hold the intermediate results, just wires directly from the output of one ALU into the input of the next (well, and registers for pipelining). No overhead for register files, instruction decoding, instruction scheduling, load/store units, cache, etc. (load/store is handled by the usual SM instruction set). This is basically the same concept you find in any "AI acceleration" hardware (Google's TPUs, various neural-thingys you find in Phones, etc). As for using in games, this hardware is generally idle, except for a few cases (typically because Nvidia did the work to light them up): - upscaling from lower resolution rendered products to higher resolution (Nvidia markets this under the name "Deep Learning Super Sampling"), which is basically taking a derivation Temporal Anti-aliasing and mapping it onto the tensor cores to exploit that hardware - Doing CNN-like denoising and reconstruction of a noisy ray traced image.

hesk 6y ago

> Are tensor cores idle during 3D gaming? The tensor cores are used for real-time raytracing, ML-based anti-aliasing during gaming, and other features. The Turing whitepaper has a lot of information on that. https://www.nvidia.com/content/dam/en-zz/Solutions/design-visualization/technologies/turing-architecture/NVIDIA-Turing-Architecture-Whitepaper.pdf https://www.nvidia.com/content/dam/en-zz/Solutions/design-vi... A more generic answer to this question is that everything NVIDIA puts in their GPUs is for the gaming market. Other use cases are incidental.

Scene_Cast2 6y ago
I think the ray tracing cores may be separate, they accelerate BVH traversal and intersection.
- mastax 6y ago
  The tensor cores are used to denoise the raytraced result.
Causality1 6y ago
Between the 30% framerate hit and the (to me) barely-perceptible visual difference, real-time raytracing still feels like a gimmick.
- mrguyorama 6y ago
  The new Minecraft stuff is the very first time I've seen raytracing produce graphics that make my brain conceive this could be a "realistically lit" scene, and it doesn't even take advantage of things like significant amounts of reflection and re-reflection. HOWEVER, I see zero benefit to the actual game from that.
  - kevingadd 6y ago
    In the case of Minecraft no new graphics feature is really going to benefit the "actual game", but players who obsess over their created spaces might really appreciate having better lighting that they can control through things like reflectivity. It's something you'd have to poll the average teen-who-still-plays-Minecraft about though.
  - Causality1 6y ago
    Have you tried the many advanced shader packs that've been available for years and don't need raytracing? https://youtu.be/JJeb1eEZYhI https://youtu.be/JJeb1eEZYhI
- kevingadd 6y ago
  If you haven't seen any screenshots of it at work in Remedy's Control, it's pretty nice there. Nothing groundbreaking but it's the first time I went 'oh!' and realized that I was seeing stuff that would've been invisible without raytracing. It's used for shadowing, reflections on glass, and a bunch of other stuff that ties the scene together and ensures things are where you expect them to be. I expect we'll see some more compelling uses of it in the next few years, given that it took until the last year or so to see games start using it in a way that wasn't just tacked on. (It makes sense that Control was one of the first titles to use it extensively, since they were publishing talks about GPU raytracing well before the hardware was in anyone's hands)

corysama 6y ago

> Are tensor cores idle during 3D gaming? Nv is making a use for them via https://en.wikipedia.org/wiki/Deep_Learning_Super_Sampling https://en.wikipedia.org/wiki/Deep_Learning_Super_Sampling > How do they differ from the regular Shader Units? Why are they more efficient? They basically implement a special-purpose, single-instruction 4x4 matrix multiply operation with 16-bit floats. https://www.anandtech.com/show/12673/titan-v-deep-learning-deep-dive/3 https://www.anandtech.com/show/12673/titan-v-deep-learning-d...