4 ms·
Indeed, you can strip out a whole host of things from the GPU, the framebuffer, the Z-buffer, the transform and lighting engine, instead filling it with more CU
by glitchc 2y ago
Indeed, you can strip out a whole host of things from the GPU, the framebuffer, the Z-buffer, the transform and lighting engine, instead filling it with more CUDA cores and a higher bandwidth memory controller with a larger bus, etc.
And, as it happens, that's exactly what NVidia's done with the H100: https://developer.nvidia.com/blog/nvidia-hopper-architecture-in-depth/ https://developer.nvidia.com/blog/nvidia-hopper-architecture...
It still needs to be programmable though. Can't get away from that.
- adastra22 2y agoYou don’t need general programmability for AI inference.
- glitchc 2y agoThe money's in the training, not the inference. If you look at Apple and Google, they already have their own hardware for inference in their smartphones. They don't need NVidia for that.
- mupuff1234 2y agoApple and Google use TPUs for training. https://www.cnbc.com/2024/07/29/apple-says-its-ai-models-were-trained-on-googles-custom-chips-.html https://www.cnbc.com/2024/07/29/apple-says-its-ai-models-wer...
- adastra22 2y agoYou don’t need programmability for AI teaining either.
- kolinko 2y agoYou can get away from that if you constrain it to a specific type of models (say attention based).