3 ms·
>HOW NVIDIA GPUs process stuff? (Inefficiency 101) Wow. Massively ignorant take. A modern GPUs is an amazing feat of engineering, particularly about making com
by moralestapia 8mo ago
>HOW NVIDIA GPUs process stuff? (Inefficiency 101)
Wow. Massively ignorant take. A modern GPUs is an amazing feat of engineering, particularly about making computation more efficient (low power/high throughput).
Then proceeds to explain, wrongly, how inference is supposssedly implemented and draws conclusions from there ...
- beAroundHere 8mo agoHey, Can you please point out explain the inaccuracies in the article? I had written this post to have a higher level understanding of traditional vs Taalas's inference. So it does abstracts lots of things.
- wmf 8mo agoArguably DRAM-based GPUs/TPUs are quite inefficient for inference compared to SRAM-based Groq/Cerebras. GPUs are highly optimized but they still lose to different architectures that are better suited for inference.
- imtringued 8mo agoThe way modern Nvidia GPUs perform inference is that they have a processor (tensor memory accelerator) that directly performs tensor memory operations which directly concedes that GPGPU as a paradigm is too inefficient for matrix multiplication.