3 ms·
GPUs are a SIMD architecture designed for 4x4 float32 matrix multiplications found in video games. But NPUs (like Googles NPU) are systolic arrays designed for
by dragontamer 3y ago
GPUs are a SIMD architecture designed for 4x4 float32 matrix multiplications found in video games.
But NPUs (like Googles NPU) are systolic arrays designed for 16x16 or even 256x256 float16 or even int8 matrix multiplications instead.
-------
NVidia builds larger matrix multiplications out of the 4x4 float16 base that a SM is designed for. After all, a 8x8 matrix multiplication is just four of the smaller 4x4 matrix multiplications.
- dmurray 3y ago> After all, a 8x8 matrix multiplication is just four of the smaller 4x4 matrix multiplications Isn't it 8 of them? That's why you can't go the other way around and use your 8x8 multiplier to efficiently multiply 4x4s in parallel.
- uxp8u61q 3y agoYes, it's eight multiplication with the naive algorithm. You can get to seven with a clever trick (Strassen's algorithm). As far as I'm aware, with these precise numbers (8x8 matrix decomposed into four 4x4) it's unknown if we can do better.
- dragontamer 3y agoYes, it is 8. I was mistaken earlier.