4 ms·
They absolutely do have similar cores to tensor cores, it's called matrix cores. And they have particular instructions to utilize them (MFMA). Note I'm talking
by boroboro4 2y ago
They absolutely do have similar cores to tensor cores, it's called matrix cores. And they have particular instructions to utilize them (MFMA).
Note I'm talking about DC compute chips, like MI300.
LLMs aren't memory bound in production loads, they are pretty much compute bound too, at least in prefill phase, but in practice in general too.
- almostgotcaught 2y agoYa people in these comments don't know what they're talking about (no one ever does in these threads). AMDGPU has had MMA and WMMA for a while now https://rocm.docs.amd.com/projects/rocWMMA/en/latest/what-is-rocwmma.html https://rocm.docs.amd.com/projects/rocWMMA/en/latest/what-is...