3 ms·
Lots of memory bandwidth, low latency high bandwidth connections between nodes, lots of floating point. The MI300 looking a bit like HPC being repurposed for AI
by JonChesterfield 2y ago
Lots of memory bandwidth, low latency high bandwidth connections between nodes, lots of floating point. The MI300 looking a bit like HPC being repurposed for AI is probably a win in terms of using the hardware for other things in the future.
I've had to double check the low precision point qualifier. I expected MI300X to run f32 and f64 at the same speed (and it does) and nvidia to have slower f64 (which it does). It looks like H100 is 250TF at half, 60TF at float, 30TF at double. MI300X is 650TF at half, 80TF at float or double. Very fast F16 indeed.
(sourced from https://www.techpowerup.com/gpu-specs/radeon-instinct-mi300x.c4179 https://www.techpowerup.com/gpu-specs/radeon-instinct-mi300x... and https://www.techpowerup.com/gpu-specs/h100-sxm5-96-gb.c3974 https://www.techpowerup.com/gpu-specs/h100-sxm5-96-gb.c3974, I might have the wrong SKUs for comparison)