3 ms·
Is there a source for the largest privately held supercomputers? This article, for instance, claims that Tesla had a 1.8 exaflop cluster in 2021. It also mentio
by derac 3y ago
Is there a source for the largest privately held supercomputers? This article, for instance, claims that Tesla had a 1.8 exaflop cluster in 2021. It also mentions Karpathy claiming it to be the 5th largest at the time.
https://blogs.nvidia.com/blog/tesla-av-training-supercomputer-nvidia-a100-gpus/ https://blogs.nvidia.com/blog/tesla-av-training-supercompute...
- boulos 3y agoI think anyone can get on the Top 500 as long as they run a verified FP64 LINPACK result. At the time, this would have been comparing against Perlmutter: https://www.top500.org/lists/top500/2021/06/ https://www.top500.org/lists/top500/2021/06/ So 6k A100s at just under 10 TFlops of FP64 => 60 PFlops. I don't remember if they said what the interconnect was, as that can hold you back from achieving those 60-ish PFlops. It's too bad NVIDIA's blog referred to the FP16 value as "flops" in the same breath, as most people at NVIDIA know that Top500 focuses on achieved FP64 LINPACK results. I swear there was some talk of using MLperf or similar to make a Top500-for-ML/reduced precision, but can't find it now.