The machines that the large ML companies are using are typically not going to run TOP500 competitively. They tend to skip the MPI stack, and weren't optimized

The machines that the large ML companies are using are typically not going to run TOP500 competitively. They tend to skip the MPI stack, and weren't optimized for LINPACK performance. When I worked on TPUs there was no interest in attempting to benchmark them using supercomputer workloads (probably a good thing).