Quoting myself on twitter: https://twitter.com/cHHillee/status/1577713102434361344 https://twitter.com/cHHillee/status/1577713102434361344 I'm quite suspiciou

Quoting myself on twitter: https://twitter.com/cHHillee/status/1577713102434361344 https://twitter.com/cHHillee/status/1577713102434361344 I'm quite suspicious about their hardware benchmarks. They're not writing custom kernels, they're relying on a graph compiler like XLA to automatically fuse their decomposed matmuls (and my guess is that XLA will not be very good at this). Moreover, as far as I can tell, they don't report absolute performance numbers anywhere. In other words, I suspect that a naive N^3 matrix multiplication would absolutely smoke them in performance.