https://dl.acm.org/doi/pdf/10.1145/3360307 https://dl.acm.org/doi/pdf/10.1145/3360307 Very fast (hundreds of gigabits per connection) networks that attach TPUs

https://dl.acm.org/doi/pdf/10.1145/3360307 https://dl.acm.org/doi/pdf/10.1145/3360307 Very fast (hundreds of gigabits per connection) networks that attach TPUs on the same board, as well as between boards. It's wired up point to point, forming a logical grid (technically a 2D or 3D torus with wraparound links).