They use their own TPUs like described in this paper [0]. They talk about 4096-chip supercomputers so this should give you an idea about what we are talking her

They use their own TPUs like described in this paper [0]. They talk about 4096-chip supercomputers so this should give you an idea about what we are talking here. The paper is pretty fascinating stuff. They are using optical interconnects for example, which sounded like science fiction a few years ago. [0] https://arxiv.org/pdf/2304.01433.pdf https://arxiv.org/pdf/2304.01433.pdf