TPUs don't have enough memory either, but they have really great interconnects, so they can build a nice high density cluster. Compare the photos of a Cerebras

TPUs don't have enough memory either, but they have really great interconnects, so they can build a nice high density cluster. Compare the photos of a Cerebras deployment to a TPU deployment. https://www.nextplatform.com/wp-content/uploads/2023/07/cerebras-g42-cg1-system-logo.jpg https://www.nextplatform.com/wp-content/uploads/2023/07/cere... https://assets.bwbx.io/images/users/iqjWHBFdfxIU/iOLs2FEQxQvM/v0/-1x-1.webp https://assets.bwbx.io/images/users/iqjWHBFdfxIU/iOLs2FEQxQv... The difference is striking.