4 ms·
How good are TPUs in comparison with state of the art Nvidia datacenter GPUs, or Groq's ASICs? Per watt, per chip, total cost, etc.? Is there any published data
by QuadrupleA 2y ago
How good are TPUs in comparison with state of the art Nvidia datacenter GPUs, or Groq's ASICs? Per watt, per chip, total cost, etc.? Is there any published data?
- jeffbee 2y agoMLPerf is a good place to start. The only problem is you don't have any verifiable information about TPU energy consumption. https://mlcommons.org/benchmarks/inference-datacenter/ https://mlcommons.org/benchmarks/inference-datacenter/
- wslh 2y agoI have some company notes from early 2024 which cannot be accurate but could help, TPU v5e [1]: not available for purchase, only through GCP, storage=5B, LLM-Model=7B, efficiency=393TFLOP. [1] https://cloud.google.com/tpu/docs/v5e https://cloud.google.com/tpu/docs/v5e