But, that said: For training, it's odd that you'd criticize the die area of the TPU when the major competitor is Volta -- 815mm^2 (!). It's pretty clear that

But, that said: For training, it's odd that you'd criticize the die area of the TPU when the major competitor is Volta -- 815mm^2 (!). It's pretty clear that TPUv2 and Volta both have similar aims in terms of high memory bandwidth (both use HBM), monstrous matrix multiply throughput, and a high-speed interconnect to be able to scale to larger clusters. Making a single-chip inference engine, which NovuTensor appears to be, is a very different thing. So it's better compared to TPUv1, which is also an inference engine. I can't find the die area of TPUv1 out there, but it's not a monster, as you can probably infer from its 75W TDP. It would be helpful in this discussion to be precise whether you're comparing to TPUv1, Cloud TPU (the v2), or something else, and if you're talking about inference or training. (I assume my disclaimer is already obvious - I work part time at Google Brain - and that everything I'm writing is my own opinion, etc., etc., etc.)