3 ms·
To clarify, Dave doesn't head the hardware efforts (I doubt he wants the management headache!). Norm Jouppi was the lead of the TPU efforts. But Dave absolute
by dgacmu 9y ago
To clarify, Dave doesn't head the hardware efforts (I doubt he wants the management headache!). Norm Jouppi was the lead of the TPU efforts. But Dave absolutely is part of it, he just doesn't have to herd the cats. :p
To the previous parent, the TPU didn't take five years - I don't know where you got that. But, in any event, I'd argue the key innovation in TPUv1 wasn't actually the design of a specific optimized processor: It was seeing the need for a year or two before anyone else in industry did!
Also - I think translation problem, what's "reticule limit"? In any event, if you're comparing what someone produced today with TPUv1, that's a bit of a weird comparison considering that the v2 is live and available in beta. :)
- baybal2 9y ago>what's "reticule limit"? The maximum area a stepper can expose at the time. Effectively the limit of how big you can make a single microchip.
- dgacmu 9y agoAh, you mean "reticle". Thanks for clarifying. (A reticule is a cute little bag -- https://upload.wikimedia.org/wikipedia/commons/thumb/8/80/Reticule.tif/lossy-page1-1200px-Reticule.tif.jpg https://upload.wikimedia.org/wikipedia/commons/thumb/8/80/Re... )
- dgacmu 9y agoBut, that said: For training, it's odd that you'd criticize the die area of the TPU when the major competitor is Volta -- 815mm^2 (!). It's pretty clear that TPUv2 and Volta both have similar aims in terms of high memory bandwidth (both use HBM), monstrous matrix multiply throughput, and a high-speed interconnect to be able to scale to larger clusters. Making a single-chip inference engine, which NovuTensor appears to be, is a very different thing. So it's better compared to TPUv1, which is also an inference engine. I can't find the die area of TPUv1 out there, but it's not a monster, as you can probably infer from its 75W TDP. It would be helpful in this discussion to be precise whether you're comparing to TPUv1, Cloud TPU (the v2), or something else, and if you're talking about inference or training. (I assume my disclaimer is already obvious - I work part time at Google Brain - and that everything I'm writing is my own opinion, etc., etc., etc.)
- deepnotderp 9y agoThe TPU is nowhere near the reticle limit, don't listen to OP. 193i immersion steppers have a reticle limit of 858mm^2, the TPU is nowhere near that.
- deepnotderp 9y agoWell, it's not like Norm Jouppi is also a very well known engineer :)