4 ms·

You do realize that David Patterson is heading up Google's hardware efforts, right? Hardly clueless and inexperienced... And what Chinese fabless was that?

baybal2 9y ago
>David Patterson Surely, he is a top cadre, and an accomplished academician. The number of his students serving on CTO level jobs is in double digits. The fact that Google went so far with cadres, and said to be throwing high six digit salaries even on people who came to the unit fresh out of universities clearly signifies the extend of their efforts and commitment. Their intent seem to be to bang money on it until it works, no matter the cost. >Hardly clueless and inexperienced... I never challenged the technical expertise per se. I'm saying that a generic web/dotcom/clickfarm business can't be normally turned into an engineering company of an inherently different nature, regardless of how much money will be banged on the exercise. People say that for the past 5 years, the TPU unit was running effectively, like a research institute of some kind: regular workshops with academicians, amount of research works written exceeding the amount of code, and so on. TPU unit people did their job splendidly, but Google's managerial unit that authorized the whole affair seem to me to be having hard time wrapping their mind around the question on how and what to do with it. On the other hand, Ren Wu, being originally a semi engineer, had a very clear idea what he wants from the very start: off-the-shelf i/o ip, axi bus, wide registers, sram fifo, directly register fed matrix multiplication units, and predominantly synchronous operation. Voila. No talkshops, company being turned into research institute, or six digit salary cadres what so ever. The chip might well be a one man project.
dgacmu 9y ago
To clarify, Dave doesn't head the hardware efforts (I doubt he wants the management headache!). Norm Jouppi was the lead of the TPU efforts. But Dave absolutely is part of it, he just doesn't have to herd the cats. :p To the previous parent, the TPU didn't take five years - I don't know where you got that. But, in any event, I'd argue the key innovation in TPUv1 wasn't actually the design of a specific optimized processor: It was seeing the need for a year or two before anyone else in industry did! Also - I think translation problem, what's "reticule limit"? In any event, if you're comparing what someone produced today with TPUv1, that's a bit of a weird comparison considering that the v2 is live and available in beta. :)
- baybal2 9y ago
  >what's "reticule limit"? The maximum area a stepper can expose at the time. Effectively the limit of how big you can make a single microchip.
  - dgacmu 9y ago
    Ah, you mean "reticle". Thanks for clarifying. (A reticule is a cute little bag -- https://upload.wikimedia.org/wikipedia/commons/thumb/8/80/Reticule.tif/lossy-page1-1200px-Reticule.tif.jpg https://upload.wikimedia.org/wikipedia/commons/thumb/8/80/Re... )
  - dgacmu 9y ago
    But, that said: For training, it's odd that you'd criticize the die area of the TPU when the major competitor is Volta -- 815mm^2 (!). It's pretty clear that TPUv2 and Volta both have similar aims in terms of high memory bandwidth (both use HBM), monstrous matrix multiply throughput, and a high-speed interconnect to be able to scale to larger clusters. Making a single-chip inference engine, which NovuTensor appears to be, is a very different thing. So it's better compared to TPUv1, which is also an inference engine. I can't find the die area of TPUv1 out there, but it's not a monster, as you can probably infer from its 75W TDP. It would be helpful in this discussion to be precise whether you're comparing to TPUv1, Cloud TPU (the v2), or something else, and if you're talking about inference or training. (I assume my disclaimer is already obvious - I work part time at Google Brain - and that everything I'm writing is my own opinion, etc., etc., etc.)
- deepnotderp 9y ago
  The TPU is nowhere near the reticle limit, don't listen to OP. 193i immersion steppers have a reticle limit of 858mm^2, the TPU is nowhere near that.
- deepnotderp 9y ago
  Well, it's not like Norm Jouppi is also a very well known engineer :)

baybal2 9y ago

>David Patterson Surely, he is a top cadre, and an accomplished academician. The number of his students serving on CTO level jobs is in double digits. The fact that Google went so far with cadres, and said to be throwing high six digit salaries even on people who came to the unit fresh out of universities clearly signifies the extend of their efforts and commitment. Their intent seem to be to bang money on it until it works, no matter the cost. >Hardly clueless and inexperienced... I never challenged the technical expertise per se. I'm saying that a generic web/dotcom/clickfarm business can't be normally turned into an engineering company of an inherently different nature, regardless of how much money will be banged on the exercise. People say that for the past 5 years, the TPU unit was running effectively, like a research institute of some kind: regular workshops with academicians, amount of research works written exceeding the amount of code, and so on. TPU unit people did their job splendidly, but Google's managerial unit that authorized the whole affair seem to me to be having hard time wrapping their mind around the question on how and what to do with it. On the other hand, Ren Wu, being originally a semi engineer, had a very clear idea what he wants from the very start: off-the-shelf i/o ip, axi bus, wide registers, sram fifo, directly register fed matrix multiplication units, and predominantly synchronous operation. Voila. No talkshops, company being turned into research institute, or six digit salary cadres what so ever. The chip might well be a one man project.

dgacmu 9y ago

To clarify, Dave doesn't head the hardware efforts (I doubt he wants the management headache!). Norm Jouppi was the lead of the TPU efforts. But Dave absolutely is part of it, he just doesn't have to herd the cats. :p To the previous parent, the TPU didn't take five years - I don't know where you got that. But, in any event, I'd argue the key innovation in TPUv1 wasn't actually the design of a specific optimized processor: It was seeing the need for a year or two before anyone else in industry did! Also - I think translation problem, what's "reticule limit"? In any event, if you're comparing what someone produced today with TPUv1, that's a bit of a weird comparison considering that the v2 is live and available in beta. :)

baybal2 9y ago
>what's "reticule limit"? The maximum area a stepper can expose at the time. Effectively the limit of how big you can make a single microchip.
- dgacmu 9y ago
  Ah, you mean "reticle". Thanks for clarifying. (A reticule is a cute little bag -- https://upload.wikimedia.org/wikipedia/commons/thumb/8/80/Reticule.tif/lossy-page1-1200px-Reticule.tif.jpg https://upload.wikimedia.org/wikipedia/commons/thumb/8/80/Re... )
- dgacmu 9y ago
  But, that said: For training, it's odd that you'd criticize the die area of the TPU when the major competitor is Volta -- 815mm^2 (!). It's pretty clear that TPUv2 and Volta both have similar aims in terms of high memory bandwidth (both use HBM), monstrous matrix multiply throughput, and a high-speed interconnect to be able to scale to larger clusters. Making a single-chip inference engine, which NovuTensor appears to be, is a very different thing. So it's better compared to TPUv1, which is also an inference engine. I can't find the die area of TPUv1 out there, but it's not a monster, as you can probably infer from its 75W TDP. It would be helpful in this discussion to be precise whether you're comparing to TPUv1, Cloud TPU (the v2), or something else, and if you're talking about inference or training. (I assume my disclaimer is already obvious - I work part time at Google Brain - and that everything I'm writing is my own opinion, etc., etc., etc.)
deepnotderp 9y ago
The TPU is nowhere near the reticle limit, don't listen to OP. 193i immersion steppers have a reticle limit of 858mm^2, the TPU is nowhere near that.
deepnotderp 9y ago
Well, it's not like Norm Jouppi is also a very well known engineer :)