5 ms·

But the chip is _not_ simple. You can't get 15TFLOPS with a "simple" chip. Even TPU1 is not simple and it took Google over 2 years to get that from the drawing

by 0xbear 9y ago

But the chip is _not_ simple. You can't get 15TFLOPS with a "simple" chip. Even TPU1 is not simple and it took Google over 2 years to get that from the drawing board to the data centers. Moreover, 15TFLOPS, in the unlikely event that they design and build it, is not going to be 5W. Unless they know something the likes of Intel, NVIDIA etc don't know. The probability of that is not just vanishingly small, it's zero.

deepnotderp 9y ago
The chip, from a design perspective, can absolutely be simple and get 15 TFLOPs. And in 5W is totally possible (in fact, greater efficiency is possible), especially if you only restrict yourself to inference with 3x3 convolutions. And it's not at all impossible that they know something that Intel/nVidia doesn't, that's how innovation from startups work ;) And as to their power efficiency number, it's not just possible, but it's possible to do even better than that, as we have. And these processors are definitely simpler than say, an OoO CPU. Note that I mean this from a design perspective, from an innovation perspective, it's your call as to whether or not it's "simple".
- 0xbear 9y ago
  15TFLOPS at 1Ghz is 15KFLOPS per cycle, every cycle, pipelined to amortize latency. And it's also sufficient cache and memory bandwidth to funnel data in and out of that thing at that speed, and keep the weight matrix close to the computation. This physically can't be simple. If my calculations are right, in order to realize 15TFLOPS on a 3x3 conv of a 224x224x32 layer with a filter bank of 64 filters you'd need 117.2GB/s of sustained memory bandwidth just to read the inputs, and twice that to write the output. Good luck achieving that with 5 watts and in 6 months.
  - deepnotderp 9y ago
    Uhhh, 117.2 GB/s isn't hard at all on die, getting ~10X that amount is possible even through the HBM2 bus. Instead of downvoting me, can you actually present a technical argument? Have you ever designed a chip or taped one out? And yes, physically it definitely is indeed simple (what would you consider a "complex" processor and a "simple" processor?). Plus these processors aren't using caches, there's no need for them.
    - 0xbear 9y ago
      This _is_ technical argument, and it's not me who's downvoting you. You have to keep this thing fed, and once you factor that in, "5W" and "simple chip" become physical impossibilities.
      - deepnotderp 9y ago
        Sure, it's an argument, but it's wrong. What do you consider to be a "simple" chip and a "complex" chip? From a design perspective, it absolutely is a "simple" chip. And keeping the thing fed in terms of memory bandwidth is not that difficult if done on chip. Now, I have no idea what these guys are doing, but according to our simulations, (and people we've showed in the semi industry agree with the notion) it's quite possible to do that.
        0xbear 9y ago
        It can't be "on chip": data has to get in and out somehow. It's "quite possible" to do a lot of things if you don't intend to actually do any of them.
        deepnotderp 9y ago
        1. Errr what? You mean the input data? Sure, but that isn't usually considered the power consumption. Otherwise you can totally have most of the data (in the sense where data is the weights and activation maps) reside on chip. That's the only way to get high power efficiency. 2. Well, only time can tell there :)

by 0xbear 9y ago

deepnotderp 9y ago

The chip, from a design perspective, can absolutely be simple and get 15 TFLOPs. And in 5W is totally possible (in fact, greater efficiency is possible), especially if you only restrict yourself to inference with 3x3 convolutions. And it's not at all impossible that they know something that Intel/nVidia doesn't, that's how innovation from startups work ;) And as to their power efficiency number, it's not just possible, but it's possible to do even better than that, as we have. And these processors are definitely simpler than say, an OoO CPU. Note that I mean this from a design perspective, from an innovation perspective, it's your call as to whether or not it's "simple".

0xbear 9y ago
15TFLOPS at 1Ghz is 15KFLOPS per cycle, every cycle, pipelined to amortize latency. And it's also sufficient cache and memory bandwidth to funnel data in and out of that thing at that speed, and keep the weight matrix close to the computation. This physically can't be simple. If my calculations are right, in order to realize 15TFLOPS on a 3x3 conv of a 224x224x32 layer with a filter bank of 64 filters you'd need 117.2GB/s of sustained memory bandwidth just to read the inputs, and twice that to write the output. Good luck achieving that with 5 watts and in 6 months.
- deepnotderp 9y ago
  Uhhh, 117.2 GB/s isn't hard at all on die, getting ~10X that amount is possible even through the HBM2 bus. Instead of downvoting me, can you actually present a technical argument? Have you ever designed a chip or taped one out? And yes, physically it definitely is indeed simple (what would you consider a "complex" processor and a "simple" processor?). Plus these processors aren't using caches, there's no need for them.
  - 0xbear 9y ago
    This _is_ technical argument, and it's not me who's downvoting you. You have to keep this thing fed, and once you factor that in, "5W" and "simple chip" become physical impossibilities.
    - deepnotderp 9y ago
      Sure, it's an argument, but it's wrong. What do you consider to be a "simple" chip and a "complex" chip? From a design perspective, it absolutely is a "simple" chip. And keeping the thing fed in terms of memory bandwidth is not that difficult if done on chip. Now, I have no idea what these guys are doing, but according to our simulations, (and people we've showed in the semi industry agree with the notion) it's quite possible to do that.
      - 0xbear 9y ago
        It can't be "on chip": data has to get in and out somehow. It's "quite possible" to do a lot of things if you don't intend to actually do any of them.
        deepnotderp 9y ago
        1. Errr what? You mean the input data? Sure, but that isn't usually considered the power consumption. Otherwise you can totally have most of the data (in the sense where data is the weights and activation maps) reside on chip. That's the only way to get high power efficiency. 2. Well, only time can tell there :)