3 ms·

Apple's new M2 Max has a neural engine which can do 15 trillion flops. Nvidias's A100 chip (released almost 3 years ago) can do 315 trillion flops. Apple is not

Apple's new M2 Max has a neural engine which can do 15 trillion flops. Nvidias's A100 chip (released almost 3 years ago) can do 315 trillion flops. Apple is not going to close this 20x gap in a few years.

davnicwil 3y ago
right, it's a huge challenge. I think the tuning the models to the hardware piece is important, and of course there is much more incentive to do this for Apple than nvidia because of the distribution and ecosystem advantages Apple have. But also, I don't know... let's see what the curve looks like! It's only been a couple of years of these neural engines. Let's see how many flops M3 can hit this year. And then m4 the next. Again, 5 years is a long time actually when real improvement is happening. I am optimistic.
moffkalast 3y ago
> this 168x gap FTFY, remember it takes 8 of those to even load the thing. And when the average laptop has that much compute, GPT 4 will seem like Cleverbot in comparison to the state of the art.
sroussey 3y ago
At some point, they will put the models in silicon. I’m curious as to when… 5yr?
- viraptor 3y ago
  That doesn't sound likely with the current architectures. There may be some kind of specialisation, but NN is like the chip design nightmare. We can't do chips that that many crossed lines. It's going to have to keep the storage+execution engine pattern unless we have done breakthroughs. "More specialised than GPU" is the game for now.
  - zamnos 3y ago
    Not even with 3d chip-stacking?
    - viraptor 3y ago
      Well, we'll see what the future manufacturing brings, but right now we're not even at thousands of layers (as far as I know... please link if there's been more), and we'd need to be in hundreds of thousands range. Given the rate of defects also adding up and the need for some way to dissipate the heat... (almost all of that chip will be engaged while running - no chance for balancing power between systems) Yeah, still lots of challenges there. (I'm assuming the original comment meant literally putting the network as is in the purpose designed chip)
anhner 3y ago
Try comparing M2 with an actual consumer level GPU, not a supercomputer...

davnicwil 3y ago

right, it's a huge challenge. I think the tuning the models to the hardware piece is important, and of course there is much more incentive to do this for Apple than nvidia because of the distribution and ecosystem advantages Apple have. But also, I don't know... let's see what the curve looks like! It's only been a couple of years of these neural engines. Let's see how many flops M3 can hit this year. And then m4 the next. Again, 5 years is a long time actually when real improvement is happening. I am optimistic.

moffkalast 3y ago

> this 168x gap FTFY, remember it takes 8 of those to even load the thing. And when the average laptop has that much compute, GPT 4 will seem like Cleverbot in comparison to the state of the art.

sroussey 3y ago

At some point, they will put the models in silicon. I’m curious as to when… 5yr?

viraptor 3y ago
That doesn't sound likely with the current architectures. There may be some kind of specialisation, but NN is like the chip design nightmare. We can't do chips that that many crossed lines. It's going to have to keep the storage+execution engine pattern unless we have done breakthroughs. "More specialised than GPU" is the game for now.
- zamnos 3y ago
  Not even with 3d chip-stacking?
  - viraptor 3y ago
    Well, we'll see what the future manufacturing brings, but right now we're not even at thousands of layers (as far as I know... please link if there's been more), and we'd need to be in hundreds of thousands range. Given the rate of defects also adding up and the need for some way to dissipate the heat... (almost all of that chip will be engaged while running - no chance for balancing power between systems) Yeah, still lots of challenges there. (I'm assuming the original comment meant literally putting the network as is in the purpose designed chip)

anhner 3y ago

Try comparing M2 with an actual consumer level GPU, not a supercomputer...