4 ms·

Can you name a better AI chip, especially in terms of TFLOPs per Watt?

by curyous 7y ago

threeseed 7y ago
You're moving the goalposts. It was about the best AI chip in general not based on some arbitrary efficiency number.
- zaroth 7y ago
  TOPS per Watt is a highly meaningful metric that matters a lot in the real world. It’s a valid definition of “best”. No one cares how many TOPS your 500watt non-redundant monster can run, because it’s not being put in an EV.
  - threeseed 7y ago
    It does matter how the best AI chip performs irrespective of efficiency. Because self driving cars are a multi-year journey and those platforms will likely become more efficient.
    - zaroth 7y ago
      Not really, because the problem is embarrassingly parallelizable, so “best” can always be obtained by stacking additional chips with greater total power draw. Remember we’re talking about systems made of multiple chips. It’s like asking “what’s the best Bitcoin mining rig?” The question is one of hashrate per joule. If someone asked if you should buy a 1000watt box that did 10TH/s versus a 200watt box that did 5TH/s, which is better? They maxed out their available power envelope (~70 watts) and the resulting performance is about 20x what they were getting from their prior nVidia platform. In other words, they will always make the chip big enough to pull ~70 watts, at the highest efficiency possible. My limited understanding of the hardware design is that once you have figured out the specific architecture you can scale up and down the power consumption almost linearly with the die size. Obviously plus or minus various hard limits (e.g. oversizing compute for the available memory bandwidth) Tesla has already been working on their next chip for over a year. This chip they just started shipping finished design 1.5 years ago. This will not be the last AP hardware they ever ship. I fully expect that as we delve further into “FSD” there will be functionality on the fringes that will need even more horsepower. It’s easy to think in the present that there’s a logical “end” where development will be “complete” but that’s almost never actually the case. So there will be a Gen4/5/6, specifically what features they will unlock is very hard for me to predict. I think this is actually the type of thing Elon sees right through like it’s obvious. Like how he sees a path to colonizing Mars which includes launching a global satellite internet company.
- deleted 7y ago
  [deleted]
mr_toad 7y ago
I doubt the chip matters nearly as much as the trained network running on it. It’s quite likely they could take the network and run it on other hardware. And the trained network doesn’t matter as much as the training pipeline, which will include hardware and software external to the cars. A neural net that you can’t update is less useful, and I highly doubt that the cars themselves are doing any learning.

Can you name a better AI chip, especially in terms of TFLOPs per Watt?

threeseed 7y ago

You're moving the goalposts. It was about the best AI chip in general not based on some arbitrary efficiency number.

zaroth 7y ago
TOPS per Watt is a highly meaningful metric that matters a lot in the real world. It’s a valid definition of “best”. No one cares how many TOPS your 500watt non-redundant monster can run, because it’s not being put in an EV.
- threeseed 7y ago
  It does matter how the best AI chip performs irrespective of efficiency. Because self driving cars are a multi-year journey and those platforms will likely become more efficient.
  - zaroth 7y ago
    Not really, because the problem is embarrassingly parallelizable, so “best” can always be obtained by stacking additional chips with greater total power draw. Remember we’re talking about systems made of multiple chips. It’s like asking “what’s the best Bitcoin mining rig?” The question is one of hashrate per joule. If someone asked if you should buy a 1000watt box that did 10TH/s versus a 200watt box that did 5TH/s, which is better? They maxed out their available power envelope (~70 watts) and the resulting performance is about 20x what they were getting from their prior nVidia platform. In other words, they will always make the chip big enough to pull ~70 watts, at the highest efficiency possible. My limited understanding of the hardware design is that once you have figured out the specific architecture you can scale up and down the power consumption almost linearly with the die size. Obviously plus or minus various hard limits (e.g. oversizing compute for the available memory bandwidth) Tesla has already been working on their next chip for over a year. This chip they just started shipping finished design 1.5 years ago. This will not be the last AP hardware they ever ship. I fully expect that as we delve further into “FSD” there will be functionality on the fringes that will need even more horsepower. It’s easy to think in the present that there’s a logical “end” where development will be “complete” but that’s almost never actually the case. So there will be a Gen4/5/6, specifically what features they will unlock is very hard for me to predict. I think this is actually the type of thing Elon sees right through like it’s obvious. Like how he sees a path to colonizing Mars which includes launching a global satellite internet company.
deleted 7y ago
[deleted]

mr_toad 7y ago

I doubt the chip matters nearly as much as the trained network running on it. It’s quite likely they could take the network and run it on other hardware. And the trained network doesn’t matter as much as the training pipeline, which will include hardware and software external to the cars. A neural net that you can’t update is less useful, and I highly doubt that the cars themselves are doing any learning.