3 ms·

It is entirely possible that Moore's Law gets assassinated by supply chain destruction as deglobalization continues. There are too many single source suppliers

It is entirely possible that Moore's Law gets assassinated by supply chain destruction as deglobalization continues. There are too many single source suppliers in the chain up to EUV lithography. We may in fact be at peak IC.

HarHarVeryFunny 4y ago
People have been predicting the end of Moore's law for decades ... eventually they will be right, but it seems there are at least a few generations of process reduction and transistor design left before we start to hit any hard physical limits. Of course there hasn't been much increase in single core clock speeds or cost reduction for a while now (certainly nothing from Intel), but TSMC and Samsung do continue to make progress, with multiple future process generations under development. As far as AI, or more specifically training (and running) massive neural nets, it's not single core performance that matters at all - it's all about specialized matrix multiplication hardware whether NVidia's Tensor Cores or Google's TPUs, and more importantly the ability to bring many of these specialized units together to work on a single problem. Distributed NN training has been a solved problem for a long time, with things like GPT-4 being trained across 10,000 machines. This type of massive parallelism can continue to grow regardless of what is happening to Moore's law and single core performance.
- mikewarot 4y ago
  I'm not suggesting a slowdown of innovation, but rather supply chain collapse which is a completely different kettle of fish. Our ability to make high end chips may plato due to a complete inability of AMSL to source all the parts for EUV steppers. Many of those components themselves have multi-national supply chains.
  - HarHarVeryFunny 4y ago
    Right, but what I was trying to say is that we're not really that dependent on cutting edge chips, at least not for AI development. Even if all we had was older pre-EUV fabs we could still train in parallel on as many less powerful chips as needed. I'm surprised that the US didn't consider tech self-reliance as a national security issue a long time ago.
    - mikewarot 4y ago
      Makes sense. I think that there are likely to be quite a few optimizations of training algorithms to be had as well.

HarHarVeryFunny 4y ago

People have been predicting the end of Moore's law for decades ... eventually they will be right, but it seems there are at least a few generations of process reduction and transistor design left before we start to hit any hard physical limits. Of course there hasn't been much increase in single core clock speeds or cost reduction for a while now (certainly nothing from Intel), but TSMC and Samsung do continue to make progress, with multiple future process generations under development. As far as AI, or more specifically training (and running) massive neural nets, it's not single core performance that matters at all - it's all about specialized matrix multiplication hardware whether NVidia's Tensor Cores or Google's TPUs, and more importantly the ability to bring many of these specialized units together to work on a single problem. Distributed NN training has been a solved problem for a long time, with things like GPT-4 being trained across 10,000 machines. This type of massive parallelism can continue to grow regardless of what is happening to Moore's law and single core performance.

mikewarot 4y ago
I'm not suggesting a slowdown of innovation, but rather supply chain collapse which is a completely different kettle of fish. Our ability to make high end chips may plato due to a complete inability of AMSL to source all the parts for EUV steppers. Many of those components themselves have multi-national supply chains.
- HarHarVeryFunny 4y ago
  Right, but what I was trying to say is that we're not really that dependent on cutting edge chips, at least not for AI development. Even if all we had was older pre-EUV fabs we could still train in parallel on as many less powerful chips as needed. I'm surprised that the US didn't consider tech self-reliance as a national security issue a long time ago.
  - mikewarot 4y ago
    Makes sense. I think that there are likely to be quite a few optimizations of training algorithms to be had as well.