4 ms·

I'm curious: Do you think there is a realistic chance for another chip maker to catch up and overtake NVidia in the AI space in the next few years or is their l

I'm curious: Do you think there is a realistic chance for another chip maker to catch up and overtake NVidia in the AI space in the next few years or is their lead and expertise insurmountable at this point?

chaxor 3y ago
I don't think that type of question or logic applies when predicting stock markets.
- edgyquant 3y ago
  Luckily no one is trying to predict a stock market here
latchkey 3y ago
AMD is trying. https://seekingalpha.com/article/4650521-amd-set-to-deliver-a-strong-product-to-ai-market-with-mi300 https://seekingalpha.com/article/4650521-amd-set-to-deliver-...
dhruvdh 3y ago
This is launched in response to MI300X, and this should still not be enough to match AMD's product. This launches 2 quarters after MI300X, but B100 should arrive before AMD's MI400 generation.
- MikeKusold 3y ago
  I thought CUDA was NVIDIA’s moat. Is that no longer the case, or did AMD come up with a good alternative?
  - zozbot234 3y ago
    CUDA code can be forward-ported to AMD's HIP, which can be used with the ROCm stack. For a more standards-focused alternative there's also SYCL, which has implementations targeting a variety of hardware backends (including HIP) and may also target Vulkan Compute in the future.
    - meragrin_ 3y ago
      > CUDA code can be forward-ported to AMD's HIP, which can be used with the ROCm stack. Maybe in some cases, but that doesn't even really matter since hardware support is poor.
  - PartiallyTyped 3y ago
    StableHLO[1] and IREE[2] are interesting projects that might help AMD here, from [1]: > Our goal is to simplify and accelerate ML development by creating more interoperability between various ML frameworks (such as TensorFlow, JAX and PyTorch) and ML compilers (such as XLA and IREE). From there, their goal would most likely be to work with XLA/OpenXLA teams on XLA[3] and IREE[2] to make RoCM a better backend. [1] https://github.com/openxla/stablehlo https://github.com/openxla/stablehlo [2] https://github.com/openxla/iree https://github.com/openxla/iree [3] https://www.tensorflow.org/xla https://www.tensorflow.org/xla
  - michaelt 3y ago
    The vast majority of work in ML isn't people working with CUDA directly - people use open source frameworks like PyTorch and TensorFlow to define a network and train it, and all the frameworks support CUDA as a backend. Other backends are also available, such as CPU-only training. And you can export networks in reasonably-standard formats. nvidia's moat is much more mature framework support than AMD's cards; widespread popularity due to that good framework support, ensuring everyone develops on nvidia, thus maintaining their support lead; much faster performance than CPU-only training; and a price that, though high, is a lot less than an ML developer's salary. If you need 24GB of vram and nvidia offers that for $1600 while AMD offers it for $1300, how many compatibility problems do you want to deal with to save a single day's wages? But nvidia's moat is far from guaranteed. Huge users like OpenAI and Facebook might find improving AMD support pays for itself.
    - RcouF1uZ4gsC 3y ago
      > Huge users like OpenAI and Facebook might find improving AMD support pays for itself. At that scale they may actually develop their own hardware a la Google TPU. If you want to just focus on the AI problem and not on infrastructure, just use NVidia. If you want control and efficiency, design your own. AMD kind of falls in a weird middle ground with respect to the massive companies.
      - ls612 3y ago
        Google’s focus on TPUs have caused them many issues though other giant players might see spending FTE-equivalents on PyTorch etc development as a better investment choice.
- Vvector 3y ago
  AMD always launches impressive hardware specs. But they are way behind in software, which is more important than hardware
  - PartiallyTyped 3y ago
    StableHLO[1] and IREE[2] are interesting projects that might help AMD here, from [1]: > Our goal is to simplify and accelerate ML development by creating more interoperability between various ML frameworks (such as TensorFlow, JAX and PyTorch) and ML compilers (such as XLA and IREE). From there, their goal would most likely be to work with XLA/OpenXLA teams on XLA[3] and IREE[2] to make RoCM a better backend. [1] https://github.com/openxla/stablehlo https://github.com/openxla/stablehlo [2] https://github.com/openxla/iree https://github.com/openxla/iree [3] https://www.tensorflow.org/xla https://www.tensorflow.org/xla
  - stravant 3y ago
    If they were allowed to get significantly ahead that status quo would likely be disrupted pretty fast.
  - photonbucket 3y ago
    If AMD launches hardware that is clearly faster, the software will move towards it
    - pixelpoet 3y ago
      That's exactly what the CUDA monopoly is meant to prevent, and as a fervent supporter of OpenCL (with two commercial apps), this is exactly the case I always make: even if some GPU came out tomorrow costing $0 and with infinite performance, all these people who paint themselves into a corner are hosed. Not that anyone cares, and everyone keeps using CUDA while simultaneously complaining about Nvidia GPU prices, as if those two things have nothing to do with each other...
      - zozbot234 3y ago
        My own understanding is that OpenCL is semi-obsolete at the moment (although newer standards revisions are still coming out, so this may change in the future) with forward-looking projects mostly targeting Vulkan Compute or SYCL. (There are some annoying differences in the low-level implementations of OpenCL vs. Vulkan Compute, due to their being based on SPIR-V compute "kernels" vs. "shaders" respectively, that make it hard for them to interop cleanly. So that's why the choice can be significant.)
xnx 3y ago
Google's TPUs are competitive, but can only be rented.
- synergy20 3y ago
  based on MLPerf, google is further lagging behind, Intel is the only that is catching up a bit, but still. AMD is trying to catch up too, so far Nvidia still remains to be the leader, a few years ahead.
  - eklitzke 3y ago
    Behind in what dimension? The most expensive Nvidia chips are much faster than Google TPUs, but the Google TPUs are competitive in terms of end to end training costs (roughly you can think of this as FLOPs per dollar).
    - jakderrida 3y ago
      I use TPUs on Colab all the time and I'm freaking happy. Maybe there's a way to use H100's do the same thing, but my code is already written to host utterly massive files on gcs buckets as tfrecord files and load them up on TPUs during the training process. First started with the free ones and now I rent the newer ones because it's not that expensive. I recommend beginners try it out. I find the other architectures more expensive, at least for my use case.
  - brucethemoose2 3y ago
    Anandtech had a good saying. There are no bad products, just bad prices. So if TPU clusters are priced right...
Mistletoe 3y ago
>I'm curious: Do you think there is a realistic chance for another computer maker to catch up and overtake IBM in the computer space in the next few years or is their lead and expertise insurmountable at this point? Nothing is insurmountable. :) https://en.wikipedia.org/wiki/The_Innovator's_Dilemma https://en.wikipedia.org/wiki/The_Innovator's_Dilemma >It describes how large incumbent companies lose market share by listening to their customers and providing what appears to be the highest-value products, but new companies that serve low-value customers with poorly developed technology can improve that technology incrementally until it is good enough to quickly take market share from established business.
- wolframhempel 3y ago
  I'm aware, but I'd argue that in addition to the competitive moats described by Hamilton Helmer (7 powers guy) there is a real moat in unique technological expertise in the chip industry. E.g. the chip making machines that ASML makes or the 3nm chips that TSMC produces have reached a level of sophistication that will take 3+ years for competitors to replicate, thus granting them a sort of quasi monopoly for the foreseeable future.
  - huijzer 3y ago
    I‘s say those are covered by Helmer with scale power. Spending millions on tiny process optimizations or other research is possible due to the large revenue streams that are coming in. For example, in terms of scale economies, only when you sell thousands high end GPUs per month you can hire people to write highly optimized compilers.
    - wolframhempel 3y ago
      Maybe - though I think it's more of a cornered resource. You have specialized knowledge that attracts specialized people which create more specialized knowledge. Creates a sorta feedback loop.
revel 3y ago
Although no lead is insurmountable, the fixed capital investment and mature software ecosystem specific to this sector makes it harder to imagine what a competitor would look like. Given how large the prize is, the next chapter of chip development is likely to be nvidia vs state sponsored projects. China, in particular, will funnel further resources into acquiring this technology by any means necessary, including (more) industrial sabotage and outright theft. It's going to be interesting to see how this will play out. Up until a few years ago China was viewed as being a formidable competitor for projects of this nature, but as the country has moved to become increasingly authoritarian, so too have its decision making and execution declined in quality.
brucethemoose2 3y ago
Maybe not overtake, but Microsoft and Amazon are going to eat some of the pie by pushing their own accelerators inside their ecosystems.
singularity2001 3y ago
photonics can run light based matrix multiplication for a fraction of the current GPUs, it's only a matter of time until they initiate a complete paradigm shift
- Loggias 3y ago
  [dead]
- jakderrida 3y ago
  I keep reading of these upcoming photonic hardware architectures capable of 100x+ increases. I sure as hell hope they hit market soon.
  - singularity2001 3y ago
    last time I checked the biggest players only did inference architectures, not training. presumably they shy away from implementing Hinton's forward forward architecture or something similar
1905 3y ago
Maybe one of the big Chinese chip makers or AMD, but the growing popularity of CUDA (not compatible with other GPUs) makes that less likely in the near future

chaxor 3y ago

I don't think that type of question or logic applies when predicting stock markets.

edgyquant 3y ago
Luckily no one is trying to predict a stock market here

latchkey 3y ago

AMD is trying. https://seekingalpha.com/article/4650521-amd-set-to-deliver-a-strong-product-to-ai-market-with-mi300 https://seekingalpha.com/article/4650521-amd-set-to-deliver-...

dhruvdh 3y ago

This is launched in response to MI300X, and this should still not be enough to match AMD's product. This launches 2 quarters after MI300X, but B100 should arrive before AMD's MI400 generation.

MikeKusold 3y ago
I thought CUDA was NVIDIA’s moat. Is that no longer the case, or did AMD come up with a good alternative?
- zozbot234 3y ago
  CUDA code can be forward-ported to AMD's HIP, which can be used with the ROCm stack. For a more standards-focused alternative there's also SYCL, which has implementations targeting a variety of hardware backends (including HIP) and may also target Vulkan Compute in the future.
  - meragrin_ 3y ago
    > CUDA code can be forward-ported to AMD's HIP, which can be used with the ROCm stack. Maybe in some cases, but that doesn't even really matter since hardware support is poor.
- PartiallyTyped 3y ago
  StableHLO[1] and IREE[2] are interesting projects that might help AMD here, from [1]: > Our goal is to simplify and accelerate ML development by creating more interoperability between various ML frameworks (such as TensorFlow, JAX and PyTorch) and ML compilers (such as XLA and IREE). From there, their goal would most likely be to work with XLA/OpenXLA teams on XLA[3] and IREE[2] to make RoCM a better backend. [1] https://github.com/openxla/stablehlo https://github.com/openxla/stablehlo [2] https://github.com/openxla/iree https://github.com/openxla/iree [3] https://www.tensorflow.org/xla https://www.tensorflow.org/xla
- michaelt 3y ago
  The vast majority of work in ML isn't people working with CUDA directly - people use open source frameworks like PyTorch and TensorFlow to define a network and train it, and all the frameworks support CUDA as a backend. Other backends are also available, such as CPU-only training. And you can export networks in reasonably-standard formats. nvidia's moat is much more mature framework support than AMD's cards; widespread popularity due to that good framework support, ensuring everyone develops on nvidia, thus maintaining their support lead; much faster performance than CPU-only training; and a price that, though high, is a lot less than an ML developer's salary. If you need 24GB of vram and nvidia offers that for $1600 while AMD offers it for $1300, how many compatibility problems do you want to deal with to save a single day's wages? But nvidia's moat is far from guaranteed. Huge users like OpenAI and Facebook might find improving AMD support pays for itself.
  - RcouF1uZ4gsC 3y ago
    > Huge users like OpenAI and Facebook might find improving AMD support pays for itself. At that scale they may actually develop their own hardware a la Google TPU. If you want to just focus on the AI problem and not on infrastructure, just use NVidia. If you want control and efficiency, design your own. AMD kind of falls in a weird middle ground with respect to the massive companies.
    - ls612 3y ago
      Google’s focus on TPUs have caused them many issues though other giant players might see spending FTE-equivalents on PyTorch etc development as a better investment choice.
Vvector 3y ago
AMD always launches impressive hardware specs. But they are way behind in software, which is more important than hardware
- PartiallyTyped 3y ago
  StableHLO[1] and IREE[2] are interesting projects that might help AMD here, from [1]: > Our goal is to simplify and accelerate ML development by creating more interoperability between various ML frameworks (such as TensorFlow, JAX and PyTorch) and ML compilers (such as XLA and IREE). From there, their goal would most likely be to work with XLA/OpenXLA teams on XLA[3] and IREE[2] to make RoCM a better backend. [1] https://github.com/openxla/stablehlo https://github.com/openxla/stablehlo [2] https://github.com/openxla/iree https://github.com/openxla/iree [3] https://www.tensorflow.org/xla https://www.tensorflow.org/xla
- stravant 3y ago
  If they were allowed to get significantly ahead that status quo would likely be disrupted pretty fast.
- photonbucket 3y ago
  If AMD launches hardware that is clearly faster, the software will move towards it
  - pixelpoet 3y ago
    That's exactly what the CUDA monopoly is meant to prevent, and as a fervent supporter of OpenCL (with two commercial apps), this is exactly the case I always make: even if some GPU came out tomorrow costing $0 and with infinite performance, all these people who paint themselves into a corner are hosed. Not that anyone cares, and everyone keeps using CUDA while simultaneously complaining about Nvidia GPU prices, as if those two things have nothing to do with each other...
    - zozbot234 3y ago
      My own understanding is that OpenCL is semi-obsolete at the moment (although newer standards revisions are still coming out, so this may change in the future) with forward-looking projects mostly targeting Vulkan Compute or SYCL. (There are some annoying differences in the low-level implementations of OpenCL vs. Vulkan Compute, due to their being based on SPIR-V compute "kernels" vs. "shaders" respectively, that make it hard for them to interop cleanly. So that's why the choice can be significant.)

xnx 3y ago

Google's TPUs are competitive, but can only be rented.

synergy20 3y ago
based on MLPerf, google is further lagging behind, Intel is the only that is catching up a bit, but still. AMD is trying to catch up too, so far Nvidia still remains to be the leader, a few years ahead.
- eklitzke 3y ago
  Behind in what dimension? The most expensive Nvidia chips are much faster than Google TPUs, but the Google TPUs are competitive in terms of end to end training costs (roughly you can think of this as FLOPs per dollar).
  - jakderrida 3y ago
    I use TPUs on Colab all the time and I'm freaking happy. Maybe there's a way to use H100's do the same thing, but my code is already written to host utterly massive files on gcs buckets as tfrecord files and load them up on TPUs during the training process. First started with the free ones and now I rent the newer ones because it's not that expensive. I recommend beginners try it out. I find the other architectures more expensive, at least for my use case.
- brucethemoose2 3y ago
  Anandtech had a good saying. There are no bad products, just bad prices. So if TPU clusters are priced right...

Mistletoe 3y ago

>I'm curious: Do you think there is a realistic chance for another computer maker to catch up and overtake IBM in the computer space in the next few years or is their lead and expertise insurmountable at this point? Nothing is insurmountable. :) https://en.wikipedia.org/wiki/The_Innovator's_Dilemma https://en.wikipedia.org/wiki/The_Innovator's_Dilemma >It describes how large incumbent companies lose market share by listening to their customers and providing what appears to be the highest-value products, but new companies that serve low-value customers with poorly developed technology can improve that technology incrementally until it is good enough to quickly take market share from established business.

wolframhempel 3y ago
I'm aware, but I'd argue that in addition to the competitive moats described by Hamilton Helmer (7 powers guy) there is a real moat in unique technological expertise in the chip industry. E.g. the chip making machines that ASML makes or the 3nm chips that TSMC produces have reached a level of sophistication that will take 3+ years for competitors to replicate, thus granting them a sort of quasi monopoly for the foreseeable future.
- huijzer 3y ago
  I‘s say those are covered by Helmer with scale power. Spending millions on tiny process optimizations or other research is possible due to the large revenue streams that are coming in. For example, in terms of scale economies, only when you sell thousands high end GPUs per month you can hire people to write highly optimized compilers.
  - wolframhempel 3y ago
    Maybe - though I think it's more of a cornered resource. You have specialized knowledge that attracts specialized people which create more specialized knowledge. Creates a sorta feedback loop.

revel 3y ago

Although no lead is insurmountable, the fixed capital investment and mature software ecosystem specific to this sector makes it harder to imagine what a competitor would look like. Given how large the prize is, the next chapter of chip development is likely to be nvidia vs state sponsored projects. China, in particular, will funnel further resources into acquiring this technology by any means necessary, including (more) industrial sabotage and outright theft. It's going to be interesting to see how this will play out. Up until a few years ago China was viewed as being a formidable competitor for projects of this nature, but as the country has moved to become increasingly authoritarian, so too have its decision making and execution declined in quality.

brucethemoose2 3y ago

Maybe not overtake, but Microsoft and Amazon are going to eat some of the pie by pushing their own accelerators inside their ecosystems.

singularity2001 3y ago

photonics can run light based matrix multiplication for a fraction of the current GPUs, it's only a matter of time until they initiate a complete paradigm shift

Loggias 3y ago
[dead]
jakderrida 3y ago
I keep reading of these upcoming photonic hardware architectures capable of 100x+ increases. I sure as hell hope they hit market soon.
- singularity2001 3y ago
  last time I checked the biggest players only did inference architectures, not training. presumably they shy away from implementing Hinton's forward forward architecture or something similar

1905 3y ago

Maybe one of the big Chinese chip makers or AMD, but the growing popularity of CUDA (not compatible with other GPUs) makes that less likely in the near future