3 ms·

Yes, but does it suck billions or trillions? The market is saying that it's willing to throw the latter at the overall problem of AI training, so even if it did

Yes, but does it suck billions or trillions? The market is saying that it's willing to throw the latter at the overall problem of AI training, so even if it did take a trillion dollars to fix AMD's software issues, that ought not be a significant hurdle either.

ivewonyoung 2y ago
Some things are hard and just throwing more money at it doesn't work because it's about company culture and leadership. Compare ICE auto industry profits since 2003 when Tesla was founded to money invested in Tesla till 2019 when Model Y was finished. Tesla had very less money to work with. Multiple iterations of Volkswagen's car software sucked so bad and the company culture and leadership just couldn't develop good software so Volkswagen created a separate startup style subsidiary just to make software. Guess what? That startup also failed and they recently bought a $5 billion stake in Rivian so that they could use Rivian's car software. Meanwhile their sales are tanking. Also look at SpaceX vs. ESA or Boeing/ULA with reusable boosters. Or China with EUV and chips. Even nation state level with essentially unlimited funding may not work.
- tmtvl 2y ago
  > Some things are hard and just throwing more money at it doesn't work because it's about company culture and leadership. Indeed, though that sword cuts both ways, if a hard problem gets fixed with lots of money being thrown at it then a competitor may be able to fix it for less if they have the right culture and leadership. > look at SpaceX vs. ESA or Boeing/ULA with reusable boosters. The way I understood it was that SpaceX was granted $3B of tax payer money to set up a moon mission and all they achieved was a single booster catch. That project is gonna go so far over budget both in terms of time and money that it's giving me a stomach ache just thinking about it.
  - jiggawatts 2y ago
    A Starship plus its booster costs less than a single one of the non-reusable rocket engines of the Space Launch System... which has five engines. There's simply no comparison when it comes to cost efficiency.
pjmlp 2y ago
GPUs do more than PyTorch.
- jiggawatts 2y ago
  Sure, but most of the money is being sunk into the execution of the a small number of distinct codes, scaled out. AMD doesn't need to duplicate everything NVIDIA provides, they just need to duplicate the parts relevant to most of the $3T spend the market seems to be expecting. Just make llama.cpp work robustly on AMD accelerators, and that might unlock a $500B slice of the pie by itself.
  - pjmlp 2y ago
    And this is why NVidia keeps winning.
    - talldayo 2y ago
      cough Apple could be winning if they weren't a pussy about OpenCL. cough Sorry, got something in my throat.
      - pjmlp 2y ago
        Why? Intel, AMD and Google never made anything useful with OpenCL. That is OpenCL 3.0 is OpenCL 1.0 without the OpenCL 2.x stuff that no one ever adopted.
  - talldayo 2y ago
    > AMD doesn't need to duplicate everything NVIDIA provides, they just need to duplicate the parts relevant to most of the $3T spend the market seems to be expecting. In effect, they already have. Both AMD, Apple and a number of smaller OEMs all wrote GPU compute shaders to do "the AI inference stuff" and shipped it upstream. That's about as much as they can do without redesigning their hardware, and they've already done it. Nvidia wins not because they have everything sorted out in software. They win because CUDA is baked into the design of every Nvidia GPU made in the past decade. The software helps, but it's so bloated at this point that only a small subset of it's functionality is ever used in production at any one point. What makes Nvidia fast and flexible is integration of compute in hardware at the SM-level. This is an architecture AMD and Apple both had the opportunity to reprise, even working together if they wanted, but chose not to. Now we're here. I tend to steelman the idea that it was AMD and especially Apple's mistake for eschewing this vision and abandoning OpenCL. But apparently a lot of people tend to think that AMD and Apple were right despite being less efficient at both raster and compute operations.

ivewonyoung 2y ago

Some things are hard and just throwing more money at it doesn't work because it's about company culture and leadership. Compare ICE auto industry profits since 2003 when Tesla was founded to money invested in Tesla till 2019 when Model Y was finished. Tesla had very less money to work with. Multiple iterations of Volkswagen's car software sucked so bad and the company culture and leadership just couldn't develop good software so Volkswagen created a separate startup style subsidiary just to make software. Guess what? That startup also failed and they recently bought a $5 billion stake in Rivian so that they could use Rivian's car software. Meanwhile their sales are tanking. Also look at SpaceX vs. ESA or Boeing/ULA with reusable boosters. Or China with EUV and chips. Even nation state level with essentially unlimited funding may not work.

tmtvl 2y ago
> Some things are hard and just throwing more money at it doesn't work because it's about company culture and leadership. Indeed, though that sword cuts both ways, if a hard problem gets fixed with lots of money being thrown at it then a competitor may be able to fix it for less if they have the right culture and leadership. > look at SpaceX vs. ESA or Boeing/ULA with reusable boosters. The way I understood it was that SpaceX was granted $3B of tax payer money to set up a moon mission and all they achieved was a single booster catch. That project is gonna go so far over budget both in terms of time and money that it's giving me a stomach ache just thinking about it.
- jiggawatts 2y ago
  A Starship plus its booster costs less than a single one of the non-reusable rocket engines of the Space Launch System... which has five engines. There's simply no comparison when it comes to cost efficiency.

pjmlp 2y ago

GPUs do more than PyTorch.

jiggawatts 2y ago
Sure, but most of the money is being sunk into the execution of the a small number of distinct codes, scaled out. AMD doesn't need to duplicate everything NVIDIA provides, they just need to duplicate the parts relevant to most of the $3T spend the market seems to be expecting. Just make llama.cpp work robustly on AMD accelerators, and that might unlock a $500B slice of the pie by itself.
- pjmlp 2y ago
  And this is why NVidia keeps winning.
  - talldayo 2y ago
    cough Apple could be winning if they weren't a pussy about OpenCL. cough Sorry, got something in my throat.
    - pjmlp 2y ago
      Why? Intel, AMD and Google never made anything useful with OpenCL. That is OpenCL 3.0 is OpenCL 1.0 without the OpenCL 2.x stuff that no one ever adopted.
- talldayo 2y ago
  > AMD doesn't need to duplicate everything NVIDIA provides, they just need to duplicate the parts relevant to most of the $3T spend the market seems to be expecting. In effect, they already have. Both AMD, Apple and a number of smaller OEMs all wrote GPU compute shaders to do "the AI inference stuff" and shipped it upstream. That's about as much as they can do without redesigning their hardware, and they've already done it. Nvidia wins not because they have everything sorted out in software. They win because CUDA is baked into the design of every Nvidia GPU made in the past decade. The software helps, but it's so bloated at this point that only a small subset of it's functionality is ever used in production at any one point. What makes Nvidia fast and flexible is integration of compute in hardware at the SM-level. This is an architecture AMD and Apple both had the opportunity to reprise, even working together if they wanted, but chose not to. Now we're here. I tend to steelman the idea that it was AMD and especially Apple's mistake for eschewing this vision and abandoning OpenCL. But apparently a lot of people tend to think that AMD and Apple were right despite being less efficient at both raster and compute operations.