5 ms·

> Give AMD Engineers more compute and engineering resources to fix and improve the AMD ecosystem, they have very few internal gpu boxes relative to what Nvidia

by nrp 2y ago

> Give AMD Engineers more compute and engineering resources to fix and improve the AMD ecosystem, they have very few internal gpu boxes relative to what Nvidia provides to their engineers. This is real. We’ve found ourselves having to give hardware to engineers at AMD because they’re unable to get allocation of it internally.

mountainriver 2y ago
This seems so insane, is anyone actually doing the work to provide an alternative to CUDA? Maybe Google?
- ryao 2y ago
  https://www.intel.com/content/www/us/en/developer/articles/technical/quick-guide-to-sycl-implementations.html https://www.intel.com/content/www/us/en/developer/articles/t...
- anon291 2y ago
  Honestly, probably NVIDIA itself, since they contribute significantly to many open-source projects (MLIR), and also make their SoTA GEMM/Conv implementations open-source and available for study (Cutlass).
  - david-gpu 2y ago
    *> also make their SoTA GEMM/Conv implementations open-source and available for study (Cutlass)" Cutlass is a fine piece of engineering, but it is not quite as good as their closed source libraries in real world workloads. There is secret sauce that is not open sourced.
- pjmlp 2y ago
  The problem is that everyone that tries, keeps missing the CUDA forest and focus only in a specific kind of tree.
Archit3ch 2y ago
Coming up next: "We bought AMD stock on the open market and used it to compensate AMD engineers".
- Lerc 2y ago
  You joke, but it is almost a genuine investment opportunity here for a large player. Spend a billion on AMD shares, Spend another Billion on a out-of-house software team to solve the software solution to more than double the share price. Taking into account that there are players that already own billions in AMD shares, they could probably do that as well. On the other hand perhaps it would be better for them, as major shareholders, to have a word with AMD management.
  - schmidtleonard 2y ago
    I don't have the inside baseball but I have seen those weird as hell interviews with Lisa Su where she gets asked point blank about the software problems and instead of "working on it, stay tuned" -- an answer that costs nothing to give -- she deflects into "performance is what matters," which is the kind of denial that rhymes exactly with the problems they are having. No, the horsepower of your F1 racecar doesn't matter if the engine doesn't start and there's a wheel missing! You need to fix those problems before the horsepower can matter! Please tell me you are fixing the starter and the wheel! Hopefully I am reading too much into this. Hopefully she doesn't have any weird hangups over investing in software and it all just takes time to Do It Right after GPGPU got starved in the AMD winter. But if it is a weird hangup then yeah, 100%, ownership needs to get management in line because whiffing a matmul benchmark years into a world where matmul is worth trillions just ain't it.
    - sangnoir 2y ago
      > she deflects into "performance is what matters," which is the kind of denial that rhymes exactly with the problems they are having. It's not a deflection, but a straightforward description of AMDs current top-down market strategy of partnering with big players instead of doubling down to have a great OOBE for consumers & others who don't order GPUs by the pallet. It's an honest reflection if their current core competencies, and the opportunity presented by Nvidia's margins. They are going for a bang-for-buck right now aiming at data center workloads, and the hyperscalers care a lot about perf/$ than raw performance at. Hyperscalers are also more self-sufficient at software: they have entire teams working on PyTorch, Jax, and writing kernels.
AlotOfReading 2y ago
Sadly common at hardware companies. The most extreme case I've heard of is ASML, who supposedly doesn't keep any machines of their own. They test against "almost-ready" machines right before they go out the door to customers.
- rahkiin 2y ago
  That’s why they cooperate closely with imec and their FAB in Leuven
- dmoy 2y ago
  ASML might be an extreme outlier though, don't those things cost like $50 million+ each?
  - amirhirsch 2y ago
    Put a 1 or 2 in front of that
  - tonetegeatinst 2y ago
    Many for last gen process nodes, and from a second or third hand supplier if you could even find one. ASML makes very few fully working machines each year, and the cost and throughput those machines have is astronomical. They have spare parts you'd bet, and I'd bet they have some SLA agreement with each customer where an engineer is basically on call nearby in case a single thing dosnt work or a random part breaks or needs servicing. Asianometry did a great video on the cost of downtime when it comes to ASML device in any fab. While I am not directly in this field and can't speak to the accuracy of the numbers john gives, he does not seem one to just make stuff up as his quality of video production for niche topics is quite good.
    - bobmcnamara 2y ago
      Almost a decade ago KFAB had a fire, power was cut, everything in process was dumped, they planned to restart but ended up being was cheaper to close the whole facility . Probably for the best though, KFAB had been discharging several tons of solvents, cleaning agents, and reagents per year into the surrounding area [for as long as it ran](https://enviro.epa.gov/facts/tri/ef-facilities/#/Release/64063TTTCH777NB https://enviro.epa.gov/facts/tri/ef-facilities/#/Release/640...) https://enviro.epa.gov/facts/tri/ef-facilities/#/Release/64063TTTCH777NB https://enviro.epa.gov/facts/tri/ef-facilities/#/Release/640...
claytonjy 2y ago
I was surprised to hear recently that the same happens at NVIDIA! Hopefully less frequently, but I can understand why it's hard to keep many units on hand given the level of external demand.
theptip 2y ago
This is baffling. I’m sure there are many technical reasons I don’t grok that AMD’s job is challenging, but it’s wild that they are dropping the ball on such obvious stuff as this. The prize is trillions of dollars, and they can print hundreds of millions if they can convince the market that they are closing the gap. It’s embarrassing that whoever actually tries to use their product hits these crass bugs (same with geohot who was really invested in making AMD’s cards work; I think he just ran their demo script in a loop and produced crashes). It seems they really don’t understand/value the developer flywheel.
- mring33621 2y ago
  In the late 90's, US manufacturers, including high-tech electronics, had 2 mantras: 1) Cash is king 2) Inventory is evil I think this mindset may still be here, in 2024
- mike_hearn 2y ago
  There's a recent interview with Lisa Su where she basically says she's never been interested in software because hardware is harder, she doesn't believe AMD has any problems in the software department anyway and AMD is doing great in AI. So make of that what you will. Suffice it to say, clearly the AMD board doesn't care either because otherwise they'd replace her.
  - pella 2y ago
    --> https://x.com/dylan522p/status/1871287937268383867 https://x.com/dylan522p/status/1871287937268383867

by nrp 2y ago

mountainriver 2y ago

This seems so insane, is anyone actually doing the work to provide an alternative to CUDA? Maybe Google?

ryao 2y ago
https://www.intel.com/content/www/us/en/developer/articles/technical/quick-guide-to-sycl-implementations.html https://www.intel.com/content/www/us/en/developer/articles/t...
anon291 2y ago
Honestly, probably NVIDIA itself, since they contribute significantly to many open-source projects (MLIR), and also make their SoTA GEMM/Conv implementations open-source and available for study (Cutlass).
- david-gpu 2y ago
  *> also make their SoTA GEMM/Conv implementations open-source and available for study (Cutlass)" Cutlass is a fine piece of engineering, but it is not quite as good as their closed source libraries in real world workloads. There is secret sauce that is not open sourced.
pjmlp 2y ago
The problem is that everyone that tries, keeps missing the CUDA forest and focus only in a specific kind of tree.

Archit3ch 2y ago

Coming up next: "We bought AMD stock on the open market and used it to compensate AMD engineers".

Lerc 2y ago
You joke, but it is almost a genuine investment opportunity here for a large player. Spend a billion on AMD shares, Spend another Billion on a out-of-house software team to solve the software solution to more than double the share price. Taking into account that there are players that already own billions in AMD shares, they could probably do that as well. On the other hand perhaps it would be better for them, as major shareholders, to have a word with AMD management.
- schmidtleonard 2y ago
  I don't have the inside baseball but I have seen those weird as hell interviews with Lisa Su where she gets asked point blank about the software problems and instead of "working on it, stay tuned" -- an answer that costs nothing to give -- she deflects into "performance is what matters," which is the kind of denial that rhymes exactly with the problems they are having. No, the horsepower of your F1 racecar doesn't matter if the engine doesn't start and there's a wheel missing! You need to fix those problems before the horsepower can matter! Please tell me you are fixing the starter and the wheel! Hopefully I am reading too much into this. Hopefully she doesn't have any weird hangups over investing in software and it all just takes time to Do It Right after GPGPU got starved in the AMD winter. But if it is a weird hangup then yeah, 100%, ownership needs to get management in line because whiffing a matmul benchmark years into a world where matmul is worth trillions just ain't it.
  - sangnoir 2y ago
    > she deflects into "performance is what matters," which is the kind of denial that rhymes exactly with the problems they are having. It's not a deflection, but a straightforward description of AMDs current top-down market strategy of partnering with big players instead of doubling down to have a great OOBE for consumers & others who don't order GPUs by the pallet. It's an honest reflection if their current core competencies, and the opportunity presented by Nvidia's margins. They are going for a bang-for-buck right now aiming at data center workloads, and the hyperscalers care a lot about perf/$ than raw performance at. Hyperscalers are also more self-sufficient at software: they have entire teams working on PyTorch, Jax, and writing kernels.

AlotOfReading 2y ago

Sadly common at hardware companies. The most extreme case I've heard of is ASML, who supposedly doesn't keep any machines of their own. They test against "almost-ready" machines right before they go out the door to customers.

rahkiin 2y ago
That’s why they cooperate closely with imec and their FAB in Leuven
dmoy 2y ago
ASML might be an extreme outlier though, don't those things cost like $50 million+ each?
- amirhirsch 2y ago
  Put a 1 or 2 in front of that
- tonetegeatinst 2y ago
  Many for last gen process nodes, and from a second or third hand supplier if you could even find one. ASML makes very few fully working machines each year, and the cost and throughput those machines have is astronomical. They have spare parts you'd bet, and I'd bet they have some SLA agreement with each customer where an engineer is basically on call nearby in case a single thing dosnt work or a random part breaks or needs servicing. Asianometry did a great video on the cost of downtime when it comes to ASML device in any fab. While I am not directly in this field and can't speak to the accuracy of the numbers john gives, he does not seem one to just make stuff up as his quality of video production for niche topics is quite good.
  - bobmcnamara 2y ago
    Almost a decade ago KFAB had a fire, power was cut, everything in process was dumped, they planned to restart but ended up being was cheaper to close the whole facility . Probably for the best though, KFAB had been discharging several tons of solvents, cleaning agents, and reagents per year into the surrounding area [for as long as it ran](https://enviro.epa.gov/facts/tri/ef-facilities/#/Release/64063TTTCH777NB https://enviro.epa.gov/facts/tri/ef-facilities/#/Release/640...) https://enviro.epa.gov/facts/tri/ef-facilities/#/Release/64063TTTCH777NB https://enviro.epa.gov/facts/tri/ef-facilities/#/Release/640...

claytonjy 2y ago

I was surprised to hear recently that the same happens at NVIDIA! Hopefully less frequently, but I can understand why it's hard to keep many units on hand given the level of external demand.

theptip 2y ago

This is baffling. I’m sure there are many technical reasons I don’t grok that AMD’s job is challenging, but it’s wild that they are dropping the ball on such obvious stuff as this. The prize is trillions of dollars, and they can print hundreds of millions if they can convince the market that they are closing the gap. It’s embarrassing that whoever actually tries to use their product hits these crass bugs (same with geohot who was really invested in making AMD’s cards work; I think he just ran their demo script in a loop and produced crashes). It seems they really don’t understand/value the developer flywheel.

mring33621 2y ago
In the late 90's, US manufacturers, including high-tech electronics, had 2 mantras: 1) Cash is king 2) Inventory is evil I think this mindset may still be here, in 2024
mike_hearn 2y ago
There's a recent interview with Lisa Su where she basically says she's never been interested in software because hardware is harder, she doesn't believe AMD has any problems in the software department anyway and AMD is doing great in AI. So make of that what you will. Suffice it to say, clearly the AMD board doesn't care either because otherwise they'd replace her.
- pella 2y ago
  --> https://x.com/dylan522p/status/1871287937268383867 https://x.com/dylan522p/status/1871287937268383867