5 ms·

>My prediction is eventually there will be anti-trust ligitation, they will be required to open the CUDA standard, after which AMD will become a competitor. If

>My prediction is eventually there will be anti-trust ligitation, they will be required to open the CUDA standard, after which AMD will become a competitor. If AMD isn't a competitor before government intervention, I don't the government forcing nvidia to open up CUDA changes much. CUDA's moat isn't due to some secret sauce - nvidia put in the developer hours; and if AMDs CUDA implementation is still broken, people will continue to buy nvidia. There has been a lot of trying to get AMD to work - Hotz has been trying for a while now[1] and has been uncovering a ton of bugs in AMD drivers. To AMD's credit, they have been fixed, but it does give you a sense of how far behind they are in regards to their own software. Now imagine them trying to implement a competitor's spec? [1] https://twitter.com/__tinygrad__/status/1765085827946942923 https://twitter.com/__tinygrad__/status/1765085827946942923

AYBABTME 3y ago
I don't understand AMD in this. Isn't it insanity that they're not throwing all they've got at their software stack?
- tempaccount420 3y ago
  Hardware people don't get along very well with software people.
  - elbear 3y ago
    Why's that?
    - imtringued 3y ago
      Because they didn't go to uni when hardware-software-codesign was being taught.
      - nebula8804 3y ago
        What unis would that include? Isn't ATI Canadian? Therefore i'd expect lots of UToronto and Waterloo people there. Aren't they some of the best in this field?
    - rapsey 3y ago
      Because it is a different type of engineering. If you manage software development like you manage hardware development your software is going to be bad. That has always been AMD's problem and it is not likely to get fixed.
      - AYBABTME 3y ago
        2t$ problem of egos?
    - deleted 3y ago
      [deleted]
- deleted 3y ago
  [deleted]
- roenxi 3y ago
  You know what happens to companies that panic and throw all their resources into knee-jerk software projects? I don't, but I'd predict it is ugly. Adding more people to a bad project generally makes it worse. The issue that AMD has is they had a long period where they clearly had no idea what they were doing. You could tell just from looking at websites, CUDA pretty much immediately gets to "here is a library for FFT", "here is a library for sparse matricies". AMD would explain that ROCM is an abbreviation of the ROCm Software platform or something unspeakably stupid. And that your graphics card wasn't supported. That changed a few months ago; so it looks like they have put some competent PMs in the chair now or something. But it'll take months for the flow on effects to reach the market. They have to figure out what the problems are which takes months to do properly; then fix the software (1-3 months more minimum); then get it into the open and the foundational libraries like PyTorch pick it up (might take another year). You can speed that up, but more cooks in the kitchen is not the way. Bandwidth use needs to be optimised. It isn't like ROCm seems lacks key features; it can technically do inference and training. My card crashes regularly though (might be a VRAM issue) so it is useless in practice. AMD can check boxes but the software doesn't really work and grappling with that organisationally is hard. Unless you have the right people in the right places, which AMD didn't have up to at least mid 2023.
  - elcomet 3y ago
    Pytorch has been supporting rocm for all last 2 years
    - blagie 3y ago
      I'd add quotes there: Pytorch has been "supporting" rocm for all last 2 years
      - elcomet 3y ago
        Can you say more?
  - Certhas 3y ago
    Look at AMDs vs Intel. They have now surpassed Intel in terms of CPUs sold and market cap. That was unthinkable even six, seven years ago. It makes perfect sense that, organisationally, they were focused on that battle. If you remember the Athlon days, AMD beat Intel before, but briefly. It didn't last. This time it looks like they beat Intel and have had the focus to stay. Intel will come back and beat them some cycles, but there is no collapse on the horizon. So it makes sense that they started looking at nVidia in the last year or so. Of course nVidia has amassed an obscene war chest in the meantime...
- imtringued 3y ago
  You have to remember that this only applies to cheap consumer GPUs, they tend to support their datacenter GPUs better. When you consider that Ryzen AI already eats the AI inference lunch, having better GPUs with better software only threatens to cannibalize their data center GPU offering. Given enough time nobody will care about using AMD GPUs for AI.
  - wmf 3y ago
    Yet we've heard about nobody doing training on AMD.
- logicchains 3y ago
  It's a political problem. Good software engineers are paid more than good hardware engineers, but AMD management is unwilling to pay up to bring on good software engineers because then they'd also need to pay their hardware engineers more, otherwise the hardware engineers would be unsatisfied. If you check NVidia salaries online you'll see NVidia pays significantly more than AMD for both hardware and software engineers; it's a classic case of AMD management being penny-wise, pound-foolish.
  - dheera 3y ago
    This is quite possibly also why Boeing is having issues. If they paid everyone $1M/year salaries maybe more people would consider going into aerospace engineering. Right now though Boeing's starting salaries aren't that much higher than what an Uber driver in the bay area makes.
- e4325f 3y ago
  They did buy Nod.ai recently
blitzar 3y ago
Getting this working might be worth a trillion $ to AMD - they should be doing more than just waiting for a bootstrapped startup to debug their drivers for them.
matt-p 3y ago
It changes alot. It is not legal to make a 'CUDA' driver for an AMD GPU as Nvidia own cuda. You can see there was a open implementation of this that AMD sponsored until they got threatened with a lawsuit by Nvidia
- dogma1138 3y ago
  ZLUDA ate the dust not because they implemented CUDA but because they were misusing complied NVIDIA libraries. If it was a clean room implementation of the API NVIDIA wouldn’t care. Heck that’s exactly what AMD did with HIP. But what you cannot do is essentially intercept calls to and reverse engineer NVIDIA binaries in real time because you can’t be arsed to build your own.
  - dheera 3y ago
    > But what you cannot do is essentially intercept calls to and reverse engineer NVIDIA binaries in real time because you can’t be arsed to build your own. And this is precisely what anti-trust ligitation would allow them to do. Preventing someone from reverse-engineering a product with the sole intention of maintaining monopoly status may be seen as anti-competitive. AMD makes really, really good CPUs now, but only after ligitation against Intel allowed them to keep up with evolving x86 standards. It's not about being "arsed" to build your own, the problem is NVIDIA controls the ecosystem-wide standard. NVIDIA can add to CUDA at any point in time and launch a GPU at the same time, the ecosystem would be forced to buy it if they want to stay on the cutting edge, and AMD would never be able to compete or reverse engineer these new standards in time.
    - dogma1138 3y ago
      They didn’t RE the CUDA API, they reused libraries like cudnn that’s a completely different case than say Oracle vs Google. What ZULDA did wasn’t to maintain a compatibility with the CUDA API and provided an open implementation of it but rather use all the CUDA based libraries that NVIDIA provided on top of it. The equivalence again would be that not only Google implemented their own Java compatible API but rather that they used the now Oracle owned JVM to do so and redistributed it. AMD already implemented CUDA essentially one to one in the form of HIP in ROCm. The issue that they face is that they don't have all the equivalent middleware to make copy pasting code actually work and this was what ZULDA did but instead of building a HIPdnn they just reused NVIDIA binaries.

AYBABTME 3y ago

I don't understand AMD in this. Isn't it insanity that they're not throwing all they've got at their software stack?

tempaccount420 3y ago
Hardware people don't get along very well with software people.
- elbear 3y ago
  Why's that?
  - imtringued 3y ago
    Because they didn't go to uni when hardware-software-codesign was being taught.
    - nebula8804 3y ago
      What unis would that include? Isn't ATI Canadian? Therefore i'd expect lots of UToronto and Waterloo people there. Aren't they some of the best in this field?
  - rapsey 3y ago
    Because it is a different type of engineering. If you manage software development like you manage hardware development your software is going to be bad. That has always been AMD's problem and it is not likely to get fixed.
    - AYBABTME 3y ago
      2t$ problem of egos?
  - deleted 3y ago
    [deleted]
deleted 3y ago
[deleted]
roenxi 3y ago
You know what happens to companies that panic and throw all their resources into knee-jerk software projects? I don't, but I'd predict it is ugly. Adding more people to a bad project generally makes it worse. The issue that AMD has is they had a long period where they clearly had no idea what they were doing. You could tell just from looking at websites, CUDA pretty much immediately gets to "here is a library for FFT", "here is a library for sparse matricies". AMD would explain that ROCM is an abbreviation of the ROCm Software platform or something unspeakably stupid. And that your graphics card wasn't supported. That changed a few months ago; so it looks like they have put some competent PMs in the chair now or something. But it'll take months for the flow on effects to reach the market. They have to figure out what the problems are which takes months to do properly; then fix the software (1-3 months more minimum); then get it into the open and the foundational libraries like PyTorch pick it up (might take another year). You can speed that up, but more cooks in the kitchen is not the way. Bandwidth use needs to be optimised. It isn't like ROCm seems lacks key features; it can technically do inference and training. My card crashes regularly though (might be a VRAM issue) so it is useless in practice. AMD can check boxes but the software doesn't really work and grappling with that organisationally is hard. Unless you have the right people in the right places, which AMD didn't have up to at least mid 2023.
- elcomet 3y ago
  Pytorch has been supporting rocm for all last 2 years
  - blagie 3y ago
    I'd add quotes there: Pytorch has been "supporting" rocm for all last 2 years
    - elcomet 3y ago
      Can you say more?
- Certhas 3y ago
  Look at AMDs vs Intel. They have now surpassed Intel in terms of CPUs sold and market cap. That was unthinkable even six, seven years ago. It makes perfect sense that, organisationally, they were focused on that battle. If you remember the Athlon days, AMD beat Intel before, but briefly. It didn't last. This time it looks like they beat Intel and have had the focus to stay. Intel will come back and beat them some cycles, but there is no collapse on the horizon. So it makes sense that they started looking at nVidia in the last year or so. Of course nVidia has amassed an obscene war chest in the meantime...
imtringued 3y ago
You have to remember that this only applies to cheap consumer GPUs, they tend to support their datacenter GPUs better. When you consider that Ryzen AI already eats the AI inference lunch, having better GPUs with better software only threatens to cannibalize their data center GPU offering. Given enough time nobody will care about using AMD GPUs for AI.
- wmf 3y ago
  Yet we've heard about nobody doing training on AMD.
logicchains 3y ago
It's a political problem. Good software engineers are paid more than good hardware engineers, but AMD management is unwilling to pay up to bring on good software engineers because then they'd also need to pay their hardware engineers more, otherwise the hardware engineers would be unsatisfied. If you check NVidia salaries online you'll see NVidia pays significantly more than AMD for both hardware and software engineers; it's a classic case of AMD management being penny-wise, pound-foolish.
- dheera 3y ago
  This is quite possibly also why Boeing is having issues. If they paid everyone $1M/year salaries maybe more people would consider going into aerospace engineering. Right now though Boeing's starting salaries aren't that much higher than what an Uber driver in the bay area makes.
e4325f 3y ago
They did buy Nod.ai recently

blitzar 3y ago

Getting this working might be worth a trillion $ to AMD - they should be doing more than just waiting for a bootstrapped startup to debug their drivers for them.

matt-p 3y ago

It changes alot. It is not legal to make a 'CUDA' driver for an AMD GPU as Nvidia own cuda. You can see there was a open implementation of this that AMD sponsored until they got threatened with a lawsuit by Nvidia

dogma1138 3y ago
ZLUDA ate the dust not because they implemented CUDA but because they were misusing complied NVIDIA libraries. If it was a clean room implementation of the API NVIDIA wouldn’t care. Heck that’s exactly what AMD did with HIP. But what you cannot do is essentially intercept calls to and reverse engineer NVIDIA binaries in real time because you can’t be arsed to build your own.
- dheera 3y ago
  > But what you cannot do is essentially intercept calls to and reverse engineer NVIDIA binaries in real time because you can’t be arsed to build your own. And this is precisely what anti-trust ligitation would allow them to do. Preventing someone from reverse-engineering a product with the sole intention of maintaining monopoly status may be seen as anti-competitive. AMD makes really, really good CPUs now, but only after ligitation against Intel allowed them to keep up with evolving x86 standards. It's not about being "arsed" to build your own, the problem is NVIDIA controls the ecosystem-wide standard. NVIDIA can add to CUDA at any point in time and launch a GPU at the same time, the ecosystem would be forced to buy it if they want to stay on the cutting edge, and AMD would never be able to compete or reverse engineer these new standards in time.
  - dogma1138 3y ago
    They didn’t RE the CUDA API, they reused libraries like cudnn that’s a completely different case than say Oracle vs Google. What ZULDA did wasn’t to maintain a compatibility with the CUDA API and provided an open implementation of it but rather use all the CUDA based libraries that NVIDIA provided on top of it. The equivalence again would be that not only Google implemented their own Java compatible API but rather that they used the now Oracle owned JVM to do so and redistributed it. AMD already implemented CUDA essentially one to one in the form of HIP in ROCm. The issue that they face is that they don't have all the equivalent middleware to make copy pasting code actually work and this was what ZULDA did but instead of building a HIPdnn they just reused NVIDIA binaries.