3 ms·

This. People complain about the "Nvidia tax". I don't like monopolies and I fully support the efforts of AMD, Intel, Apple, anyone to chip away at this. That

This. People complain about the "Nvidia tax". I don't like monopolies and I fully support the efforts of AMD, Intel, Apple, anyone to chip away at this. That said as-is with ROCm you will: - Absolutely burn hours/days/weeks getting many (most?) things to work at all. If you get it working you need to essentially "freeze" the configuration because an upgrade means do it all over again. - In the event you get it to work at all you'll realize performance is nowhere near the hardware specs. - Throw up your hands and go back to CUDA. Between what it takes to get ROCm to work and the performance issues the Nvidia tax becomes a dividend nearly instantly once you factor in human time, less-than-optimal performance, and opportunity cost. Nvidia says roughly 30% of their costs are on software. That's what you need to do to deliver something that's actually usable in the real world. With the "Nvidia tax" they're also reaping the benefit of the ~15 years they've been sinking resources into CUDA.

dkjaudyeqooe 3y ago
Wow! It's incredible how Nvidia has created the dark voodoo magic, and how only they can deliver the strong juju for AI. How are the so incredibly smart and powerful!? I wonder if it has anything to do with the strategy they used in 3D graphics, where game developers ended up writing for NV drivers in order to maximise performance, and Nvidia abused their market position and used every trick in the book to make AMD cards run poorly. People complained about AMD driver quality, but the actual problem was that they were not NV drivers and AMD couldn't defeat their software moat. So here we are again, this time with AI. You'd think we'd learnt our lesson but instead people are fooled yet again and instead of understanding the importance of diversity and competition in the lifeblood of their art, myopia and amnesia is the order of the day. Tinygrad are doing god's work and I won't be giving Nvidia a single fucking cent of my money until the software is hardware neutral and there is real competition.
- whimsicalism 3y ago
  No, I think it is much more due to AMD absolutely failing to invest heavily in software at all. Honestly, they have had years - it is difficult to see how Nvidia abused their market position in AI when this is effectively a new market. I find this vague reflexive anti-corpo leftism that seems to have become extremely popular post-2020 really tiresome.
  - infogulch 3y ago
    For my part I'm not a leftist, and I'm not so much anti-corpo as pro-free market. I can still acknowledge that Nvidia is persistently pursuing anticompetitive policies and abusing their market position, without holing AMD on a pedestal assuming it would be different if the shoe was on the other foot.
  - dkjaudyeqooe 3y ago
    > I find this vague reflexive anti-corpo leftism that seems to have become extremely popular post-2020 really tiresome. Ah ideology, such a great alternative to actual thinking. Don't investigate or reason, just just blame it on the 'lefties'. Tiresome indeed. Not sure how me simply stating the obvious makes me a 'lefty'. If you think monopiles, regardless of how they come about, are a good idea, that companies should be allowed to lock up an important market for any reason, then that makes you a corporatist fascist, right? Wow, this mindless name calling is so much fun! I feel like a total genius. The simple fact is that it is the nature of software, its complexity and dependence on a multitude of fairly arbitrary technical choices makes it a very effective as a moat, even if its not intentional. CUDA, etc is 100% a software compatibility issue, and that's it. There's more than one way to skin a cat but we're stuck with this one. Nvidia isn't interested in interoperability, even though it's critical for the industry in the longer term. I'd wouldn't be either if it was money in my pocket. The point that is entirely missed here is that we, as a community, are screwing up by not steering the field toward better hardware compatibility, as in anyone being able to produce new hardware. In the rush to improve or or try out the latest model or software we have lost sight of this, and it will be to our great detriment. With the concentration of money in one company we will have a lot less innovation overall. Prices will be higher and resources will be misallocated. Everyone suffers. It's very possible that AI withers on the vine due to lagging hardware. It's going to need a lot of compute, and maybe a different kind of compute to boot. We may need a million or a billion times what we have to even get close to AGI. But if one company locks that up, and uses that position to squeeze out every dollar from its customers (really, have a look at the almost comical 'upgrades' Nvidia offers in their GPUs other than at the very high end) then it's going to take much longer to progress, and maybe we never get there because some small group of talented maverick researchers were never able to get their hands on the hardware they needed and never produce some critical breakthrough.
- kkielhofner 3y ago
  Sarcasm aside... Can we drop the "Nvidia is the only self-interested evil company in existence" schtick? I'm not being "fooled" by anyone. I've been trying to use ROCm since the initial release six years ago (on Vega at the time). I've spent thousands of dollars on AMD hardware over the years hoping to see progress for myself. I've burned untold amounts of time fighting with ROCm, hoping it's even remotely a viable competitor to CUDA/Nvidia. Here we are in 2024 and they're still doing braindead stuff like dropping a new ROCm release to support their flagship $1000 consumer card a full year after release... ROCm 6 looks good? Check the docker containers[0]. Their initial release for ROCm only supported Python 3.9 for some strange reason even though the previous ROCm 5.7 containers were based on Python 3.10. Python 3.10 is more-or-less the minimum for nearly anything out there. It took them 1.5 months to address this... This is merely one example, spend some time actually working with this and you will find dozens of similar "WTF?!?" bombs all over the place. I suggest you put your money and time where your mouth is (as I have) to actually try to work with ROCm. You will find that it is nowhere near the point of actually being a viable competitor to CUDA/Nvidia for anyone who's trying to get work done. > Tinygrad are doing god's work Tinygrad is packaging hardware with off the shelf components plus substantial markup. There is nothing special about this hardware and they aren't doing anything you couldn't have done in the past year. They have been vocal on calling out AMD but show me their commits to ROCm and I'll agree they are "doing god's work". We'll save the work being done on their framework for another thread. [0] - https://hub.docker.com/r/rocm/pytorch/tags https://hub.docker.com/r/rocm/pytorch/tags
  - latchkey 3y ago
    > There is nothing special about this hardware and they aren't doing anything you couldn't have done in the past year. What they are doing is all of the hardware engineering work that it takes to build something like this. You're dismissing the amount of time they spent on figuring stuff like this out: "Beating back all the PCI-E AER errors was hard, as anyone knows who has tried to build a system like this."
    - kkielhofner 3y ago
      > "Beating back all the PCI-E AER errors was hard, as anyone knows who has tried to build a system like this." Define "hard". The crypto mining community has had this working for at least half a decade with AMD cards. With Nvidia it's a non-issue. I'd be very, very curious to get more technical details on what new work they did here.
hackerlight 3y ago
> Between what it takes to get ROCm to work It's not that bad. You just copy and paste ~10 bash commands from their the official guide. 7900XTX is now officially supported by AMD. Andrew Ng says it's much better than 1 year ago and isn't as bad as people say.

dkjaudyeqooe 3y ago

Wow! It's incredible how Nvidia has created the dark voodoo magic, and how only they can deliver the strong juju for AI. How are the so incredibly smart and powerful!? I wonder if it has anything to do with the strategy they used in 3D graphics, where game developers ended up writing for NV drivers in order to maximise performance, and Nvidia abused their market position and used every trick in the book to make AMD cards run poorly. People complained about AMD driver quality, but the actual problem was that they were not NV drivers and AMD couldn't defeat their software moat. So here we are again, this time with AI. You'd think we'd learnt our lesson but instead people are fooled yet again and instead of understanding the importance of diversity and competition in the lifeblood of their art, myopia and amnesia is the order of the day. Tinygrad are doing god's work and I won't be giving Nvidia a single fucking cent of my money until the software is hardware neutral and there is real competition.

whimsicalism 3y ago
No, I think it is much more due to AMD absolutely failing to invest heavily in software at all. Honestly, they have had years - it is difficult to see how Nvidia abused their market position in AI when this is effectively a new market. I find this vague reflexive anti-corpo leftism that seems to have become extremely popular post-2020 really tiresome.
- infogulch 3y ago
  For my part I'm not a leftist, and I'm not so much anti-corpo as pro-free market. I can still acknowledge that Nvidia is persistently pursuing anticompetitive policies and abusing their market position, without holing AMD on a pedestal assuming it would be different if the shoe was on the other foot.
- dkjaudyeqooe 3y ago
  > I find this vague reflexive anti-corpo leftism that seems to have become extremely popular post-2020 really tiresome. Ah ideology, such a great alternative to actual thinking. Don't investigate or reason, just just blame it on the 'lefties'. Tiresome indeed. Not sure how me simply stating the obvious makes me a 'lefty'. If you think monopiles, regardless of how they come about, are a good idea, that companies should be allowed to lock up an important market for any reason, then that makes you a corporatist fascist, right? Wow, this mindless name calling is so much fun! I feel like a total genius. The simple fact is that it is the nature of software, its complexity and dependence on a multitude of fairly arbitrary technical choices makes it a very effective as a moat, even if its not intentional. CUDA, etc is 100% a software compatibility issue, and that's it. There's more than one way to skin a cat but we're stuck with this one. Nvidia isn't interested in interoperability, even though it's critical for the industry in the longer term. I'd wouldn't be either if it was money in my pocket. The point that is entirely missed here is that we, as a community, are screwing up by not steering the field toward better hardware compatibility, as in anyone being able to produce new hardware. In the rush to improve or or try out the latest model or software we have lost sight of this, and it will be to our great detriment. With the concentration of money in one company we will have a lot less innovation overall. Prices will be higher and resources will be misallocated. Everyone suffers. It's very possible that AI withers on the vine due to lagging hardware. It's going to need a lot of compute, and maybe a different kind of compute to boot. We may need a million or a billion times what we have to even get close to AGI. But if one company locks that up, and uses that position to squeeze out every dollar from its customers (really, have a look at the almost comical 'upgrades' Nvidia offers in their GPUs other than at the very high end) then it's going to take much longer to progress, and maybe we never get there because some small group of talented maverick researchers were never able to get their hands on the hardware they needed and never produce some critical breakthrough.
kkielhofner 3y ago
Sarcasm aside... Can we drop the "Nvidia is the only self-interested evil company in existence" schtick? I'm not being "fooled" by anyone. I've been trying to use ROCm since the initial release six years ago (on Vega at the time). I've spent thousands of dollars on AMD hardware over the years hoping to see progress for myself. I've burned untold amounts of time fighting with ROCm, hoping it's even remotely a viable competitor to CUDA/Nvidia. Here we are in 2024 and they're still doing braindead stuff like dropping a new ROCm release to support their flagship $1000 consumer card a full year after release... ROCm 6 looks good? Check the docker containers[0]. Their initial release for ROCm only supported Python 3.9 for some strange reason even though the previous ROCm 5.7 containers were based on Python 3.10. Python 3.10 is more-or-less the minimum for nearly anything out there. It took them 1.5 months to address this... This is merely one example, spend some time actually working with this and you will find dozens of similar "WTF?!?" bombs all over the place. I suggest you put your money and time where your mouth is (as I have) to actually try to work with ROCm. You will find that it is nowhere near the point of actually being a viable competitor to CUDA/Nvidia for anyone who's trying to get work done. > Tinygrad are doing god's work Tinygrad is packaging hardware with off the shelf components plus substantial markup. There is nothing special about this hardware and they aren't doing anything you couldn't have done in the past year. They have been vocal on calling out AMD but show me their commits to ROCm and I'll agree they are "doing god's work". We'll save the work being done on their framework for another thread. [0] - https://hub.docker.com/r/rocm/pytorch/tags https://hub.docker.com/r/rocm/pytorch/tags
- latchkey 3y ago
  > There is nothing special about this hardware and they aren't doing anything you couldn't have done in the past year. What they are doing is all of the hardware engineering work that it takes to build something like this. You're dismissing the amount of time they spent on figuring stuff like this out: "Beating back all the PCI-E AER errors was hard, as anyone knows who has tried to build a system like this."
  - kkielhofner 3y ago
    > "Beating back all the PCI-E AER errors was hard, as anyone knows who has tried to build a system like this." Define "hard". The crypto mining community has had this working for at least half a decade with AMD cards. With Nvidia it's a non-issue. I'd be very, very curious to get more technical details on what new work they did here.

hackerlight 3y ago

> Between what it takes to get ROCm to work It's not that bad. You just copy and paste ~10 bash commands from their the official guide. 7900XTX is now officially supported by AMD. Andrew Ng says it's much better than 1 year ago and isn't as bad as people say.