15 ms·

Sure, if you want to waste your time on getting stuff working on AMD instead of spending it on actual model training...

kkielhofner 3y ago
This. People complain about the "Nvidia tax". I don't like monopolies and I fully support the efforts of AMD, Intel, Apple, anyone to chip away at this. That said as-is with ROCm you will: - Absolutely burn hours/days/weeks getting many (most?) things to work at all. If you get it working you need to essentially "freeze" the configuration because an upgrade means do it all over again. - In the event you get it to work at all you'll realize performance is nowhere near the hardware specs. - Throw up your hands and go back to CUDA. Between what it takes to get ROCm to work and the performance issues the Nvidia tax becomes a dividend nearly instantly once you factor in human time, less-than-optimal performance, and opportunity cost. Nvidia says roughly 30% of their costs are on software. That's what you need to do to deliver something that's actually usable in the real world. With the "Nvidia tax" they're also reaping the benefit of the ~15 years they've been sinking resources into CUDA.
- dkjaudyeqooe 3y ago
  Wow! It's incredible how Nvidia has created the dark voodoo magic, and how only they can deliver the strong juju for AI. How are the so incredibly smart and powerful!? I wonder if it has anything to do with the strategy they used in 3D graphics, where game developers ended up writing for NV drivers in order to maximise performance, and Nvidia abused their market position and used every trick in the book to make AMD cards run poorly. People complained about AMD driver quality, but the actual problem was that they were not NV drivers and AMD couldn't defeat their software moat. So here we are again, this time with AI. You'd think we'd learnt our lesson but instead people are fooled yet again and instead of understanding the importance of diversity and competition in the lifeblood of their art, myopia and amnesia is the order of the day. Tinygrad are doing god's work and I won't be giving Nvidia a single fucking cent of my money until the software is hardware neutral and there is real competition.
  - whimsicalism 3y ago
    No, I think it is much more due to AMD absolutely failing to invest heavily in software at all. Honestly, they have had years - it is difficult to see how Nvidia abused their market position in AI when this is effectively a new market. I find this vague reflexive anti-corpo leftism that seems to have become extremely popular post-2020 really tiresome.
    - infogulch 3y ago
      For my part I'm not a leftist, and I'm not so much anti-corpo as pro-free market. I can still acknowledge that Nvidia is persistently pursuing anticompetitive policies and abusing their market position, without holing AMD on a pedestal assuming it would be different if the shoe was on the other foot.
    - dkjaudyeqooe 3y ago
      > I find this vague reflexive anti-corpo leftism that seems to have become extremely popular post-2020 really tiresome. Ah ideology, such a great alternative to actual thinking. Don't investigate or reason, just just blame it on the 'lefties'. Tiresome indeed. Not sure how me simply stating the obvious makes me a 'lefty'. If you think monopiles, regardless of how they come about, are a good idea, that companies should be allowed to lock up an important market for any reason, then that makes you a corporatist fascist, right? Wow, this mindless name calling is so much fun! I feel like a total genius. The simple fact is that it is the nature of software, its complexity and dependence on a multitude of fairly arbitrary technical choices makes it a very effective as a moat, even if its not intentional. CUDA, etc is 100% a software compatibility issue, and that's it. There's more than one way to skin a cat but we're stuck with this one. Nvidia isn't interested in interoperability, even though it's critical for the industry in the longer term. I'd wouldn't be either if it was money in my pocket. The point that is entirely missed here is that we, as a community, are screwing up by not steering the field toward better hardware compatibility, as in anyone being able to produce new hardware. In the rush to improve or or try out the latest model or software we have lost sight of this, and it will be to our great detriment. With the concentration of money in one company we will have a lot less innovation overall. Prices will be higher and resources will be misallocated. Everyone suffers. It's very possible that AI withers on the vine due to lagging hardware. It's going to need a lot of compute, and maybe a different kind of compute to boot. We may need a million or a billion times what we have to even get close to AGI. But if one company locks that up, and uses that position to squeeze out every dollar from its customers (really, have a look at the almost comical 'upgrades' Nvidia offers in their GPUs other than at the very high end) then it's going to take much longer to progress, and maybe we never get there because some small group of talented maverick researchers were never able to get their hands on the hardware they needed and never produce some critical breakthrough.
- hackerlight 3y ago
  > Between what it takes to get ROCm to work It's not that bad. You just copy and paste ~10 bash commands from their the official guide. 7900XTX is now officially supported by AMD. Andrew Ng says it's much better than 1 year ago and isn't as bad as people say.
dkjaudyeqooe 3y ago
Resistance is useless! Lets just accept our fate and toe the line. Why feel bad about paying essentially double, or getting half the compute for our money when we can just choose the easy route and accept our fate and feed the monopoly a little more money so they can charge us even more money. Who needs competition!
lbotos 3y ago
Isn't the entire point of tinygrad and the tinybox the "apple style" of we are building this software to work best on our hardware?
- whimsicalism 3y ago
  it's not there yet - and i don't really understand why they don't just try to upstream stuff to pytorch

This. People complain about the "Nvidia tax". I don't like monopolies and I fully support the efforts of AMD, Intel, Apple, anyone to chip away at this. That said as-is with ROCm you will: - Absolutely burn hours/days/weeks getting many (most?) things to work at all. If you get it working you need to essentially "freeze" the configuration because an upgrade means do it all over again. - In the event you get it to work at all you'll realize performance is nowhere near the hardware specs. - Throw up your hands and go back to CUDA. Between what it takes to get ROCm to work and the performance issues the Nvidia tax becomes a dividend nearly instantly once you factor in human time, less-than-optimal performance, and opportunity cost. Nvidia says roughly 30% of their costs are on software. That's what you need to do to deliver something that's actually usable in the real world. With the "Nvidia tax" they're also reaping the benefit of the ~15 years they've been sinking resources into CUDA.

dkjaudyeqooe 3y ago
Wow! It's incredible how Nvidia has created the dark voodoo magic, and how only they can deliver the strong juju for AI. How are the so incredibly smart and powerful!? I wonder if it has anything to do with the strategy they used in 3D graphics, where game developers ended up writing for NV drivers in order to maximise performance, and Nvidia abused their market position and used every trick in the book to make AMD cards run poorly. People complained about AMD driver quality, but the actual problem was that they were not NV drivers and AMD couldn't defeat their software moat. So here we are again, this time with AI. You'd think we'd learnt our lesson but instead people are fooled yet again and instead of understanding the importance of diversity and competition in the lifeblood of their art, myopia and amnesia is the order of the day. Tinygrad are doing god's work and I won't be giving Nvidia a single fucking cent of my money until the software is hardware neutral and there is real competition.
- whimsicalism 3y ago
  No, I think it is much more due to AMD absolutely failing to invest heavily in software at all. Honestly, they have had years - it is difficult to see how Nvidia abused their market position in AI when this is effectively a new market. I find this vague reflexive anti-corpo leftism that seems to have become extremely popular post-2020 really tiresome.
  - infogulch 3y ago
    For my part I'm not a leftist, and I'm not so much anti-corpo as pro-free market. I can still acknowledge that Nvidia is persistently pursuing anticompetitive policies and abusing their market position, without holing AMD on a pedestal assuming it would be different if the shoe was on the other foot.
  - dkjaudyeqooe 3y ago
    > I find this vague reflexive anti-corpo leftism that seems to have become extremely popular post-2020 really tiresome. Ah ideology, such a great alternative to actual thinking. Don't investigate or reason, just just blame it on the 'lefties'. Tiresome indeed. Not sure how me simply stating the obvious makes me a 'lefty'. If you think monopiles, regardless of how they come about, are a good idea, that companies should be allowed to lock up an important market for any reason, then that makes you a corporatist fascist, right? Wow, this mindless name calling is so much fun! I feel like a total genius. The simple fact is that it is the nature of software, its complexity and dependence on a multitude of fairly arbitrary technical choices makes it a very effective as a moat, even if its not intentional. CUDA, etc is 100% a software compatibility issue, and that's it. There's more than one way to skin a cat but we're stuck with this one. Nvidia isn't interested in interoperability, even though it's critical for the industry in the longer term. I'd wouldn't be either if it was money in my pocket. The point that is entirely missed here is that we, as a community, are screwing up by not steering the field toward better hardware compatibility, as in anyone being able to produce new hardware. In the rush to improve or or try out the latest model or software we have lost sight of this, and it will be to our great detriment. With the concentration of money in one company we will have a lot less innovation overall. Prices will be higher and resources will be misallocated. Everyone suffers. It's very possible that AI withers on the vine due to lagging hardware. It's going to need a lot of compute, and maybe a different kind of compute to boot. We may need a million or a billion times what we have to even get close to AGI. But if one company locks that up, and uses that position to squeeze out every dollar from its customers (really, have a look at the almost comical 'upgrades' Nvidia offers in their GPUs other than at the very high end) then it's going to take much longer to progress, and maybe we never get there because some small group of talented maverick researchers were never able to get their hands on the hardware they needed and never produce some critical breakthrough.
hackerlight 3y ago
> Between what it takes to get ROCm to work It's not that bad. You just copy and paste ~10 bash commands from their the official guide. 7900XTX is now officially supported by AMD. Andrew Ng says it's much better than 1 year ago and isn't as bad as people say.

dkjaudyeqooe 3y ago

Resistance is useless! Lets just accept our fate and toe the line. Why feel bad about paying essentially double, or getting half the compute for our money when we can just choose the easy route and accept our fate and feed the monopoly a little more money so they can charge us even more money. Who needs competition!

lbotos 3y ago

Isn't the entire point of tinygrad and the tinybox the "apple style" of we are building this software to work best on our hardware?

whimsicalism 3y ago
it's not there yet - and i don't really understand why they don't just try to upstream stuff to pytorch