4 ms·

Everyone has been writing cuda code. Hopefully we continue the shift towards using higher level frameworks/abstractions that can determine the optimum machine c

Everyone has been writing cuda code. Hopefully we continue the shift towards using higher level frameworks/abstractions that can determine the optimum machine code for the underlying hardware. I don’t really get it either. Nvidia is pouring $$$ into protecting their advantage as anemic as it might be. They give free compute and gpu resources to universities etc to cement the dependence on cuda. When you’re a beginner or someone who needs to just “get shit done” you’ll go nvidia at the expense of the greater good.

dannyw 3y ago
Also, the full consumer (RTX) stack is supported. Lots of people who are gamers, or want a hobbyist rig can go out to the shops and start with a RTX. AMD's ML / ROCm support for their consumer cards is still awful.
- plasticchris 3y ago
  They did add official support for the latest gen rdna cards to rocm.
  - ekianjo 3y ago
    Why only the latest gen?
    - svnt 3y ago
      Because they want to recover their investment
      - aryonoco 3y ago
        Actually because ROCm ships native machine code, as opposed to a bytecode, so supporting any additional architecture is a tonne of effort. I expect support will eventually be extended to a generation or two prior GPUs, but nothing beyond that.
  - dannyw 3y ago
    Not on Linux!
    - plasticchris 3y ago
      What? It was only for Linux. https://rocm.docs.amd.com/projects/radeon/en/latest/index.html https://rocm.docs.amd.com/projects/radeon/en/latest/index.ht...
pavelstoev 3y ago
Try Hidet - open source deep learning compiler written in Python, generating optimized CUDA https://pypi.org/project/hidet/ https://pypi.org/project/hidet/
jakderrida 3y ago
> They give free compute and gpu resources to universities etc to cement the dependence on cuda. It's actually quite brilliant. It never occurred to me before that the resources we had access to at university weren't the most popular because that's what the library wanted. They were the most popular because the provider made them cheap/free to widen their moat over the competition. It reminds me of Thomson Reuters launching Eikon, a superior alternative (imho) to Bloomberg terminals. Every business school has Bloomberg terminals. But few/none had Eikon, which is why nobody knows how to use one.
- kimixa 3y ago
  It's the same reason that students get free MSDN access and unlimited windows keys. It's a pretty solid strategy, but naturally favors the current incumbent who has the $$$ to throw around.
anvuong 3y ago
What the hell is the greater good here? I'm a researcher in a R1 university, I can quickly try stupid things on my personal 3080, then if it works I can virtually git clone the codes onto my lab's server and bam it chops through the problems. After a few weeks I'll get (hopefully) some publisable results that (hopefully) advance my field a little. Do you really expect most researchers to use half of their time debugging the unfinished ROCM stack that is just not there, sacrificing their output, and maybe even their tenure prospect? We love Nvidia solutions, at least in our field, not because it's a monopoly, but because it's actually the best solution right now.
- donor20 3y ago
  I really like that you can prototype on your own consumer level card - the rocm stack has not matched that historically and has over promised and underdelivered for years now.
  - cyanydeez 3y ago
    It's getting closer, and well, what matters is price points and what, billions of people crunching away on the various problems. There's huge momentum on the "user" side of the equation that just getting cheap hardware and passable performance means Nvidia losing dominance.
nemothekid 3y ago
>Nvidia is pouring $$$ into protecting their advantage as anemic as it might be. This description isn't fair. Nvidia was pouring money into CUDA for 16 years. AMD/Intel only woke up once ChatGPT launched and nvidia started printing money. It's not fair to call it "protecting their advantage" when nvidia invested into the ecosystem for years and AMD couldn't even bother to ship working examples with OpenCL. I'm not sure why choosing the company that decided to invest in the platform for years is at the expense of the "greater good." What "greater good" is there above actually working software?
- bryanlarsen 3y ago
  Nvidia had boatloads of money to spend, and 6X the number of engineers. AMD was skating on the edge of bankruptcy. AMD bet the farm on Zen, and that bet paid off so now they have money to spend on other things. If they would have split resources off of Zen to spend on CUDA they would have failed at both and AMD would be a bankrupt husk.
- cyanydeez 3y ago
  You can get a consumer laptop with AI: https://www.amd.com/en/products/apu/amd-ryzen-7-7840hs https://www.amd.com/en/products/apu/amd-ryzen-7-7840hs The thing AMD doesn't have is the software stack.
cyanydeez 3y ago
The obvious answer is to just let the LLM boostrap use better abstractions.

dannyw 3y ago

Also, the full consumer (RTX) stack is supported. Lots of people who are gamers, or want a hobbyist rig can go out to the shops and start with a RTX. AMD's ML / ROCm support for their consumer cards is still awful.

plasticchris 3y ago
They did add official support for the latest gen rdna cards to rocm.
- ekianjo 3y ago
  Why only the latest gen?
  - svnt 3y ago
    Because they want to recover their investment
    - aryonoco 3y ago
      Actually because ROCm ships native machine code, as opposed to a bytecode, so supporting any additional architecture is a tonne of effort. I expect support will eventually be extended to a generation or two prior GPUs, but nothing beyond that.
- dannyw 3y ago
  Not on Linux!
  - plasticchris 3y ago
    What? It was only for Linux. https://rocm.docs.amd.com/projects/radeon/en/latest/index.html https://rocm.docs.amd.com/projects/radeon/en/latest/index.ht...

pavelstoev 3y ago

Try Hidet - open source deep learning compiler written in Python, generating optimized CUDA https://pypi.org/project/hidet/ https://pypi.org/project/hidet/

jakderrida 3y ago

> They give free compute and gpu resources to universities etc to cement the dependence on cuda. It's actually quite brilliant. It never occurred to me before that the resources we had access to at university weren't the most popular because that's what the library wanted. They were the most popular because the provider made them cheap/free to widen their moat over the competition. It reminds me of Thomson Reuters launching Eikon, a superior alternative (imho) to Bloomberg terminals. Every business school has Bloomberg terminals. But few/none had Eikon, which is why nobody knows how to use one.

kimixa 3y ago
It's the same reason that students get free MSDN access and unlimited windows keys. It's a pretty solid strategy, but naturally favors the current incumbent who has the $$$ to throw around.

anvuong 3y ago

What the hell is the greater good here? I'm a researcher in a R1 university, I can quickly try stupid things on my personal 3080, then if it works I can virtually git clone the codes onto my lab's server and bam it chops through the problems. After a few weeks I'll get (hopefully) some publisable results that (hopefully) advance my field a little. Do you really expect most researchers to use half of their time debugging the unfinished ROCM stack that is just not there, sacrificing their output, and maybe even their tenure prospect? We love Nvidia solutions, at least in our field, not because it's a monopoly, but because it's actually the best solution right now.

donor20 3y ago
I really like that you can prototype on your own consumer level card - the rocm stack has not matched that historically and has over promised and underdelivered for years now.
- cyanydeez 3y ago
  It's getting closer, and well, what matters is price points and what, billions of people crunching away on the various problems. There's huge momentum on the "user" side of the equation that just getting cheap hardware and passable performance means Nvidia losing dominance.

nemothekid 3y ago

>Nvidia is pouring $$$ into protecting their advantage as anemic as it might be. This description isn't fair. Nvidia was pouring money into CUDA for 16 years. AMD/Intel only woke up once ChatGPT launched and nvidia started printing money. It's not fair to call it "protecting their advantage" when nvidia invested into the ecosystem for years and AMD couldn't even bother to ship working examples with OpenCL. I'm not sure why choosing the company that decided to invest in the platform for years is at the expense of the "greater good." What "greater good" is there above actually working software?

bryanlarsen 3y ago
Nvidia had boatloads of money to spend, and 6X the number of engineers. AMD was skating on the edge of bankruptcy. AMD bet the farm on Zen, and that bet paid off so now they have money to spend on other things. If they would have split resources off of Zen to spend on CUDA they would have failed at both and AMD would be a bankrupt husk.
cyanydeez 3y ago
You can get a consumer laptop with AI: https://www.amd.com/en/products/apu/amd-ryzen-7-7840hs https://www.amd.com/en/products/apu/amd-ryzen-7-7840hs The thing AMD doesn't have is the software stack.

cyanydeez 3y ago

The obvious answer is to just let the LLM boostrap use better abstractions.