4 ms·

A possible prescription for AMD regarding AI and CUDA: 1) Open-source driver development as mentioned in this post 2) Set up 24/7 free software tech support o

by tikkun 3y ago

A possible prescription for AMD regarding AI and CUDA: 1) Open-source driver development as mentioned in this post 2) Set up 24/7 free software tech support on discord. Maybe for all use cases, maybe only AI use cases. Do the tech support via screen sharing, and have a writer join for all calls, so that every issue once solved gets a blog post 3) Have employees run all popular AI tools and get them working on AMD hardware, publish written guides and videos showing how to do it.

zirgs 3y ago
4) Release a consumer GPU with 32-80 GB of VRAM.
- ciupicri 3y ago
  What are you planning to play on that thing?
  - freedomben 3y ago
    I think Hogwart's Legacy could benefit from it. It crashes on my 16GB system due to being out of video memory regularly. It might be lying about the cause but that's what it claims.
  - Filligree 3y ago
    Llama 2.
- Tuna-Fish 3y ago
  It sounds silly, but people would endure a lot of pain to fit significantly larger models into ram.
- fbdab103 3y ago
  It could even be 2-3 generations behind and that thing would still sell.
- latchkey 3y ago
  Or at least make it so that you could rent the larger gpus by the hour. As far as I know, there isn't a single service that offers bare metal access to MI210/MI250's.
- codedokode 3y ago
  AMD has CPUs with built-in GPU with hundreds of ALUs and unified memory (GPU uses system memory). Cannot such CPU be used for ML tasks without purchasing expensive graphic cards? 64 Gb of ordinary RAM will be cheaper than 64 Gb of VRAM. Currently such built-in GPUs seem to be optimized for games only.
  - dur-randir 3y ago
    VRAM speed is a major factor. Even ddr5 with 4 channels would be slow compared to current GDDRs.
    - codedokode 3y ago
      That's correct, but if you are, for example, multiplying two 1024x1024 matrices (typical for ML loads) then you need to perform roughly 1B operations on 1M numbers. There is no need for loading lot of data. So we can do without fast memory in this case, can't we? Hopefully the cache and registers of GPU can handle this.
hedgehog 3y ago
The problem is not that people within the company don't have good ideas of how to improve deep learning research end user experience, it's just not a priority for AMD. It's annoying as a potential customer but arguably whatever their overall strategy is it's working.
- deleted 3y ago
  [deleted]
lul_open 3y ago
Yes, let's build an open source community on top of a closed platform. It's not like twitter, reddit, Facebook etc have taught us anything. Open a mailing list like every project that survives more than 5 years. Hell the barrier to entry will ensure you get people who can use a text editor and save you half the question of 'how do I install this on on an Intel a2 Mac???'
- branon 3y ago
  Yes, absolutely do not use Discord for this, Matrix would be better

by tikkun 3y ago

zirgs 3y ago

4) Release a consumer GPU with 32-80 GB of VRAM.

ciupicri 3y ago
What are you planning to play on that thing?
- freedomben 3y ago
  I think Hogwart's Legacy could benefit from it. It crashes on my 16GB system due to being out of video memory regularly. It might be lying about the cause but that's what it claims.
- Filligree 3y ago
  Llama 2.
Tuna-Fish 3y ago
It sounds silly, but people would endure a lot of pain to fit significantly larger models into ram.
fbdab103 3y ago
It could even be 2-3 generations behind and that thing would still sell.
latchkey 3y ago
Or at least make it so that you could rent the larger gpus by the hour. As far as I know, there isn't a single service that offers bare metal access to MI210/MI250's.
codedokode 3y ago
AMD has CPUs with built-in GPU with hundreds of ALUs and unified memory (GPU uses system memory). Cannot such CPU be used for ML tasks without purchasing expensive graphic cards? 64 Gb of ordinary RAM will be cheaper than 64 Gb of VRAM. Currently such built-in GPUs seem to be optimized for games only.
- dur-randir 3y ago
  VRAM speed is a major factor. Even ddr5 with 4 channels would be slow compared to current GDDRs.
  - codedokode 3y ago
    That's correct, but if you are, for example, multiplying two 1024x1024 matrices (typical for ML loads) then you need to perform roughly 1B operations on 1M numbers. There is no need for loading lot of data. So we can do without fast memory in this case, can't we? Hopefully the cache and registers of GPU can handle this.

hedgehog 3y ago

The problem is not that people within the company don't have good ideas of how to improve deep learning research end user experience, it's just not a priority for AMD. It's annoying as a potential customer but arguably whatever their overall strategy is it's working.

deleted 3y ago
[deleted]

lul_open 3y ago

Yes, let's build an open source community on top of a closed platform. It's not like twitter, reddit, Facebook etc have taught us anything. Open a mailing list like every project that survives more than 5 years. Hell the barrier to entry will ensure you get people who can use a text editor and save you half the question of 'how do I install this on on an Intel a2 Mac???'

branon 3y ago
Yes, absolutely do not use Discord for this, Matrix would be better