4 ms·

OT but question from someone curious..... is Cuda still entrenched as the only option for doing AI or is there growing support for AMD/Intel/Other ways of doing

OT but question from someone curious..... is Cuda still entrenched as the only option for doing AI or is there growing support for AMD/Intel/Other ways of doing AI?

BraverHeart 3y ago
George Hotz is attempting to solve this: https://github.com/tinygrad/tinygrad https://github.com/tinygrad/tinygrad
- ZoomerCretin 3y ago
  He loudly gave up on AMD after they did not fix a blocker he had for 5+ months and gave him the runaround the entire time when he asked for the code to fix it himself. He is still shipping the AMD tinybox with huge warning labels.
  - Art9681 3y ago
    Didn't they recently announce that everything was open sourced? Would be cool if he took another look at it once all of the souce code is available (if not already).
    - xjay 3y ago
      > They haven't open sourced anything. They posted a tweet. [1] [1@2024-04-06] https://www.youtube.com/watch?v=j7MRj4N2Cyk&t=429s https://www.youtube.com/watch?v=j7MRj4N2Cyk&t=429s [Twitch] https://twitch.tv/georgehotz https://twitch.tv/georgehotz
  - magicalhippo 3y ago
    Randomly stumbled over this[1] post with another fed up open source contributor, due to several serious issues with AMDs GPU drivers and firmware that remain unresolved for years. It also references the geohot decision you mention. Some quotes: I find it incredible that these companies that have large support contracts with you and have invested hundreds of thousands of dollars into your products, have been forced to turn to me, a mostly unknown self-employed hacker with very limited resources to try to work around these bugs (design faults?) in your hardware. In the VFIO space we no longer recommend AMD GPUs at all, in every instance where people ask for which GPU to use for their new build, the advise is to use NVidia. [1]: https://www.reddit.com/r/Amd/comments/1bsjm5a/letter_to_amd_ongoing_amd/ https://www.reddit.com/r/Amd/comments/1bsjm5a/letter_to_amd_...
- fwip 3y ago
  I'm not sure if he's "attempting to solve it" so much as he's looking for yet another way to keep himself famous. The guy did one good jailbreak for the iPhone, and as near as I can tell, the rest of his work has been a lot of boasting, half-assed hyped-up implementations (e.g: his self-driving car), and trying to befriend other powerful people in tech (see: his promise to single-handedly fix Musk's Twitter). He might be a smart dude, but he vastly overrates his own accomplishments, and doesn't finish near anything he starts.
WithinReason 3y ago
There are some stirrings but don't hold your breath
blackeyeblitzar 3y ago
See my comment on this here: https://news.ycombinator.com/item?id=39973816 https://news.ycombinator.com/item?id=39973816
sigmoid10 3y ago
There are a few attempts here and there in various stages of progression. But right now, nothing matches Nvidia+CUDA in speed and usability.
adam_arthur 3y ago
You can run inference today on pretty much any card. Download Ollama on a modern MacBook and can run 13B and even higher (if your RAM allows) at fast speeds. People run smaller models locally on their phones Google has trained their latest models on their own TPUs... not using Nvidia to my knowledge. So, no, there are alternatives. CUDA has the largest mindshare on the training side though.
taminka 3y ago
there are obv alternatives from both intel and amd, performant blas/dnn packages, but small teams don’t use them bc cuda is easier to use and has more support, and larger teams don’t use them bc they have deals w/ nvidia or not enough GPUs are available or they’re after the absolute best performance (which is still nvidia) or bc of other stuff like unstable drivers or smth
towelpluswater 3y ago
Modular Mojo is the most well funded and full of respectable players for making an alternative possible
- pavelstoev 3y ago
  Check out Hidet [1]. Not as well funded, but delivers Python based ML acceleration with GPU support (unlike Mojo). [1] https://github.com/hidet-org/hidet https://github.com/hidet-org/hidet
cwegener 3y ago
Yes. CUDA is still entrenched due to its ease of use when compared to specialised AI hardware SDKs. OpenMP is still a thing in 2024, but I presume that is not the kind of scale you are asking about. If you have MI accelerators, you'll be using ROCm anyway (although AMD contracted Andrzej Janik in 2022 to make ZLUDA run on AMD GPUs. I have no idea what the practical applications of it are at the moment.) The only other serious challenger (apart from the existing GPU manufactures like AMD) who is trying to give Nvidia a run for its money is Tenstorrent and their TT-Buda software kit.

OT but question from someone curious..... is Cuda still entrenched as the only option for doing AI or is there growing support for AMD/Intel/Other ways of doing AI?

BraverHeart 3y ago

George Hotz is attempting to solve this: https://github.com/tinygrad/tinygrad https://github.com/tinygrad/tinygrad

ZoomerCretin 3y ago
He loudly gave up on AMD after they did not fix a blocker he had for 5+ months and gave him the runaround the entire time when he asked for the code to fix it himself. He is still shipping the AMD tinybox with huge warning labels.
- Art9681 3y ago
  Didn't they recently announce that everything was open sourced? Would be cool if he took another look at it once all of the souce code is available (if not already).
  - xjay 3y ago
    > They haven't open sourced anything. They posted a tweet. [1] [1@2024-04-06] https://www.youtube.com/watch?v=j7MRj4N2Cyk&t=429s https://www.youtube.com/watch?v=j7MRj4N2Cyk&t=429s [Twitch] https://twitch.tv/georgehotz https://twitch.tv/georgehotz
- magicalhippo 3y ago
  Randomly stumbled over this[1] post with another fed up open source contributor, due to several serious issues with AMDs GPU drivers and firmware that remain unresolved for years. It also references the geohot decision you mention. Some quotes: I find it incredible that these companies that have large support contracts with you and have invested hundreds of thousands of dollars into your products, have been forced to turn to me, a mostly unknown self-employed hacker with very limited resources to try to work around these bugs (design faults?) in your hardware. In the VFIO space we no longer recommend AMD GPUs at all, in every instance where people ask for which GPU to use for their new build, the advise is to use NVidia. [1]: https://www.reddit.com/r/Amd/comments/1bsjm5a/letter_to_amd_ongoing_amd/ https://www.reddit.com/r/Amd/comments/1bsjm5a/letter_to_amd_...
fwip 3y ago
I'm not sure if he's "attempting to solve it" so much as he's looking for yet another way to keep himself famous. The guy did one good jailbreak for the iPhone, and as near as I can tell, the rest of his work has been a lot of boasting, half-assed hyped-up implementations (e.g: his self-driving car), and trying to befriend other powerful people in tech (see: his promise to single-handedly fix Musk's Twitter). He might be a smart dude, but he vastly overrates his own accomplishments, and doesn't finish near anything he starts.

WithinReason 3y ago

There are some stirrings but don't hold your breath

blackeyeblitzar 3y ago

See my comment on this here: https://news.ycombinator.com/item?id=39973816 https://news.ycombinator.com/item?id=39973816

sigmoid10 3y ago

There are a few attempts here and there in various stages of progression. But right now, nothing matches Nvidia+CUDA in speed and usability.

adam_arthur 3y ago

You can run inference today on pretty much any card. Download Ollama on a modern MacBook and can run 13B and even higher (if your RAM allows) at fast speeds. People run smaller models locally on their phones Google has trained their latest models on their own TPUs... not using Nvidia to my knowledge. So, no, there are alternatives. CUDA has the largest mindshare on the training side though.

taminka 3y ago

there are obv alternatives from both intel and amd, performant blas/dnn packages, but small teams don’t use them bc cuda is easier to use and has more support, and larger teams don’t use them bc they have deals w/ nvidia or not enough GPUs are available or they’re after the absolute best performance (which is still nvidia) or bc of other stuff like unstable drivers or smth

towelpluswater 3y ago

Modular Mojo is the most well funded and full of respectable players for making an alternative possible

pavelstoev 3y ago
Check out Hidet [1]. Not as well funded, but delivers Python based ML acceleration with GPU support (unlike Mojo). [1] https://github.com/hidet-org/hidet https://github.com/hidet-org/hidet

cwegener 3y ago

Yes. CUDA is still entrenched due to its ease of use when compared to specialised AI hardware SDKs. OpenMP is still a thing in 2024, but I presume that is not the kind of scale you are asking about. If you have MI accelerators, you'll be using ROCm anyway (although AMD contracted Andrzej Janik in 2022 to make ZLUDA run on AMD GPUs. I have no idea what the practical applications of it are at the moment.) The only other serious challenger (apart from the existing GPU manufactures like AMD) who is trying to give Nvidia a run for its money is Tenstorrent and their TT-Buda software kit.