2 ms·
I see there are a few options to run inference for LLM and Stable Diffusion outside Nvidia. There is Intel Arc, Apple Ms and now AMD Ryzen AI Max. It is obvious
by mrbonner 2y ago
I see there are a few options to run inference for LLM and Stable Diffusion outside Nvidia. There is Intel Arc, Apple Ms and now AMD Ryzen AI Max. It is obvious that running in Nvidia would be the most optimal way. But given the availability of high VRAM Nvidia cards at reasonable price, I can't stop thinking about getting one that is not Nvidia. So, if I'm not interested in training or fine tuning, would any of those solutions actually works? On a Linux machine?
- 999900000999 2y agoIf you actually want to seriously do this, go with Nvidia. This article is basically Intel saying remember us, we made a GPU! And they make great budget cards, but the ecosystem is just so far behind. Honestly this is not something you can really do on a budget.