2 ms·
With ROCm 6 I think new-ish AMD GPUs are pretty good too. https://ollama.com/blog/amd-preview https://ollama.com/blog/amd-preview With that said, I dont think
by daghamm 3y ago
With ROCm 6 I think new-ish AMD GPUs are pretty good too.
https://ollama.com/blog/amd-preview https://ollama.com/blog/amd-preview
With that said, I dont think you need anything special to run LLMs these days. I can run 7B models on a 4 year old AMD or Intel CPU (no GPU), for programming tasks.
- Havoc 3y agoIndeed...bought AMD shares on the bet that they'll catch up. However this is still true only in a fairly narrow space - inference - once you stray off that very narrow path they're still a fair bit behind