4 ms·
Isn't it much better to get a Mac Studio with an M2 Max and 192gb of Ram and 31 terraflops for $6599 and run llama.cpp?
by narrator 3y ago
Isn't it much better to get a Mac Studio with an M2 Max and 192gb of Ram and 31 terraflops for $6599 and run llama.cpp?
- magic_hamster 3y agoMacs don't support CUDA which means all that wonderful hardware will be useless when trying to do anything with AI for at least a few years. There's Metal but it has its own set of problems, biggest one being it isn't a drop in CUDA replacement.
- doublepg23 3y agoI'm assuming this won't support CUDA either?
- adam_arthur 3y agoYou can do LLM inference without CUDA just fine. Download Ollama and see for yourself
- egorfine 3y agoFor LLM inference - yes absolutely.