3 ms·
Be more specific - AI is a very broad field. nVidia GPUs have the best inference speed (particularly around SDXL, Hunyuan, Flux, etc), but unless you're buying
by vunderba 2y ago
Be more specific - AI is a very broad field.
nVidia GPUs have the best inference speed (particularly around SDXL, Hunyuan, Flux, etc), but unless you're buying several used 3090s SLI style, you're going to have to split larger LLM GGUFs across main memory and GPU. I'm excluding the RTX 5090 since two of them (plus tax) would basically blow your budget out the water.
With Apple I think you can get up to 192GB of shared memory allowing for very large LLMs.
Another thing is your experience. Unless you want to shell out even more money, you'll likely have to build the PC. It's not hard but it's definitely more work than just grabbing a Mac Studio from the nearest Apple Store.
- zippyman55 2y agoI bought a fully tricked out Mac Studio and had planned to use it but I found I was far too busy in my final career and actually it sits idle. So, while I do not regret my decision based on what I knew at the time, my life trajectory has left it unused. So, I want to get back to doing LLM work on it for fun, Julia and R programming.