3 ms·
What's everyones recommendation for one to run a good local LLM model on?
by siavosh 1mo ago
What's everyones recommendation for one to run a good local LLM model on?
- bigyabai 1mo agoIf you don't want to wait for prefill, you're going to want a CUDA dGPU system.
- manmal 1mo ago1-2 RTX5090 will be better value than Macs because they have the memory bandwidth for somewhat fast local inference.
- asimovDev 1mo agoand they will become obsolete much slower if at all compared to when the mac will be barely usable once macos stop supporting it in 8 years or so. although maybe by that time there will be local models that you could run straight on the same mac that will reverse engineer it for you and write linux drivers