4 ms·
Its not what you asked, but on the off chance you use local LLM’s for anything, getting a macbook with unified memory is a great price/performance ratio without
by kingkongjaffa 1y ago
Its not what you asked, but on the off chance you use local LLM’s for anything, getting a macbook with unified memory is a great price/performance ratio without spending tons on graphics cards.
Personally if I was in the market for a new laptop I would be upgrading my 16gb M1 macbook pro, to whatever the latest MBP is with as much RAM as I could afford.
16gb can run 14b models slowly and they are semi usable, good for small tasks.
24gb or 32gb can get you into 27b models that start getting pretty usable.
- khurs 1y agoThe tokens per seconds are not great and fans can start to run (I have a M1 Max 64gb and tried 32b on ollama, then went down to 14b for speed)