3 ms·
I started buying Macs with more memory, no regrets. An M4 Max with 64GB (in a laptop, no less!) runs most small models comfortably (but get 96GB or more if you
by jwr 2y ago
I started buying Macs with more memory, no regrets. An M4 Max with 64GB (in a laptop, no less!) runs most small models comfortably (but get 96GB or more if you really intend to use 70B models regularly). And when I'm not running LLMs, the memory is useful for other stuff.
- ignoramous 2y ago> And when I'm not running LLMs, the memory is useful for other stuff Let's be honest, the other stuff is just Chrome: Tell me 96gb is enough?
- simonw 2y agoFirefox and VS Code for me: I have 64GB and I can't run Llama 70B locally without closing a ton of windows and tabs first!
- LtdJorge 2y agoHave you tried auto tab discard?
- user_7832 2y agoNot the person you replied to but auto discard only helps so much. Even with it on I often use 80-90% memory (albeit on Windows, not sure if macOS is any different in this regard).
- bornfreddy 2y agoFirefox here too, no such problems. I do however run Google services in a separate browser (Brave or Chromium) because most of them hog the Firefox browser. For example running earth.google.com on FF is... a very special experience. :-/