3 ms·
Also in the “glad I got 64gb” camp - even though it seemed ridiculous when I bought it, technology has advanced so quickly that now it’s actually very useful.
by tomduncalf 3y ago
Also in the “glad I got 64gb” camp - even though it seemed ridiculous when I bought it, technology has advanced so quickly that now it’s actually very useful.
Now I wish I’d bought 4tb rather than 2tb hard drive lol but that’s just me being lazy - that upgrade definitely felt like a step too far.
- cced 3y agoWould you recommend the 64 over 128? What kind of models will 128 open up over 64? Can you get most of them out of the 64?
- tomduncalf 3y agoI’m not sure, not really an expert in this field, just casually interested as an outsider. When I bought my Mac, 64 was as high as you could go.
- SparkyMcUnicorn 3y agoFor LLM models, TheBloke has provided memory requirements for all of their quantisations. For Apple Silicon you want to look at the GGUF models. Here's Mixtral-8x7B-Instruct: https://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-GGUF#provided-files https://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-G... 64GB of memory will struggle on 65-70B (or bigger) models, and you'll be limited to running 70B on Q3 or Q4 if you want to use it somewhat comfortably.