3 ms·

Add to that, that you can't upgrade RAM in most Desktop Macs anymore. I want to buy a Mac soon, and I'm really struggling to decide how much RAM I should order

by MilaM 3y ago

Add to that, that you can't upgrade RAM in most Desktop Macs anymore. I want to buy a Mac soon, and I'm really struggling to decide how much RAM I should order. Unfortunately, my budget is limited. If it wasn't, I would probably go for at least 32GB. I'm still hoping Apple might change their RAM pricing, but probably in vain.

SparkyMcUnicorn 3y ago
> I want to buy a Mac soon, and I'm really struggling to decide how much RAM I should order. I'd recommend getting at least 32GB if you're on the fence. Not being able to upgrade it is a bummer, and your future self will thank you for getting the most you possibly can. For my most recent upgrade I went for 64GB (previously 32GB) and I'm really glad I did, especially since llama.cpp became a thing shortly after getting it.
- tomduncalf 3y ago
  Also in the “glad I got 64gb” camp - even though it seemed ridiculous when I bought it, technology has advanced so quickly that now it’s actually very useful. Now I wish I’d bought 4tb rather than 2tb hard drive lol but that’s just me being lazy - that upgrade definitely felt like a step too far.
  - cced 3y ago
    Would you recommend the 64 over 128? What kind of models will 128 open up over 64? Can you get most of them out of the 64?
    - tomduncalf 3y ago
      I’m not sure, not really an expert in this field, just casually interested as an outsider. When I bought my Mac, 64 was as high as you could go.
    - SparkyMcUnicorn 3y ago
      For LLM models, TheBloke has provided memory requirements for all of their quantisations. For Apple Silicon you want to look at the GGUF models. Here's Mixtral-8x7B-Instruct: https://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-GGUF#provided-files https://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-G... 64GB of memory will struggle on 65-70B (or bigger) models, and you'll be limited to running 70B on Q3 or Q4 if you want to use it somewhat comfortably.

by MilaM 3y ago

SparkyMcUnicorn 3y ago

> I want to buy a Mac soon, and I'm really struggling to decide how much RAM I should order. I'd recommend getting at least 32GB if you're on the fence. Not being able to upgrade it is a bummer, and your future self will thank you for getting the most you possibly can. For my most recent upgrade I went for 64GB (previously 32GB) and I'm really glad I did, especially since llama.cpp became a thing shortly after getting it.

tomduncalf 3y ago
Also in the “glad I got 64gb” camp - even though it seemed ridiculous when I bought it, technology has advanced so quickly that now it’s actually very useful. Now I wish I’d bought 4tb rather than 2tb hard drive lol but that’s just me being lazy - that upgrade definitely felt like a step too far.
- cced 3y ago
  Would you recommend the 64 over 128? What kind of models will 128 open up over 64? Can you get most of them out of the 64?
  - tomduncalf 3y ago
    I’m not sure, not really an expert in this field, just casually interested as an outsider. When I bought my Mac, 64 was as high as you could go.
  - SparkyMcUnicorn 3y ago
    For LLM models, TheBloke has provided memory requirements for all of their quantisations. For Apple Silicon you want to look at the GGUF models. Here's Mixtral-8x7B-Instruct: https://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-GGUF#provided-files https://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-G... 64GB of memory will struggle on 65-70B (or bigger) models, and you'll be limited to running 70B on Q3 or Q4 if you want to use it somewhat comfortably.