3 ms·
Processing is adequate but RAM is not. The 15 Max Pro only has 8GB RAM. You need 32 GB to run any decent sized LLMs. (You can quantize a smaller model but the q
by wenc 3y ago
Processing is adequate but RAM is not. The 15 Max Pro only has 8GB RAM. You need 32 GB to run any decent sized LLMs. (You can quantize a smaller model but the quality of the output will be noticeably bad)
I have an M2 Pro with 16gb ram and even I can’t run any decent sized LLMs locally.
- bpye 3y agoI would bet against Apple shipping any mobile phone with 32GB of RAM - even if the cost were acceptable the increased power draw likely would not be.
- wenc 3y agoThey would probably ship a specialized model trained provide specific answers on device and offload to remote for more general answers. Probably not a full Llama or Falcon. Their latest keyboard autocomplete for instance is a GPT-2 model.