4 ms·
Disappointing it wouldn't be available on the 14 pro or 15 pro, both of which have hardware that far exceeds their normal use cases. I'd be happy if it could st
by kylehotchkiss 3y ago
Disappointing it wouldn't be available on the 14 pro or 15 pro, both of which have hardware that far exceeds their normal use cases. I'd be happy if it could still go to the network for some of the processing.
- wenc 3y agoProcessing is adequate but RAM is not. The 15 Max Pro only has 8GB RAM. You need 32 GB to run any decent sized LLMs. (You can quantize a smaller model but the quality of the output will be noticeably bad) I have an M2 Pro with 16gb ram and even I can’t run any decent sized LLMs locally.
- bpye 3y agoI would bet against Apple shipping any mobile phone with 32GB of RAM - even if the cost were acceptable the increased power draw likely would not be.
- wenc 3y agoThey would probably ship a specialized model trained provide specific answers on device and offload to remote for more general answers. Probably not a full Llama or Falcon. Their latest keyboard autocomplete for instance is a GPT-2 model.