4 ms·
Why couldn't the AI be done on-device rather than server side? Isn't that where things are going with mobile hardware?
by ugh123 2y ago
Why couldn't the AI be done on-device rather than server side? Isn't that where things are going with mobile hardware?
- unsatchmo 2y agoMobile devices don’t have the memory or the memory bandwidth to run an LLM that’s big enough to be good at much. Plus the fixed battery and thermal constraints.
- bendews 2y agomy M1 Macbook air can run LLM's pretty well.. worse specs than the latest iPad Pro (and iPhone pro wouldn't be too far behind).
- nkozyra 2y agoRunning them is a whole lot less resource intensive than training them. Unless the plan was just to build a RAG source from your personal data, in which case it would be yet another underwhelming feature.
- ugh123 2y agoI guess that will never change huh