2 ms·
They won’t really be going with on-device AI, the models require way too much RAM, but maybe they claim on-device AI “pre-processes” and removes personal inform
by ideamotor 2y ago
They won’t really be going with on-device AI, the models require way too much RAM, but maybe they claim on-device AI “pre-processes” and removes personal information before requests are sent to Apple. Apple has a marketing team to spin this better than my description.
- seanmcdirmid 2y agoI think on-device models would be really useful. Imagine a conversational interface with much less latency so the conversations felt real. I wonder what kind of computing power we will need before we get there (e.g. running an LLM with lots of prompt data + on device speech recognition), maybe 5-10 years?
- Shish2k 2y ago> Imagine a conversational interface with much less latency With current models the latency comes from processing, not from the network — going from a high-power remote server to a low-power local phone is likely to increase latency more than it reduces it
- ec109685 2y agoIt depends on how big the model is. They are using a little LLM to help correct text and that would be dreadful if it took a server round trip.