3 ms·

Hopefully them going all in with on-device AI will get them to double their memory offerings.

• threeseed 3y ago
  It is being designed for the iPhone which ships with 6GB RAM. And the idea seems for it to be more of an LAM with fallback to OpenAI for the LLM.
• ideamotor 3y ago
  They won’t really be going with on-device AI, the models require way too much RAM, but maybe they claim on-device AI “pre-processes” and removes personal information before requests are sent to Apple. Apple has a marketing team to spin this better than my description.
  • seanmcdirmid 2y ago
    I think on-device models would be really useful. Imagine a conversational interface with much less latency so the conversations felt real. I wonder what kind of computing power we will need before we get there (e.g. running an LLM with lots of prompt data + on device speech recognition), maybe 5-10 years?
    • Shish2k 2y ago
      > Imagine a conversational interface with much less latency With current models the latency comes from processing, not from the network — going from a high-power remote server to a low-power local phone is likely to increase latency more than it reduces it
      • ec109685 2y ago
        It depends on how big the model is. They are using a little LLM to help correct text and that would be dreadful if it took a server round trip.