4 ms·
AI noob here. Is every single model in iOS really just a thin adapter on top of one base model? Can everything they announced today really be built on top of on
by onesociety2022 2y ago
AI noob here. Is every single model in iOS really just a thin adapter on top of one base model? Can everything they announced today really be built on top of one base LLM model with a specific type of architecture? What about image generation? What about text-to-speech?
If they’re obviously different models, they can’t load them all at once into RAM. If they have to load from storage every time an app is opened, how will they do this fast enough to maintain low latency?
- wmf 2y agoThe main LLM is only 1.5 GB so it should only take a half second to load. Or they could keep it loaded. The other models may be even smaller.
- glial 2y agoMaybe they use the "Siri is waking up and the screen wabbles" animation time for loading the model. That would be clever.
- mholm 2y agoThey'll have plenty of time to load the model; It still needs to wait for the user to actually voice/type their request. Invoking Siri happens well before the request is ready.