4 ms·
Siri isn't that slow for me generally, but the responses are relatively primitive compared to GPT. Very likely Siri is lagging GPT based agents due to the high
by abalone 2y ago
Siri isn't that slow for me generally, but the responses are relatively primitive compared to GPT. Very likely Siri is lagging GPT based agents due to the high processing cost of LLM queries.
Startups can launch with today's GPT agents because their utilization is relatively tiny. Siri operates at planet scale, plus Apple is committed to on-device privacy, so they need to develop highly optimized on-device models.
No doubt this is going to drive a major hardware upgrade cycle for Apple.
- slau 2y agoOnly extremely basic requests are processed on device, like “what time is it”. Anything else is still processed on Apple’s servers.
- abalone 2y agoI think “intelligent search” is going to build on Siri and Spotlight, adding LLM for things like summaries and suggestions. This should bring a 10x magnitude of improvement and justify hardware upgrades to satiate the RAM and coprocessor thirst of LLMs. Spotlight does pretty extensive if “traditional” index-based search of your local personal data, along with certain privacy-preserving web queries. You can imagine what on-device LLM-powered summaries would bring to this type of search and suggested actions.