3 ms·
I get that they're wary of an LLM-based Siri making embarrassing mistakes. It would be a bad look if some kid's phone provided helpful instructions for killing
by jiggawatts 4y ago
I get that they're wary of an LLM-based Siri making embarrassing mistakes. It would be a bad look if some kid's phone provided helpful instructions for killing themselves when asked. Mummy and Daddy would sue.
However, directly answering queries is not the only way LLMs can be used. In fact, when ChatGPT 4 came out, the first thing I thought of was: "Wow, this could make Siri so much better!"
For example, an LLM like GPT coupled with voice recognition was used to create Whisper, an AI that has nearly perfect text-from-audio recognition. One of my biggest gripes with Siri is that it is basically useless in a car because even slight background noise confuses its voice recognition. An LLM would fix this.
Another point is that many people don't realise that LLMs re-read their entire input for every word they generate! Their writing speed is so-so, but they can read really fast even when running on mobile device hardware. Think 10K to 100K words per second. An LLM could read through all of the text on your device when prompted for search queries in a fraction of a second. As long as this was carefully set up, it wouldn't be able to produce "bad output", because it would just be matching data to your prompt.
E.g.: Imagine GPT being prompted with: "Does this email match the query <q>? Say only YES if it does or NO if it does not. <email>"
It doesn't matter if it occasionality hallucinates and outputs gibberish, you just mark that as a "NO" and move on. This is also very easy to train out of a specialised version of the model using reinforcement learning.
PS: I just played around with GPT 4 to see how it behaves when asked to recognise requests for creating calendar entries, and it's pretty good. For example, it can correctly compute things like "next long weekend". Interestingly, ChatGPT 4 is already doing some similar prompt injection, and I can't override its sense of "current time".
Apple's Siri team has failed really badly. Everyone else is sprinting away while they're not even aware there's a race going on.
- caycep 4y agoI think reading the Information article, there were 2 issues, which it wasn't good at separating. The AI model sophistication and usage are one - and yeah, I think they could be more creative in using LLMs and such. But the other one seems to be a back end issue, i.e. that brief mention of "clunky database slowing down iterations/upgrades". That always seems to be an Apple Achilles heel...and I'd hope also they are NOT using that terrible IT infrastructure company (Agilent or whatever) to outsource their backend...
- circuit10 4y ago> Another point is that many people don't realise that LLMs re-read their entire input for every word they generate! Not every word, for every generation (which on something like ChatGPT is a message) > they can read really fast even when running on mobile device hardware. Think 10K to 100K words per second That’s definitely not right, I’ll try to find some numbers
- jiggawatts 4y agoPlease do, I was about to include that statement in a presentation, so if I’m wrong I’d love to be corrected!
- circuit10 4y agoI’m really struggling to find any actual numbers so I could be very wrong but I vaguely remember input tokens being around 10x faster Basing it on pricing doesn’t seem like a good way to estimate it because it varies so much but: - OpenAI charges the same for input and output tokens, except for GPT4 where output tokens cost twice as much as inputs - textsynth.org charges 25 times as much for outputs than for inputs - goose.ai doesn’t charge for inputs Also you have limited context length so that limits how much data you can read
- mike_hearn 4y agoI don't think that's quite correct. The underlying algorithms start by processing the prompt tokens in parallel, and once that's processed, the intermediate KV vectors are re-used for each new token prediction by the feed-forward. What you may be thinking of is that once the model stops predicting at the end of an API call the state is dropped from RAM, and if you go back with another message in a chat, then the LLM will re-read the chat log up to that point as part of the prompt. So this is true on a per-message basis, but not on a per-word or per-token basis.