3 ms·

They are adding their first LLM in iOS 17's keyboard.

by asadm 3y ago

gnicholas 3y ago
This will be great. Hopefully it will be able to figure out that it should capitalize my last name, which is on my Contact card, and that of a dozen of my relatives. When I went to the Apple Store, they told me that I could either add a custom autocorrect to my library, or reset all settings. They did admit this was some sort of bug, and that it would be massive overkill to reset all settings (lose all wifi passwords, etc.).
- coder543 3y ago
  > or reset all settings > that it would be massive overkill to reset all settings (lose all wifi passwords, etc.). I don't think that's what anyone was recommending... Settings -> General -> Transfer or Reset iPhone -> Reset -> Reset Keyboard Dictionary is almost certainly what they were recommending. What does resetting your keyboard dictionary have to do with your wifi passwords?
  - gnicholas 3y ago
    Nope, the two employees I spoke to were talking about a full reset (which affects network settings). Regardless of what the Keyboard Dictionary says, the iPhone should be autocompleting/capitalizing the last name of contacts, and especially the owner's name.
    - coder543 3y ago
      Why would that implicitly be "regardless" of what the keyboard dictionary says? I would expect the learned dictionary to be prioritized over other sources of information, just as a practical matter, even if someone might reasonably assume there are other things that should be prioritized over it. None of that explains how resetting everything would have any affect on capitalization of names if resetting the keyboard dictionary wouldn't, and you didn't say whether you tried resetting keyboard dictionary.
      - gnicholas 3y ago
        Nope, they didn't suggest that. After all, if the dictionary is 'above' the contact list, then resetting the dictionary wouldn't fix the problem (since the word would be coming from the contact list itself, not the dictionary). The issue was that iOS was not properly accounting for the contact list. They said that hopefully it will be fixed in a future iOS update, and said I could submit a bug fix online.
        coder543 3y ago
        > The issue was that iOS was not properly accounting for the contact list. But the autocorrect does account for the contact list. I can personally vouch for that, as it works properly on my phone. > > I would expect the learned dictionary to be prioritized > After all, if the dictionary is 'above' the contact list, then resetting the dictionary wouldn't fix the problem (since the word would be coming from the contact list itself, not the dictionary). "The dictionary" is not what I said. But, even if I had said that, then any words that aren't in the dictionary would be sourced from the contact list and other lower priority sources, wouldn't they? What I said was the learned dictionary. The dictionary made of words learned from what the user types, when the user corrects the autocorrect. iOS does not allow you to access or modify the list of words that it has learned, only to clear them. The words that it learns seem to be prioritized above everything else, which includes special capitalizations for words (not just sequences of letters lacking capitalization). Sometimes it learns the wrong capitalization for a word, and it sticks to it. Just searching for "ios capitalizing wrong words" on google turns up a ton of results from reddit and other discussion forums that mention this exact problem with iOS learning the wrong capitalization. FWIW, iOS 17 sounds like an all-new autocorrect system, so maybe it will fix your problem just by throwing out the data anyways, since it would make sense to me for them to start from scratch with such a radically different autocorrect algorithm.
        gnicholas 3y ago
        > But the autocorrect does account for the contact list. I can personally vouch for that, as it works properly on my phone. I'm glad it works on your phone. It doesn't work on mine! I was aware of the upcoming changes in iOS17, which is why I'm not making changes and hoping that it will be fixed. It is annoying to have to manually capitalize my own name!
cubefox 3y ago
"L"LM. No way this model will be "large" in any modern sense. For mobile devices both RAM and power consumption are very limited.
- candiodari 3y ago
  Some terms don't age well. "Deep" neural networks used to mean "more than one layer". Ie. 5 emulated neurons or more. What makes Language Models large, originally, was meant a model that emulates behavior that before needed to be programmed in explicitly, as size increased. The first time this was observed and described, it was in GPT-2, with 1.5B parameters. You can do this on current mobile devices with current technology (ie. 4-bit quantization).
  - cubefox 3y ago
    But such small models are too dumb to be acceptable for conversation. Far more likely that there will be an API call to an actually large model in the cloud. If you really don't have any network connection, you are simply out of luck. It's not as if transferring a little bit text would require a lot of bandwidth anyway.
yreg 3y ago
That's just for autocorrect, right? I would like to see an LLM in Siri and eventually even have it interact with/control the rest of the system. Ideally with Whisper-level speech recognition of course.
- bigfudge 3y ago
  Apples speech recognition is pretty good, at least for me. I always assumed the delta was because it does it in near real time, which is not possible with whisper.
  - yreg 3y ago
    I don't have a great accent, but Whisper understands me >99%. So do my colleagues. I've tried to talk to ChatGPT through a Siri shortcut for a day and Siri transcribed pretty much all of my requests wrong, to the point that GPT seldom understood what I want. Even the Hey Siri … Ask ChatGPT trigger phrase fails ~50% of the time for me.
    - deleted 3y ago
      [deleted]
  - coder543 3y ago
    Siri speech recognition is consistently terrible compared to the alternatives, in my experience. Google and Microsoft have much better speech recognition technology. Whisper is phenomenal by comparison Siri and arguably even what Google and Microsoft use, and no, there is nothing that stops Whisper from being used in real time. I can run real-time Whisper on my iPhone using the Hello Transcribe app, but the last time I tried it, the app itself was too flawed to recommend besides as a demo of real-time transcription. I am looking forward to trying out the new transcription model that Apple is bringing to iOS 17.
- coder543 3y ago
  Apple is supposedly bringing a better, transformer-based speech recognition model to iOS 17 as well, although I don't think either of these transformer models would be classified as a large language model. Link to the announcement timestamp: https://developer.apple.com/videos/play/wwdc2023/101/?time=1726 https://developer.apple.com/videos/play/wwdc2023/101/?time=1...

They are adding their first LLM in iOS 17's keyboard.

by asadm 3y ago

They are adding their first LLM in iOS 17's keyboard.

This will be great. Hopefully it will be able to figure out that it should capitalize my last name, which is on my Contact card, and that of a dozen of my relatives. When I went to the Apple Store, they told me that I could either add a custom autocorrect to my library, or reset all settings. They did admit this was some sort of bug, and that it would be massive overkill to reset all settings (lose all wifi passwords, etc.).

coder543 3y ago
> or reset all settings > that it would be massive overkill to reset all settings (lose all wifi passwords, etc.). I don't think that's what anyone was recommending... Settings -> General -> Transfer or Reset iPhone -> Reset -> Reset Keyboard Dictionary is almost certainly what they were recommending. What does resetting your keyboard dictionary have to do with your wifi passwords?
- gnicholas 3y ago
  Nope, the two employees I spoke to were talking about a full reset (which affects network settings). Regardless of what the Keyboard Dictionary says, the iPhone should be autocompleting/capitalizing the last name of contacts, and especially the owner's name.
  - coder543 3y ago
    Why would that implicitly be "regardless" of what the keyboard dictionary says? I would expect the learned dictionary to be prioritized over other sources of information, just as a practical matter, even if someone might reasonably assume there are other things that should be prioritized over it. None of that explains how resetting everything would have any affect on capitalization of names if resetting the keyboard dictionary wouldn't, and you didn't say whether you tried resetting keyboard dictionary.
    - gnicholas 3y ago
      Nope, they didn't suggest that. After all, if the dictionary is 'above' the contact list, then resetting the dictionary wouldn't fix the problem (since the word would be coming from the contact list itself, not the dictionary). The issue was that iOS was not properly accounting for the contact list. They said that hopefully it will be fixed in a future iOS update, and said I could submit a bug fix online.
      - coder543 3y ago
        > The issue was that iOS was not properly accounting for the contact list. But the autocorrect does account for the contact list. I can personally vouch for that, as it works properly on my phone. > > I would expect the learned dictionary to be prioritized > After all, if the dictionary is 'above' the contact list, then resetting the dictionary wouldn't fix the problem (since the word would be coming from the contact list itself, not the dictionary). "The dictionary" is not what I said. But, even if I had said that, then any words that aren't in the dictionary would be sourced from the contact list and other lower priority sources, wouldn't they? What I said was the learned dictionary. The dictionary made of words learned from what the user types, when the user corrects the autocorrect. iOS does not allow you to access or modify the list of words that it has learned, only to clear them. The words that it learns seem to be prioritized above everything else, which includes special capitalizations for words (not just sequences of letters lacking capitalization). Sometimes it learns the wrong capitalization for a word, and it sticks to it. Just searching for "ios capitalizing wrong words" on google turns up a ton of results from reddit and other discussion forums that mention this exact problem with iOS learning the wrong capitalization. FWIW, iOS 17 sounds like an all-new autocorrect system, so maybe it will fix your problem just by throwing out the data anyways, since it would make sense to me for them to start from scratch with such a radically different autocorrect algorithm.
        gnicholas 3y ago
        > But the autocorrect does account for the contact list. I can personally vouch for that, as it works properly on my phone. I'm glad it works on your phone. It doesn't work on mine! I was aware of the upcoming changes in iOS17, which is why I'm not making changes and hoping that it will be fixed. It is annoying to have to manually capitalize my own name!

cubefox 3y ago

"L"LM. No way this model will be "large" in any modern sense. For mobile devices both RAM and power consumption are very limited.

candiodari 3y ago
Some terms don't age well. "Deep" neural networks used to mean "more than one layer". Ie. 5 emulated neurons or more. What makes Language Models large, originally, was meant a model that emulates behavior that before needed to be programmed in explicitly, as size increased. The first time this was observed and described, it was in GPT-2, with 1.5B parameters. You can do this on current mobile devices with current technology (ie. 4-bit quantization).
- cubefox 3y ago
  But such small models are too dumb to be acceptable for conversation. Far more likely that there will be an API call to an actually large model in the cloud. If you really don't have any network connection, you are simply out of luck. It's not as if transferring a little bit text would require a lot of bandwidth anyway.

yreg 3y ago

That's just for autocorrect, right? I would like to see an LLM in Siri and eventually even have it interact with/control the rest of the system. Ideally with Whisper-level speech recognition of course.

bigfudge 3y ago
Apples speech recognition is pretty good, at least for me. I always assumed the delta was because it does it in near real time, which is not possible with whisper.
- yreg 3y ago
  I don't have a great accent, but Whisper understands me >99%. So do my colleagues. I've tried to talk to ChatGPT through a Siri shortcut for a day and Siri transcribed pretty much all of my requests wrong, to the point that GPT seldom understood what I want. Even the Hey Siri … Ask ChatGPT trigger phrase fails ~50% of the time for me.
  - deleted 3y ago
    [deleted]
- coder543 3y ago
  Siri speech recognition is consistently terrible compared to the alternatives, in my experience. Google and Microsoft have much better speech recognition technology. Whisper is phenomenal by comparison Siri and arguably even what Google and Microsoft use, and no, there is nothing that stops Whisper from being used in real time. I can run real-time Whisper on my iPhone using the Hello Transcribe app, but the last time I tried it, the app itself was too flawed to recommend besides as a demo of real-time transcription. I am looking forward to trying out the new transcription model that Apple is bringing to iOS 17.
coder543 3y ago
Apple is supposedly bringing a better, transformer-based speech recognition model to iOS 17 as well, although I don't think either of these transformer models would be classified as a large language model. Link to the announcement timestamp: https://developer.apple.com/videos/play/wwdc2023/101/?time=1726 https://developer.apple.com/videos/play/wwdc2023/101/?time=1...