14 ms·

Personalized Hey Siri

117 points by dantiberian 9y ago

vormalpg 8y ago
While not a new problem at all, this is a perfectly good application of machine learning in support of a useful feature. The article is more direct and matter of fact than you'd expect from a hype obsessed company like Apple, suggesting that an engineer is in charge of these blog posts. I'd imagine that the marketing team isn't too thrilled with something as mundane as Siri being able to recognize the device owner's voice being portrayed as an achievement when she's supposed to have solved world hunger by now, but I found it refreshing.
smortaz 8y ago
On a separate note, I have yet to get Siri to transcribe "Isle of Dogs" correctly. It's always "I love dogs" which is quite ok by me :).
- gok 8y ago
  For phonetically ambiguous utterances like this, it usually helps continuous speech recognition systems if you pause between words like “Isle. Of. Dogs.”
  - smortaz 8y ago
    Yep, tried that too... but admittedly the sound very similar. I keep getting "I'll of dogs" and "I love dogs".
- duskwuff 8y ago
  The title of that movie was chosen specifically to be a homophone for "I love dogs", so Siri's confusion is understandable. :)
  - smortaz 8y ago
    Ah! Makes total sense now.
  - unfunco 8y ago
    It's also a place in the east end of London and has been for almost 500 years: https://en.wikipedia.org/wiki/Isle_of_Dogs https://en.wikipedia.org/wiki/Isle_of_Dogs
  - sowbug 8y ago
    In 1992 Apple gave me a T-shirt for supplying speech samples used to train their System 7 voice recognition. Below a picture of bunch of apple cores strewn on a beach was the caption "I helped Apple wreck a nice beach!"
    - dkonofalski 8y ago
      That's amazing. "I helped Apple recognize speech" for those struggling.
  - evincarofautumn 8y ago
    This is an interesting case because they’re phonemically the same (long-I, L, schwa, V) but may be realised differently phonetically depending on your accent†. I bring this up because I’ve always wanted to be able to tell speech recognition systems specific features of my accent, say through a series of questions (“Do you pronounce ‘cot’ and ‘caught’ the same or differently?”, &c.) in addition to ML training, because as it is, they persistently mix up certain words—“I’ll” and “all”, for instance. † With “isle of”, I have two syllables and a velarised (“dark”) L in “isle”, a mid-central vowel (/ə/) in “of”; with “I love”, I have one syllable in “I”, a non-velarised (“light”) L and an open-mid back unrounded vowel (/ʌ/) in “love”.
- Angostura 8y ago
  I just tried 'Hey Siri where is 'Isle of Dogs' playing - and it got it first time.
  - gok 8y ago
    That’s easier because the language model knows that “where is I love dogs playing” doesn’t make sense grammatically, whereas “where is <movie name> playing” does. On the other hand, just saying “I love dogs” might be a reasonable thing you may want to text message or something.
    - ascagnel_ 8y ago
      You could also give the recognition system an additional hint by dropping in "the movie", "the play", etc. -- "where is the movie 'Isle of Dogs' playing?", "where is the play 'Death of a Salesman' playing?"
- lozaning 8y ago
  I was testing some Alexa stuff for work the other day and named a light "Lighty McLight Face". All Alexa would ever hear was "lightly lick my face".
- TheSpiceIsLife 8y ago
  You should be able to say "Hey Siri, listen carefully" brief pause "isle ... of ... dogs", or some such to have the device enter a special mode where it attempts to identify individual words more accurately.
  - anonfunction 8y ago
    I've never heard about this special mode.
    - TheSpiceIsLife 8y ago
      I meant to imply it should have this mode, not that it does.
  - eridius 8y ago
    What's wrong with just saying "isle … of … dogs"? Why would you need to enter a special mode before enunciating?
    - pwinnski 8y ago
      Pausing long enough to be sure the AI knows you're done pronouncing the previous words is often enough to trigger the end of the phrase. We want our AI assistant to be performant, after all! Without a special mode, you'd likely say "Hey, Siri, play Isle..." "Okay, I found--" "Dammit, Siri, I wasn't done!"
      - eridius 8y ago
        You don't have to wait 5 seconds between words. You just have to enunciate clearly enough that the divisions between words become truly unambiguous.
        TheSpiceIsLife 8y ago
        It’s worth noting that, elsewhere in this thread, user smortaz wrote: Yep, tried that too... but admittedly the sound very similar. I keep getting "I'll of dogs" and "I love dogs".
- __david__ 8y ago
  I've never gotten Siri to successfully play "Cuz It's Hot" by "My Life With The Thrill Kill Kult". Also the only place I have the song is "WaxTrax! Black Box: The First 13 Years (Disc 3)", which is also pretty much impossible to get Siri to play. Sigh.
- dkonofalski 8y ago
  You can tap anything that Siri hears you say and fix it. Unfortunately, that means you might end up with the opposite problem in the future.
tomphoolery 8y ago
I'm not convinced this Siri thing is really that revolutionary until I can invoke something on my phone by screaming "AYYYYYYY MUST BE THE SIRIIIIII"
zython 8y ago
related: https://maachinelearning.apple.com/2017/08/06/siri-voices.html https://maachinelearning.apple.com/2017/08/06/siri-voices.ht...
- sp332 8y ago
  Your link has an extra "a" in it. https://machinelearning.apple.com/2017/08/06/siri-voices.html https://machinelearning.apple.com/2017/08/06/siri-voices.htm...
gervase 8y ago
To clarify, this is about training your phone to recognize only "Hey Siri" as spoken by you, as opposed to letting you personalize your trigger phrase to something other than "Hey Siri".
- mattigames 8y ago
  Yeah, they don't want to give users too much control; Siri is an important brand by itself and Apple doesn't want people going around skipping it.
  - draw_down 8y ago
    Shhhhhh
  - woolvalley 8y ago
    It's too bad, because I sure wish I could make the homepod use a different phrase. The homepod knows to activate when I say hey siri, but my friends phone activates instead of the homepod whenever they say 'hey siri' about %50 of the time.
    - deleted 8y ago
      [deleted]
  - Raphmedia 8y ago
    At least Siri is easy to say internationally. I never manage to use the proper accent when I say "Okay Google".
  - saagarjha 8y ago
    The non-cynical viewpoint would be that "Hey Siri" is an easy phrase to detect because of how it's said, which isn't really that similar to many other phrases.
    - gok 8y ago
      “Hey Siri” actually a pretty challenging trigger phrase. It’s short (few phonemes), and it’s phonetically confuseable with phrases like “Hey Serial is out” (could be “Hey Siri all is out”).
pwython 8y ago
Cool little insight into their progress. I'm excited for the day Siri can differentiate my voice and my wife's so we can use a single HomePod with two iCloud accounts. For example, Siri reading off calendar appointments from an iCloud account based on who's asking.
- AlphaWeaver 8y ago
  This feature is one of my favorite things about the Google Home.
- jorvi 8y ago
  That feature is not blocked by technical but by arbitrary limits. Whether it is a family iPad, Apple TV or HomePod, Apple strongly believes in 1:1 device-person (or more accurate, device-iCloud account) mapping.
  - geerlingguy 8y ago
    Macs do multi account quite well, and iOS/iPad does it for schools. I’m not sure why Apple hasn’t at least made iPads for consumers have the same multi-iCloud-account feature available, it would make things a lot easier at home!
  - dkonofalski 8y ago
    The HomePod was designed not to be a single-user device. Especially after reading the OP article, I think it's more of a technical limit than anything because you could potentially give away information that didn't belong to a user if the user isn't detected correctly.
snootyresearch 8y ago
God this is depressing. Only 3 years behind Amazon and Google.
- deleted 8y ago
  [deleted]
- __david__ 8y ago
  Really? I'm pretty sure "Alexa" is known for responding to the TV.
  - gaius 8y ago
    Yep https://www.theregister.co.uk/2017/01/07/tv_anchor_says_alexa_buy_me_a_dollhouse_and_she_does/ https://www.theregister.co.uk/2017/01/07/tv_anchor_says_alex...
  - pwinnski 8y ago
    When it comes to Apple being behind in machine learning, perception is all that matters. Technical explanations about how Apple makes choices that preserve privacy are irrelevant. Actual head-to-head tests showing that Apple is ahead in some areas and behind in others are irrelevant. All that matters is that we all know that Apple is behind in machine learning.
chongli 8y ago
I would love this feature if they found a way to implement it without draining the battery by listening constantly and trying to recognize 'Hey Siri' from all the background noise. It's especially bad when you play music nearby!
- gok 8y ago
  Are you sure you’re actually getting your battery drained from this? The phrase spotting happens in a super low power processor which can run without the rest of the phone doing anything.
  - chongli 8y ago
    I remember one day I played music through some speakers connected to my phone for about 6 hours and the battery usage showed 40% for music and 40% for Siri. I put two and two together and figured the music was keeping Siri awake.
brd529 8y ago
My biggest problem with the homepod is that my 5 year old has learned that he can turn on _any song he wants_ whenever he wants by yelling "hey siri." He loves it! He yells at the dinner table, when his little sister is asleep, and when we are on the phone. His favorite phrase is "Hey Siri, set volume to 100%." His favorite song is the PJ Masks song, which Siri dutifully plays when called upon. Every. Single. Time. I'd love it if HomePod recognized who was speaking, but I'd settle for Siri just recognizing "this is a kids voice" and ignoring it!
- aaronmiler 8y ago
  I feel like this is a big missing piece for voice activated things. I've got an Echo device ~10ft from my TV, and it gets activated from Netflix (Parks & Rec mostly... I blame Rob Lowe) all the time. I should be able to go into the app and label it as a false activation so it can ignore it in the future
  - Art9681 8y ago
    The app already has a feature for that. It will keep a list of all the times it was activated and a recording you can play back of what phrase activated it. You can also tap a button to send feedback if it activated with the wrong phrase.
- Endama 8y ago
  I'm sorry, this warmed my heart and terrified me at the same time. A hilariously adorable and obnoxious use of technology.
- StephenMelon 8y ago
  Unfortunately “Hey Siri” starts to grate the 70th time you say it in 3 days. I would actually now rather walk across the room to touch the Homepod than have to say the phrase so much. It’s still in uncanny valley territory a lot of the time as it isn’t smart enough to keep context between interactions or react in a human-like way. I think a retroactive activation feature would be the way to go, ie. be able to say “Siri” at the end of a sentence and have it parse the phrase that came before. It feels more natural to say “skip this please Siri” than “Hey Siri, skip” because it lets you amend the mistake of not identifying the subject earlier in the sentence rather than the phrase-exasperation-hey Siri-phrase. I asked Siri to DJ and my daughter said “hey Siri, I don’t like that song”. Siri said “I’ll remember that” and played something else. Now I have no idea how to amend Siri’s notion that I dislike one of my favourite tracks. There should really be a Homepod app where you can see recent interactions and correct misinterpreted ones. There are also some artists and tracks with unusual spellings that are always going to cause problems for a voice interface and so they need a solution for that.
srcmap 8y ago
What's the technical reason to limit the wake words to only "Hey Siri", "OK Google", etc? Should one be able to "Personalized" to his/her own phase PER Device such as "Hey Commander Data" for Iphone 6, "Hey Counselor Troi " for IPad Pro or "Ok Skynet" for the google phone? That should also solve the problem where someone said "OK Google" in TV Shows/Podcast and wakeup all the google devices. With the latest "Deep Voice" tech, the device should be able to response with the actors' voice too. I remember seeing a clip of Chinese TV show when the actress said (in Chinese) "Honey, Honey where are you?" and the phone on chair wake up and said "Honey, I am here, I am here. " And tons of folks in the youtube comments ask "What kind of phone is that and I want it!". I told my wife that and she immediately said she wanted it too. I still don't know which brand of Chinese phone actually has that features or if that was just imagination of the TV show writer. Someone should build and and I know my wife wants it.
- pwinnski 8y ago
  I don't work on such projects, but on other ML projects, and I suspect the answer is: training data. There are many, many, many samples of people saying the approved trigger phrases. Many, many, many samples. To use your own custom trigger phrases, you would likely need to say it thousands of times to being to approach the accuracy the current systems ship with out of the box. Maybe tens of thousands. That's just my conjecture, though.

117 points by dantiberian 9y ago

While not a new problem at all, this is a perfectly good application of machine learning in support of a useful feature. The article is more direct and matter of fact than you'd expect from a hype obsessed company like Apple, suggesting that an engineer is in charge of these blog posts. I'd imagine that the marketing team isn't too thrilled with something as mundane as Siri being able to recognize the device owner's voice being portrayed as an achievement when she's supposed to have solved world hunger by now, but I found it refreshing.

smortaz 8y ago

On a separate note, I have yet to get Siri to transcribe "Isle of Dogs" correctly. It's always "I love dogs" which is quite ok by me :).

gok 8y ago
For phonetically ambiguous utterances like this, it usually helps continuous speech recognition systems if you pause between words like “Isle. Of. Dogs.”
- smortaz 8y ago
  Yep, tried that too... but admittedly the sound very similar. I keep getting "I'll of dogs" and "I love dogs".
duskwuff 8y ago
The title of that movie was chosen specifically to be a homophone for "I love dogs", so Siri's confusion is understandable. :)
- smortaz 8y ago
  Ah! Makes total sense now.
- unfunco 8y ago
  It's also a place in the east end of London and has been for almost 500 years: https://en.wikipedia.org/wiki/Isle_of_Dogs https://en.wikipedia.org/wiki/Isle_of_Dogs
- sowbug 8y ago
  In 1992 Apple gave me a T-shirt for supplying speech samples used to train their System 7 voice recognition. Below a picture of bunch of apple cores strewn on a beach was the caption "I helped Apple wreck a nice beach!"
  - dkonofalski 8y ago
    That's amazing. "I helped Apple recognize speech" for those struggling.
- evincarofautumn 8y ago
  This is an interesting case because they’re phonemically the same (long-I, L, schwa, V) but may be realised differently phonetically depending on your accent†. I bring this up because I’ve always wanted to be able to tell speech recognition systems specific features of my accent, say through a series of questions (“Do you pronounce ‘cot’ and ‘caught’ the same or differently?”, &c.) in addition to ML training, because as it is, they persistently mix up certain words—“I’ll” and “all”, for instance. † With “isle of”, I have two syllables and a velarised (“dark”) L in “isle”, a mid-central vowel (/ə/) in “of”; with “I love”, I have one syllable in “I”, a non-velarised (“light”) L and an open-mid back unrounded vowel (/ʌ/) in “love”.
Angostura 8y ago
I just tried 'Hey Siri where is 'Isle of Dogs' playing - and it got it first time.
- gok 8y ago
  That’s easier because the language model knows that “where is I love dogs playing” doesn’t make sense grammatically, whereas “where is <movie name> playing” does. On the other hand, just saying “I love dogs” might be a reasonable thing you may want to text message or something.
  - ascagnel_ 8y ago
    You could also give the recognition system an additional hint by dropping in "the movie", "the play", etc. -- "where is the movie 'Isle of Dogs' playing?", "where is the play 'Death of a Salesman' playing?"
lozaning 8y ago
I was testing some Alexa stuff for work the other day and named a light "Lighty McLight Face". All Alexa would ever hear was "lightly lick my face".
TheSpiceIsLife 8y ago
You should be able to say "Hey Siri, listen carefully" brief pause "isle ... of ... dogs", or some such to have the device enter a special mode where it attempts to identify individual words more accurately.
- anonfunction 8y ago
  I've never heard about this special mode.
  - TheSpiceIsLife 8y ago
    I meant to imply it should have this mode, not that it does.
- eridius 8y ago
  What's wrong with just saying "isle … of … dogs"? Why would you need to enter a special mode before enunciating?
  - pwinnski 8y ago
    Pausing long enough to be sure the AI knows you're done pronouncing the previous words is often enough to trigger the end of the phrase. We want our AI assistant to be performant, after all! Without a special mode, you'd likely say "Hey, Siri, play Isle..." "Okay, I found--" "Dammit, Siri, I wasn't done!"
    - eridius 8y ago
      You don't have to wait 5 seconds between words. You just have to enunciate clearly enough that the divisions between words become truly unambiguous.
      - TheSpiceIsLife 8y ago
        It’s worth noting that, elsewhere in this thread, user smortaz wrote: Yep, tried that too... but admittedly the sound very similar. I keep getting "I'll of dogs" and "I love dogs".
__david__ 8y ago
I've never gotten Siri to successfully play "Cuz It's Hot" by "My Life With The Thrill Kill Kult". Also the only place I have the song is "WaxTrax! Black Box: The First 13 Years (Disc 3)", which is also pretty much impossible to get Siri to play. Sigh.
dkonofalski 8y ago
You can tap anything that Siri hears you say and fix it. Unfortunately, that means you might end up with the opposite problem in the future.

tomphoolery 8y ago

I'm not convinced this Siri thing is really that revolutionary until I can invoke something on my phone by screaming "AYYYYYYY MUST BE THE SIRIIIIII"

zython 8y ago

related: https://maachinelearning.apple.com/2017/08/06/siri-voices.html https://maachinelearning.apple.com/2017/08/06/siri-voices.ht...

sp332 8y ago
Your link has an extra "a" in it. https://machinelearning.apple.com/2017/08/06/siri-voices.html https://machinelearning.apple.com/2017/08/06/siri-voices.htm...

gervase 8y ago

To clarify, this is about training your phone to recognize only "Hey Siri" as spoken by you, as opposed to letting you personalize your trigger phrase to something other than "Hey Siri".

mattigames 8y ago
Yeah, they don't want to give users too much control; Siri is an important brand by itself and Apple doesn't want people going around skipping it.
- draw_down 8y ago
  Shhhhhh
- woolvalley 8y ago
  It's too bad, because I sure wish I could make the homepod use a different phrase. The homepod knows to activate when I say hey siri, but my friends phone activates instead of the homepod whenever they say 'hey siri' about %50 of the time.
  - deleted 8y ago
    [deleted]
- Raphmedia 8y ago
  At least Siri is easy to say internationally. I never manage to use the proper accent when I say "Okay Google".
- saagarjha 8y ago
  The non-cynical viewpoint would be that "Hey Siri" is an easy phrase to detect because of how it's said, which isn't really that similar to many other phrases.
  - gok 8y ago
    “Hey Siri” actually a pretty challenging trigger phrase. It’s short (few phonemes), and it’s phonetically confuseable with phrases like “Hey Serial is out” (could be “Hey Siri all is out”).

pwython 8y ago

Cool little insight into their progress. I'm excited for the day Siri can differentiate my voice and my wife's so we can use a single HomePod with two iCloud accounts. For example, Siri reading off calendar appointments from an iCloud account based on who's asking.

AlphaWeaver 8y ago
This feature is one of my favorite things about the Google Home.
jorvi 8y ago
That feature is not blocked by technical but by arbitrary limits. Whether it is a family iPad, Apple TV or HomePod, Apple strongly believes in 1:1 device-person (or more accurate, device-iCloud account) mapping.
- geerlingguy 8y ago
  Macs do multi account quite well, and iOS/iPad does it for schools. I’m not sure why Apple hasn’t at least made iPads for consumers have the same multi-iCloud-account feature available, it would make things a lot easier at home!
- dkonofalski 8y ago
  The HomePod was designed not to be a single-user device. Especially after reading the OP article, I think it's more of a technical limit than anything because you could potentially give away information that didn't belong to a user if the user isn't detected correctly.

snootyresearch 8y ago

God this is depressing. Only 3 years behind Amazon and Google.

deleted 8y ago
[deleted]
__david__ 8y ago
Really? I'm pretty sure "Alexa" is known for responding to the TV.
- gaius 8y ago
  Yep https://www.theregister.co.uk/2017/01/07/tv_anchor_says_alexa_buy_me_a_dollhouse_and_she_does/ https://www.theregister.co.uk/2017/01/07/tv_anchor_says_alex...
- pwinnski 8y ago
  When it comes to Apple being behind in machine learning, perception is all that matters. Technical explanations about how Apple makes choices that preserve privacy are irrelevant. Actual head-to-head tests showing that Apple is ahead in some areas and behind in others are irrelevant. All that matters is that we all know that Apple is behind in machine learning.

chongli 8y ago

I would love this feature if they found a way to implement it without draining the battery by listening constantly and trying to recognize 'Hey Siri' from all the background noise. It's especially bad when you play music nearby!

gok 8y ago
Are you sure you’re actually getting your battery drained from this? The phrase spotting happens in a super low power processor which can run without the rest of the phone doing anything.
- chongli 8y ago
  I remember one day I played music through some speakers connected to my phone for about 6 hours and the battery usage showed 40% for music and 40% for Siri. I put two and two together and figured the music was keeping Siri awake.

brd529 8y ago

My biggest problem with the homepod is that my 5 year old has learned that he can turn on _any song he wants_ whenever he wants by yelling "hey siri." He loves it! He yells at the dinner table, when his little sister is asleep, and when we are on the phone. His favorite phrase is "Hey Siri, set volume to 100%." His favorite song is the PJ Masks song, which Siri dutifully plays when called upon. Every. Single. Time. I'd love it if HomePod recognized who was speaking, but I'd settle for Siri just recognizing "this is a kids voice" and ignoring it!

aaronmiler 8y ago
I feel like this is a big missing piece for voice activated things. I've got an Echo device ~10ft from my TV, and it gets activated from Netflix (Parks & Rec mostly... I blame Rob Lowe) all the time. I should be able to go into the app and label it as a false activation so it can ignore it in the future
- Art9681 8y ago
  The app already has a feature for that. It will keep a list of all the times it was activated and a recording you can play back of what phrase activated it. You can also tap a button to send feedback if it activated with the wrong phrase.
Endama 8y ago
I'm sorry, this warmed my heart and terrified me at the same time. A hilariously adorable and obnoxious use of technology.
StephenMelon 8y ago
Unfortunately “Hey Siri” starts to grate the 70th time you say it in 3 days. I would actually now rather walk across the room to touch the Homepod than have to say the phrase so much. It’s still in uncanny valley territory a lot of the time as it isn’t smart enough to keep context between interactions or react in a human-like way. I think a retroactive activation feature would be the way to go, ie. be able to say “Siri” at the end of a sentence and have it parse the phrase that came before. It feels more natural to say “skip this please Siri” than “Hey Siri, skip” because it lets you amend the mistake of not identifying the subject earlier in the sentence rather than the phrase-exasperation-hey Siri-phrase. I asked Siri to DJ and my daughter said “hey Siri, I don’t like that song”. Siri said “I’ll remember that” and played something else. Now I have no idea how to amend Siri’s notion that I dislike one of my favourite tracks. There should really be a Homepod app where you can see recent interactions and correct misinterpreted ones. There are also some artists and tracks with unusual spellings that are always going to cause problems for a voice interface and so they need a solution for that.

srcmap 8y ago

What's the technical reason to limit the wake words to only "Hey Siri", "OK Google", etc? Should one be able to "Personalized" to his/her own phase PER Device such as "Hey Commander Data" for Iphone 6, "Hey Counselor Troi " for IPad Pro or "Ok Skynet" for the google phone? That should also solve the problem where someone said "OK Google" in TV Shows/Podcast and wakeup all the google devices. With the latest "Deep Voice" tech, the device should be able to response with the actors' voice too. I remember seeing a clip of Chinese TV show when the actress said (in Chinese) "Honey, Honey where are you?" and the phone on chair wake up and said "Honey, I am here, I am here. " And tons of folks in the youtube comments ask "What kind of phone is that and I want it!". I told my wife that and she immediately said she wanted it too. I still don't know which brand of Chinese phone actually has that features or if that was just imagination of the TV show writer. Someone should build and and I know my wife wants it.

pwinnski 8y ago
I don't work on such projects, but on other ML projects, and I suspect the answer is: training data. There are many, many, many samples of people saying the approved trigger phrases. Many, many, many samples. To use your own custom trigger phrases, you would likely need to say it thousands of times to being to approach the accuracy the current systems ship with out of the box. Maybe tens of thousands. That's just my conjecture, though.