6 ms·

According to an episode of Acquired[0], none of Siri's original technology is present in Apple's current products. Siri originally licensed Nuance's (also an SR

by wyc 10y ago

According to an episode of Acquired[0], none of Siri's original technology is present in Apple's current products. Siri originally licensed Nuance's (also an SRI spin-out company) voice recognition technology and built some pretty (comparatively) simple tech on top of it, never changing out the speech-to-text engine. All current players (Google, Amazon, etc) use neural nets, so Apple scooped up many Nuance employees to double down and compete[1]. Rosenthal and Gilbert agreed that Siri was a failure: "the only thing I use Siri for is setting alarms." I'd have to agree from personal experience. They chalked the failure up to the disadvantages of a UI where the full extent of the functionality is hidden and difficult to understand, leading to bad experiences. I imagine this technology will have many of the same struggles. [0] http://www.acquired.fm/episodes/2015/12/14/episode-5-siri http://www.acquired.fm/episodes/2015/12/14/episode-5-siri [1] http://9to5mac.com/2014/06/30/why-is-apple-hiring-nuance-engineers-apparently-to-replace-siris-nuance-powered-backend/ http://9to5mac.com/2014/06/30/why-is-apple-hiring-nuance-eng...

skylark 10y ago
I think the biggest problem with voice commands is that if the command doesn't work the first time, it would have been faster to simply input the command yourself. I'm not confident that "get me directions to <place>" will actually get me those directions. Because of that, I also use Siri exclusively to set alarms (which does work every time).
- mwfunk 10y ago
  This is the #1 problem with anything based on voice recognition and natural language processing IMO. It has to work 99% of the time to overcome this issue, but it seems like the nature of the technologies ensure that it will only asymptotically approach this threshold, and is currently stuck at 75% reliability at best. Plus, even if it did work, it's just a really inexact and inefficient way to do anything. You always end up trying to see through the abstraction layer (natural language in this case) to the much more well-defined hidden system underneath. It's like reverse engineering an expert system, as a UI paradigm. Programming languages made to resemble natural language face a similar poison pill, AppleScript being a prime example. I totally get the appeal of trying to make software understand people better instead of forcing people to understand software, but in practice it just ends up being one more abstraction layer users have to struggle to get past in order to unlock the functionality of the underlying system.
  - modeless 10y ago
    The funny thing is that voice recognition is no longer the problem. For years the reason you couldn't do voice command interfaces was that transcription didn't work, but today transcription works amazingly well. My commands are transcribed perfectly, even in my car when I'm doing 65 on the freeway with my phone stuck in a cupholder. The problem now is Android randomly kills the "OK Google" background listening process, or it fails because it can't handle a handover between wifi and cell data, or it "can't open microphone" even though it just heard me say OK Google, or any one of many other Android problems rears its ugly head. The reliability of voice transcription now is far better than the general reliability of Android on my Nexus 5X, and the Android team ought to feel pretty bad about that.
    - jordanthoms 10y ago
      Exactly this - There is a bug where if your phone is locked, and in a position where it thinks it's in landscape, when you say 'OK Google' the phone will unlock in portrait, start listening, rotate to landscape and in the process kill the app listening to your voice, and then rotate back to portrait and lock itself again. It's also incredibly easy to fall off the blessed path where you can interact solely with voice - when setting a reminder, for example, if it doesn't hear 'Yes' when it's expecting to it'll just sit there and you need to touch the screen to continue - defeating the whole point of voice interaction. Google's voice search has incredible voice recognition and text to speech and can tap into an amazing amount of information through the knowledge graph, but they don't seem to be capable of fixing the basic bugs and UX issues preventing all that technology from actually being usable.
      - visarga 10y ago
        I was recently trying to find why this happens. For me, sometimes the phone hears "Ok Google" but then just sits there as if there is no microphone, while being in portrait mode all the time, so, no screen rotation. Other times, as you said, being screen-locked makes it deaf. I have to manually unlock it to restart OK Google. We're not suffering from too stupid AI, we're suffering from stupid app design and laziness when it comes to adding more variations to conversation. Just put a team of script writers to imagine thousands of replies, it is not that hard, people have been doing this kind of rudimentary chat bot scripts for decades now. I'd like to be able to install an App and suddenly, new commands are available to me.
- visarga 10y ago
  I also use "Call my wife" and "Text my wife" or "Tell my wife that ...". They work every time. Music playing is disappointing - if I want to open YouTube to play something, it can't. I don't understand why they didn't at least add a few commands for YouTube. At least "play Band", "play Song by Band", "play Music-Genre", "play something like X", "I feel sad/happy/lonely/bored play some music". If you want to understand the extent to which Siri is useful, you just have to look at the integration it has with the main apps. It's just Alarms, Calendar, Phone, SMS, Music, Weather and similar. It's not really groundbreaking AI, just a voiced command list to the main apps. I am eagerly waiting for the moment when we could have more in-depth chats with our bots, but the current crop is dragging its feet when it comes even to simple commands sent to apps. Maybe they should ask the app creators to add hooks for voice commands, to make Siri more useful. For example, if I can disgress a little, Google Now can't start playing the first video in the YouTube app on Android. If I am already in the YT app and say "OK Google, play the first track", it goes to web search instead and searches for "the first track" on the web! The same company made the chatbot, the app and the OS and they don't work well enough together. At least let users add new functions to the bot, if they can't be bothered to do it themselves.
  - coldtea 10y ago
    >I don't understand why they didn't at least add a few commands for YouTube. At least "play Band", "play Song by Band", "play Music-Genre", "play something like X", "I feel sad/happy/lonely/bored play some music". Because YouTube ain't a music player? [1] And besides they have their own music app. [1] Seriously, even though some people use it as such, this is far from most. According to YouTube itself, the average user only spends 1 hour per month listening to music on YouTube -- consumption is mostly non-music videos.
    - deleted 10y ago
      [deleted]
- dzhiurgis 10y ago
  > which does work every time Except when it doesn't. I had number of times when siri would fail to set reminders. I do not have a need to set alarms that often, but people seem to have problems with it too: https://discussions.apple.com/thread/7280346?start=0&tstart=0 https://discussions.apple.com/thread/7280346?start=0&tstart=...
slededit 10y ago
Its not just the hidden functionality, for me Siri will frequently assume I want to call somebody when I ask it something. Explaining why I accidentally called someone I hadn't talked to in years (and by the how are you doing?) makes playing with Siri a high risk endeavour.
- visarga 10y ago
  I wanted to prevent this and only allow calling my favorite contacts (under 10) instead of my whole phone book which is over 1000 contacts. There is no way to set this option in Google Now or Siri.
- zarify 10y ago
  I've been saved most of the time by having lots of similar sounding people in my address book. The other night I was lying in bed and was setting a reminder and it almost called one of my coworkers (or someone I trained with a few years ago). So the obvious solution is either to have duplicate contact entries or only associate with people who sound like those you already know, so that Siri has to go to the "do you mean Person A or Person B?" state :)
cageface 10y ago
This is why I'm very skeptical of this current craze for chat bots. It seems to me that they're really only useful if they can infer a wide range of intent from the user's input, and that seems like it's still very much an unsolved problem. Otherwise they're just glorified menu systems.
- Animats 10y ago
  This is easy to do badly and very hard to do well. It's not just about inferring intent. Most discourse systems have little or no model of the consequences of their actions. Systems that can act on the user's behalf need to know if they're potentially making a big mistake or a little one, as guidance of when to ask for clarification. This requires some degree of "common sense". AI systems still suck at that. Systems that actually do something need to be much better at this than ones that just answer questions. There needs to be some cost/benefit analysis of whether it's necessary to repeat back info and ask for confirmation. Without that, the system will either ask for confirmation too much, or screw up too much. The system needs to have a model of how clear the user is in their intent. Pizza ordering is relatively low-risk. Travel reservations are a higher-stakes item. Medical is a long way off. As I wrote last time, I recommend watching "The Devil Wears Prada", the scenes of Andy's first day as Miranda's personal assistant. That's the level of performance you want.
  - cageface 10y ago
    Good point. I've been thinking so much about the difficulty of inferring intent I hadn't really considered the consequences of getting it wrong. The more responsible we allow these chat bots to be the greater the risks become. This is why I think that eventually people will bring this technology up to a useful level it's only going to be the kinds of companies that can make huge time and money investments and have access to huge datasets. Not startup territory really.
  - Animats 10y ago
    For those of you who haven't seen "The Devil Wears Prada", here's Miranda giving orders to her assistant: I need 10 or 15 skirts from Calvin Klein. -What kind of skirts do you… -Please bore someone else with your questions. And make sure we have Pier 59 at 8:00 a.m. Tomorrow. Remind Jocelyn I need to see a few of those satchels that Marc is doing in the pony. And then tell Simone I'll take Jackie if Maggie isn't available. -Did Demarchelier confirm? -D-Did D-Demarchel… Demarchelier. Did he… Get him on the phone. Uh, o… okay. -And, Emily? -Yes? That's all.
gumby 10y ago
> They chalked the failure up to the disadvantages of a UI where the full extent of the functionality is hidden and difficult to understand, leading to bad experiences. I imagine this technology will have many of the same struggles. This is super hard. Not only do you have hidden functionality (what a great insight!) but they are trying to do something better than the current state of the art (which is human-to-human). What do I mean? Just listen to yourself when you get something over the phone. You don't call the restaurant and say "give me a reservation for four at 8." You typically use an interactive process of query and response ("can you fit four in at 8pm?" "Yes, today." "No, not five, four". Then they repeat it back to you just in case. We get annoyed at how crappy the voice recognition systems are, and they are crappy, but human voice recognition reminds me of a TCP session negotiation, or maybe the baud rate negotiation of a bell-compatible Hayes-style modem. It's hardly a one-shot process, yet we want our machines to be.
- ttrmw 10y ago
  Which is probably why solutions like Open Table exist and succeed. For me at least, avoiding this kind of inexact interaction is desirable. Simple forms on web pages can provide a smoother UX than talking to huamns for the this type of interaction. If these can provide that one shot process, then inevitably that is the expectation set of any replacement UI.
  - adventured 10y ago
    The solution to that problem will be AI to AI instead of AI to human, using standardized interfacing (likely one of a few big common standards, with each AI being capable of utilizing any of them) so the AI can easily understand each other. In the next ten years there will be a large battle - which will play out on HN daily - over the AI to AI standards; it'll remind older developers of the XML / JSON days.
    - gumby 10y ago
      The beginning of the chain is a human -- the user! And at the end of the day this is all about the user! Once you know what the user wants it's "easy" (well, somewhat easy anyway :-). It's the iterative process of understanding the user in an unconstrained opaque ad hoc UI that is big-H Hard. The problem with these spoken interfaces is not just that we don't know what we don't know, we don't even know what we want to accomplish!
mcintyre1994 10y ago
I've never seen anything suggesting Siri is any smarter than just looking for keywords. One thing I've tried using it for is a timer while cooking, already a bit rubbish because iOS only supports one. But here's a sample of how that goes: Open Siri (hands free Hey Siri works for me 5% of the time at best) "timer 20 minutes" "Okay, 20 minutes and counting" A little later, oops that'll need 5 minutes longer than I thought "add 5 minutes to the timer" "are you sure you want to change the timer?" "yes, change it" "Okay, 5 minutes and counting" Great... Other than simple keyword commands like that its voice recognition doesn't seem to be good enough for things like transcribing messages anyway. It's sometimes quicker than spotlight for a web search because of that weird way spotlight hides the search web button for a few seconds when it can't find anything to show.

by wyc 10y ago

skylark 10y ago

I think the biggest problem with voice commands is that if the command doesn't work the first time, it would have been faster to simply input the command yourself. I'm not confident that "get me directions to <place>" will actually get me those directions. Because of that, I also use Siri exclusively to set alarms (which does work every time).

mwfunk 10y ago
This is the #1 problem with anything based on voice recognition and natural language processing IMO. It has to work 99% of the time to overcome this issue, but it seems like the nature of the technologies ensure that it will only asymptotically approach this threshold, and is currently stuck at 75% reliability at best. Plus, even if it did work, it's just a really inexact and inefficient way to do anything. You always end up trying to see through the abstraction layer (natural language in this case) to the much more well-defined hidden system underneath. It's like reverse engineering an expert system, as a UI paradigm. Programming languages made to resemble natural language face a similar poison pill, AppleScript being a prime example. I totally get the appeal of trying to make software understand people better instead of forcing people to understand software, but in practice it just ends up being one more abstraction layer users have to struggle to get past in order to unlock the functionality of the underlying system.
- modeless 10y ago
  The funny thing is that voice recognition is no longer the problem. For years the reason you couldn't do voice command interfaces was that transcription didn't work, but today transcription works amazingly well. My commands are transcribed perfectly, even in my car when I'm doing 65 on the freeway with my phone stuck in a cupholder. The problem now is Android randomly kills the "OK Google" background listening process, or it fails because it can't handle a handover between wifi and cell data, or it "can't open microphone" even though it just heard me say OK Google, or any one of many other Android problems rears its ugly head. The reliability of voice transcription now is far better than the general reliability of Android on my Nexus 5X, and the Android team ought to feel pretty bad about that.
  - jordanthoms 10y ago
    Exactly this - There is a bug where if your phone is locked, and in a position where it thinks it's in landscape, when you say 'OK Google' the phone will unlock in portrait, start listening, rotate to landscape and in the process kill the app listening to your voice, and then rotate back to portrait and lock itself again. It's also incredibly easy to fall off the blessed path where you can interact solely with voice - when setting a reminder, for example, if it doesn't hear 'Yes' when it's expecting to it'll just sit there and you need to touch the screen to continue - defeating the whole point of voice interaction. Google's voice search has incredible voice recognition and text to speech and can tap into an amazing amount of information through the knowledge graph, but they don't seem to be capable of fixing the basic bugs and UX issues preventing all that technology from actually being usable.
    - visarga 10y ago
      I was recently trying to find why this happens. For me, sometimes the phone hears "Ok Google" but then just sits there as if there is no microphone, while being in portrait mode all the time, so, no screen rotation. Other times, as you said, being screen-locked makes it deaf. I have to manually unlock it to restart OK Google. We're not suffering from too stupid AI, we're suffering from stupid app design and laziness when it comes to adding more variations to conversation. Just put a team of script writers to imagine thousands of replies, it is not that hard, people have been doing this kind of rudimentary chat bot scripts for decades now. I'd like to be able to install an App and suddenly, new commands are available to me.
visarga 10y ago
I also use "Call my wife" and "Text my wife" or "Tell my wife that ...". They work every time. Music playing is disappointing - if I want to open YouTube to play something, it can't. I don't understand why they didn't at least add a few commands for YouTube. At least "play Band", "play Song by Band", "play Music-Genre", "play something like X", "I feel sad/happy/lonely/bored play some music". If you want to understand the extent to which Siri is useful, you just have to look at the integration it has with the main apps. It's just Alarms, Calendar, Phone, SMS, Music, Weather and similar. It's not really groundbreaking AI, just a voiced command list to the main apps. I am eagerly waiting for the moment when we could have more in-depth chats with our bots, but the current crop is dragging its feet when it comes even to simple commands sent to apps. Maybe they should ask the app creators to add hooks for voice commands, to make Siri more useful. For example, if I can disgress a little, Google Now can't start playing the first video in the YouTube app on Android. If I am already in the YT app and say "OK Google, play the first track", it goes to web search instead and searches for "the first track" on the web! The same company made the chatbot, the app and the OS and they don't work well enough together. At least let users add new functions to the bot, if they can't be bothered to do it themselves.
- coldtea 10y ago
  >I don't understand why they didn't at least add a few commands for YouTube. At least "play Band", "play Song by Band", "play Music-Genre", "play something like X", "I feel sad/happy/lonely/bored play some music". Because YouTube ain't a music player? [1] And besides they have their own music app. [1] Seriously, even though some people use it as such, this is far from most. According to YouTube itself, the average user only spends 1 hour per month listening to music on YouTube -- consumption is mostly non-music videos.
  - deleted 10y ago
    [deleted]
dzhiurgis 10y ago
> which does work every time Except when it doesn't. I had number of times when siri would fail to set reminders. I do not have a need to set alarms that often, but people seem to have problems with it too: https://discussions.apple.com/thread/7280346?start=0&tstart=0 https://discussions.apple.com/thread/7280346?start=0&tstart=...

slededit 10y ago

Its not just the hidden functionality, for me Siri will frequently assume I want to call somebody when I ask it something. Explaining why I accidentally called someone I hadn't talked to in years (and by the how are you doing?) makes playing with Siri a high risk endeavour.

visarga 10y ago
I wanted to prevent this and only allow calling my favorite contacts (under 10) instead of my whole phone book which is over 1000 contacts. There is no way to set this option in Google Now or Siri.
zarify 10y ago
I've been saved most of the time by having lots of similar sounding people in my address book. The other night I was lying in bed and was setting a reminder and it almost called one of my coworkers (or someone I trained with a few years ago). So the obvious solution is either to have duplicate contact entries or only associate with people who sound like those you already know, so that Siri has to go to the "do you mean Person A or Person B?" state :)

cageface 10y ago

This is why I'm very skeptical of this current craze for chat bots. It seems to me that they're really only useful if they can infer a wide range of intent from the user's input, and that seems like it's still very much an unsolved problem. Otherwise they're just glorified menu systems.

Animats 10y ago
This is easy to do badly and very hard to do well. It's not just about inferring intent. Most discourse systems have little or no model of the consequences of their actions. Systems that can act on the user's behalf need to know if they're potentially making a big mistake or a little one, as guidance of when to ask for clarification. This requires some degree of "common sense". AI systems still suck at that. Systems that actually do something need to be much better at this than ones that just answer questions. There needs to be some cost/benefit analysis of whether it's necessary to repeat back info and ask for confirmation. Without that, the system will either ask for confirmation too much, or screw up too much. The system needs to have a model of how clear the user is in their intent. Pizza ordering is relatively low-risk. Travel reservations are a higher-stakes item. Medical is a long way off. As I wrote last time, I recommend watching "The Devil Wears Prada", the scenes of Andy's first day as Miranda's personal assistant. That's the level of performance you want.
- cageface 10y ago
  Good point. I've been thinking so much about the difficulty of inferring intent I hadn't really considered the consequences of getting it wrong. The more responsible we allow these chat bots to be the greater the risks become. This is why I think that eventually people will bring this technology up to a useful level it's only going to be the kinds of companies that can make huge time and money investments and have access to huge datasets. Not startup territory really.
- Animats 10y ago
  For those of you who haven't seen "The Devil Wears Prada", here's Miranda giving orders to her assistant: I need 10 or 15 skirts from Calvin Klein. -What kind of skirts do you… -Please bore someone else with your questions. And make sure we have Pier 59 at 8:00 a.m. Tomorrow. Remind Jocelyn I need to see a few of those satchels that Marc is doing in the pony. And then tell Simone I'll take Jackie if Maggie isn't available. -Did Demarchelier confirm? -D-Did D-Demarchel… Demarchelier. Did he… Get him on the phone. Uh, o… okay. -And, Emily? -Yes? That's all.

gumby 10y ago

> They chalked the failure up to the disadvantages of a UI where the full extent of the functionality is hidden and difficult to understand, leading to bad experiences. I imagine this technology will have many of the same struggles. This is super hard. Not only do you have hidden functionality (what a great insight!) but they are trying to do something better than the current state of the art (which is human-to-human). What do I mean? Just listen to yourself when you get something over the phone. You don't call the restaurant and say "give me a reservation for four at 8." You typically use an interactive process of query and response ("can you fit four in at 8pm?" "Yes, today." "No, not five, four". Then they repeat it back to you just in case. We get annoyed at how crappy the voice recognition systems are, and they are crappy, but human voice recognition reminds me of a TCP session negotiation, or maybe the baud rate negotiation of a bell-compatible Hayes-style modem. It's hardly a one-shot process, yet we want our machines to be.

ttrmw 10y ago
Which is probably why solutions like Open Table exist and succeed. For me at least, avoiding this kind of inexact interaction is desirable. Simple forms on web pages can provide a smoother UX than talking to huamns for the this type of interaction. If these can provide that one shot process, then inevitably that is the expectation set of any replacement UI.
- adventured 10y ago
  The solution to that problem will be AI to AI instead of AI to human, using standardized interfacing (likely one of a few big common standards, with each AI being capable of utilizing any of them) so the AI can easily understand each other. In the next ten years there will be a large battle - which will play out on HN daily - over the AI to AI standards; it'll remind older developers of the XML / JSON days.
  - gumby 10y ago
    The beginning of the chain is a human -- the user! And at the end of the day this is all about the user! Once you know what the user wants it's "easy" (well, somewhat easy anyway :-). It's the iterative process of understanding the user in an unconstrained opaque ad hoc UI that is big-H Hard. The problem with these spoken interfaces is not just that we don't know what we don't know, we don't even know what we want to accomplish!

mcintyre1994 10y ago

I've never seen anything suggesting Siri is any smarter than just looking for keywords. One thing I've tried using it for is a timer while cooking, already a bit rubbish because iOS only supports one. But here's a sample of how that goes: Open Siri (hands free Hey Siri works for me 5% of the time at best) "timer 20 minutes" "Okay, 20 minutes and counting" A little later, oops that'll need 5 minutes longer than I thought "add 5 minutes to the timer" "are you sure you want to change the timer?" "yes, change it" "Okay, 5 minutes and counting" Great... Other than simple keyword commands like that its voice recognition doesn't seem to be good enough for things like transcribing messages anyway. It's sometimes quicker than spotlight for a web search because of that weird way spotlight hides the search web button for a few seconds when it can't find anything to show.