3 ms·

I thought this was a great moment to take stock of these voice assistants and do a postmortem on why the platform failed to live up to expectations. Something

I thought this was a great moment to take stock of these voice assistants and do a postmortem on why the platform failed to live up to expectations. Something telling from the article is that Alexa is used to play music and check the weather and not much else, no matter how hard the division tries to get people to order Ubers or toothpaste. I have been wondering if a next gen “GTP-4” like language model could take these AI assistants from being jukeboxes to actual assistants.

PaulHoule 4y ago
Some of the problem is not the A.I. technology but the model of the user interface. The model of Alexa is [voice input] -> [action] but to do more complex interactions it has to be more like [input 1] -> [ask for clarification] -> [input 2] -> [propose action, ask for confirmation ] -> [action] from the viewpoint of developing a training set this is exponentially more complicated. Maybe it is like playing chess or learning Atari games by reinforcement learning, but those games involve a small number of elements that combine in complex ways, whereas a conversation involves a space of millions of concepts. (e.g. "20 questions") It's not so much that they need any specific feature ("buy toothpaste") that they need some way to recover the cost, for instance being more tied to the Amazon ecosystem, more likely to subscribe to Prime, etc. It's not easy though. With roughly 70 million users they would need to generate about $150 a user in profit to cover $10 billion. At 2% margin they'd need to sell $7500 worth of stuff that they wouldn't otherwise sell. What I don't get is where the $10 billion went: how much of it subsidizing the device? (If they sold 10 million devices last year it would be a $1000 subsidy which is incredible.) are they paying AWS prices to run the backend service? how much are they spending in software development, model training, etc?
- cscharenberg 4y ago
  And in those more complex interactions there is not enough adjustment capability. I often want Alexa to speak faster or just shorter (reduce filler words and pauses). There's no way (I am aware of) to make it more efficient for the determined user.
- otikik 4y ago
  > [input 1] -> [ask for clarification] -> [input 2] -> [propose action, ask for confirmation ] -> [action] Don’t forget that between all those steps there’s a “store everything you say in order to sell you stuff later” step
- laidoffamazon 4y ago
  Asking for clarifications/disambiguation was built into Alexa several years ago.
- geoelectric 4y ago
  My biggest problem with the flow nowadays is instead of getting briefer and simpler as you and the device learn each other better, it stays just as onerous all the way through--and, in fact, becomes more so as they introduce more "By the way, you can...[new feature discovery I did NOT want forced down my throat when trying to reset my alarm while half asleep]..." asides. It's like every lesson about not disrupting GUI flow with modal banners, and about creating shortcuts and other accelerated paths for advanced users was forgotten in VUI. At best you get a one-size-fits-all brief mode that usually isn't very brief, and doesn't reward learning so doesn't increase engagement. VUI vendors also plainly have no clue how to enable discovery without either fucking up your home screen with some ugly "things you can do" list or being annoying about volunteering stuff when responding to your commands. It's as if they might need to *gasp* document their features instead because voice devices aren't web apps you can just fiddle with. The net result is I actually get way more frustrated with these devices now that I'm super-experienced with the tech, not less. That's the wrong direction.
  - PaulHoule 4y ago
    I'd say the "not disrupting GUI flow with modal banners" has been by no means learned. Sometimes it seems like this is the style manual for software today https://tvtropes.org/pmwiki/pmwiki.php/Funny/Idiocracy https://tvtropes.org/pmwiki/pmwiki.php/Funny/Idiocracy That anti-pattern of interrupting you to tell you about a new feature is common and it's the worst. All the time I feel like I am behind the 8-ball and I really need to finish some small task right now but Firefox, Photoshop and seemingly every other product wants to get in my way of doing the task. It's a deep problem though matching the expectations of a user for a VUI with what it can so, Alexa's greatest triumph is that it had a domain that it seemed pretty complete over and that mostly people could "get" what it was capable of doing.
- samth 4y ago
  Supposedly they had 10k people working on it. Over 5 years that only 200k per person per year just to account for the whole 10 billion.
koyote 4y ago
I feel like there are just not many truly game changing use-cases for a voice-activated assistant. They only ever seem to be useful if you do not have free hands (e.g. cooking) which is why they are almost exclusively used as timers (at least in my household). I have tried getting it to read a recipe but that's just cumbersome (I often want to have a quick glance at ingredients, not have someone list all of them in slow succession). Other actions such as "play stranger things on netflix" seem like useful actions at first, but again only really make sense if you are truly busy with your hands (which you would not be if you are on the couch with the remote near-by). Using the remote to turn on the tv and play something on netflix is often not more than two or three button presses (assuming the show is in your 'continue watching' list, which it usually is if it's something you specifically want to watch). And turning on lights? Again, a novelty. Unless your house is designed in a weird way where your light switches are on opposite ends of the room you are walking into, it's always faster to just flip the switch when walking into the room than shouting out and waiting for your assistant to do it for you (never mind the fact that shouting to turn on the lights during a bathroom run at night would be highly annoying to other household members).
- geoelectric 4y ago
  I like my routine that turns off my entire house when I go to sleep at night, because I'm ADHD and I won't flip off lights or the TV leaving a room as a general rule--especially in the evenings when I'm low-meds. With the recent heat-sensing models, I could even automate them to be presence detecting lights and fix that issue room by room. I also like being able to sequence the set of several lighting, audio, and video operations that sets my theater room up for various uses. I like being able to quickly check the house temp and tweak the thermostat from the other side of the house. And I like being able to turn whole rooms on and off at once, or to set their light dimness and color for my use of the moment without getting up or fiddling with my phone, etc. I especially like the routine that does a sunrise wake on my bedroom lights; turns off my brown noise machine a minute before my alarm since sometimes that will peacefully wake me without having to hear the beep; chatters jokes, news, etc, incessantly at me in the morning once I dismiss my alarm to make sure I actually get up; then automatically shifts into whole-house Sirius XM to play hard rock as I start my day. And say what you want, I like being able to ask the thing over my shoulder what my office fan 5 feet behind me is set to and change that while coding and still looking at the screen, rather than stopping, screwing around with a cheap remote control and trying to see a little LCD from across the room. Like I said, I'm ADHD. If I stop to do that, who knows when I get back on task again. I agree for simple "turn on a switch" tasks they're way overkill, but for actual automation (and perhaps accommodation) these devices are useful, especially once you start replacing ALL your environmental devices with controllable ones. It's more a question as to whether you can leverage the automation.
laidoffamazon 4y ago
LLM chaining seems like it works pretty incredibly - between the newly released Whisper Model and chaining building up a more sophisticated assistant than the current open source ones seems extremely plausible even for smaller teams. [0] https://twitter.com/hwchase17/status/1589648226449514496 https://twitter.com/hwchase17/status/1589648226449514496
- dougmwne 4y ago
  That is an insane demo and really makes me think that building Alexa was not wrong, just early. I don’t know if that next-level capability would have been more useful to consumers or able to be monetized, but I suppose we will have to wait a few years to find out. Best of luck in your future endeavors.

PaulHoule 4y ago

Some of the problem is not the A.I. technology but the model of the user interface. The model of Alexa is [voice input] -> [action] but to do more complex interactions it has to be more like [input 1] -> [ask for clarification] -> [input 2] -> [propose action, ask for confirmation ] -> [action] from the viewpoint of developing a training set this is exponentially more complicated. Maybe it is like playing chess or learning Atari games by reinforcement learning, but those games involve a small number of elements that combine in complex ways, whereas a conversation involves a space of millions of concepts. (e.g. "20 questions") It's not so much that they need any specific feature ("buy toothpaste") that they need some way to recover the cost, for instance being more tied to the Amazon ecosystem, more likely to subscribe to Prime, etc. It's not easy though. With roughly 70 million users they would need to generate about $150 a user in profit to cover $10 billion. At 2% margin they'd need to sell $7500 worth of stuff that they wouldn't otherwise sell. What I don't get is where the $10 billion went: how much of it subsidizing the device? (If they sold 10 million devices last year it would be a $1000 subsidy which is incredible.) are they paying AWS prices to run the backend service? how much are they spending in software development, model training, etc?

cscharenberg 4y ago
And in those more complex interactions there is not enough adjustment capability. I often want Alexa to speak faster or just shorter (reduce filler words and pauses). There's no way (I am aware of) to make it more efficient for the determined user.
otikik 4y ago
> [input 1] -> [ask for clarification] -> [input 2] -> [propose action, ask for confirmation ] -> [action] Don’t forget that between all those steps there’s a “store everything you say in order to sell you stuff later” step
laidoffamazon 4y ago
Asking for clarifications/disambiguation was built into Alexa several years ago.
geoelectric 4y ago
My biggest problem with the flow nowadays is instead of getting briefer and simpler as you and the device learn each other better, it stays just as onerous all the way through--and, in fact, becomes more so as they introduce more "By the way, you can...[new feature discovery I did NOT want forced down my throat when trying to reset my alarm while half asleep]..." asides. It's like every lesson about not disrupting GUI flow with modal banners, and about creating shortcuts and other accelerated paths for advanced users was forgotten in VUI. At best you get a one-size-fits-all brief mode that usually isn't very brief, and doesn't reward learning so doesn't increase engagement. VUI vendors also plainly have no clue how to enable discovery without either fucking up your home screen with some ugly "things you can do" list or being annoying about volunteering stuff when responding to your commands. It's as if they might need to *gasp* document their features instead because voice devices aren't web apps you can just fiddle with. The net result is I actually get way more frustrated with these devices now that I'm super-experienced with the tech, not less. That's the wrong direction.
- PaulHoule 4y ago
  I'd say the "not disrupting GUI flow with modal banners" has been by no means learned. Sometimes it seems like this is the style manual for software today https://tvtropes.org/pmwiki/pmwiki.php/Funny/Idiocracy https://tvtropes.org/pmwiki/pmwiki.php/Funny/Idiocracy That anti-pattern of interrupting you to tell you about a new feature is common and it's the worst. All the time I feel like I am behind the 8-ball and I really need to finish some small task right now but Firefox, Photoshop and seemingly every other product wants to get in my way of doing the task. It's a deep problem though matching the expectations of a user for a VUI with what it can so, Alexa's greatest triumph is that it had a domain that it seemed pretty complete over and that mostly people could "get" what it was capable of doing.
samth 4y ago
Supposedly they had 10k people working on it. Over 5 years that only 200k per person per year just to account for the whole 10 billion.

koyote 4y ago

I feel like there are just not many truly game changing use-cases for a voice-activated assistant. They only ever seem to be useful if you do not have free hands (e.g. cooking) which is why they are almost exclusively used as timers (at least in my household). I have tried getting it to read a recipe but that's just cumbersome (I often want to have a quick glance at ingredients, not have someone list all of them in slow succession). Other actions such as "play stranger things on netflix" seem like useful actions at first, but again only really make sense if you are truly busy with your hands (which you would not be if you are on the couch with the remote near-by). Using the remote to turn on the tv and play something on netflix is often not more than two or three button presses (assuming the show is in your 'continue watching' list, which it usually is if it's something you specifically want to watch). And turning on lights? Again, a novelty. Unless your house is designed in a weird way where your light switches are on opposite ends of the room you are walking into, it's always faster to just flip the switch when walking into the room than shouting out and waiting for your assistant to do it for you (never mind the fact that shouting to turn on the lights during a bathroom run at night would be highly annoying to other household members).

geoelectric 4y ago
I like my routine that turns off my entire house when I go to sleep at night, because I'm ADHD and I won't flip off lights or the TV leaving a room as a general rule--especially in the evenings when I'm low-meds. With the recent heat-sensing models, I could even automate them to be presence detecting lights and fix that issue room by room. I also like being able to sequence the set of several lighting, audio, and video operations that sets my theater room up for various uses. I like being able to quickly check the house temp and tweak the thermostat from the other side of the house. And I like being able to turn whole rooms on and off at once, or to set their light dimness and color for my use of the moment without getting up or fiddling with my phone, etc. I especially like the routine that does a sunrise wake on my bedroom lights; turns off my brown noise machine a minute before my alarm since sometimes that will peacefully wake me without having to hear the beep; chatters jokes, news, etc, incessantly at me in the morning once I dismiss my alarm to make sure I actually get up; then automatically shifts into whole-house Sirius XM to play hard rock as I start my day. And say what you want, I like being able to ask the thing over my shoulder what my office fan 5 feet behind me is set to and change that while coding and still looking at the screen, rather than stopping, screwing around with a cheap remote control and trying to see a little LCD from across the room. Like I said, I'm ADHD. If I stop to do that, who knows when I get back on task again. I agree for simple "turn on a switch" tasks they're way overkill, but for actual automation (and perhaps accommodation) these devices are useful, especially once you start replacing ALL your environmental devices with controllable ones. It's more a question as to whether you can leverage the automation.

laidoffamazon 4y ago

LLM chaining seems like it works pretty incredibly - between the newly released Whisper Model and chaining building up a more sophisticated assistant than the current open source ones seems extremely plausible even for smaller teams. [0] https://twitter.com/hwchase17/status/1589648226449514496 https://twitter.com/hwchase17/status/1589648226449514496

dougmwne 4y ago
That is an insane demo and really makes me think that building Alexa was not wrong, just early. I don’t know if that next-level capability would have been more useful to consumers or able to be monetized, but I suppose we will have to wait a few years to find out. Best of luck in your future endeavors.