5 ms·

Theres another way to view alexa. It is in 100m+ homes. Its limited right now due to the historical limits of NLP technology. But with the advent of LLMs, amazo

Theres another way to view alexa. It is in 100m+ homes. Its limited right now due to the historical limits of NLP technology. But with the advent of LLMs, amazon has a chance to dramatically upgrade the capabilities of alexa, which is already in so many homes. A product that was too early, but may end up being able to capture the uptrend in AI.

smt88 4y ago
> with the advent of LLMs, amazon has a chance to dramatically upgrade the capabilities of alexa This is a terrible idea. NLP, with or without LLMs, isn't remotely close to being able to interpret commands in someone's house. And who would want a non-deterministic voice assistant anyway? Imagine you say, "Alexa, turn up the air conditioning," and Alexa interprets this to mean you want to increase the temperature. There are uncountable and unforeseeable failure modes. Note that this happens even when talking to humans with near-perfect NLP/brains. The major flaw with voice assistants in general is that they only make sense if they have a pretty narrow and pre-defined universe of accepted commands, which is incredibly confusing to someone who mistakes them for actual NLP. It's just a really, really bad UI. I use mine all the time, but only to set timers and turn lights on or off.
- fdgsdfogijq 4y ago
  You sound like you have no clue what youre talking about. The limitations of alexa and complaints about alexa are really complaints about the limits of NLP, which have just been expanded.
  - smt88 4y ago
    You sound like you have no idea what you're talking about when you say that ChatGPT is a magic fix for Alexa. I gave concrete examples of: - why ChatGPT's style of "convincing bullshit" is a bad fit for a user interface like Alexa - failure modes of NLP in UIs - why NLP still won't save Alexa if it gets to a human level of accuracy because humans aren't accurate enough If you have any actual knowledge or argument other than, "You're ignorant, [technology that doesn't exist] is going to fix Alexa," then let's hear it.
    - fdgsdfogijq 4y ago
      I worked on core ML systems in alexa :) My knowledge is that I know how these systems work, and why its so hard to get them to work properly.
      - smt88 4y ago
        This comment still has no content in it other than "this is how it is" and "trust me bro". Do you have anything substantial to add? Do you have any reason to refute anything I said? It doesn't matter if you're Amazon's ML expert or the world's best surgeon or a barista if all you do is make an unsubstantiated statement and give your credentials.
        fdgsdfogijq 4y ago
        No because its proprietary, and secondly all of your arguments are made out of thin air. Do you want me to produce a design doc about why getting alexa to work correctly is hard?
        shigawire 4y ago
        At the minimum explain how LLMs would make it much better... At least for the slow kids like me who don't know how LLMs or Alexa work.
- at_a_remove 4y ago
  I quasi-jokingly have suggested that we will end up with a conlang like lojban (which is designed to be unambiguous) as a kind of "command language" to machines.
  - smt88 4y ago
    It's definitely not a joke at all. It's exactly how I use the voice assistants. But unlike a traditional programming language, there's no documentation, it's non-deterministic, and the same command will suddenly stop working. It's insane. So here's my process with Google Asisstant: - guess what it will understand through trial and error - use that command pattern, like "set [something] to [level]%", for a few months - try it again one day, and suddenly it doesn't work It makes you scared to trust it in almost any situation.
    - crooked-v 4y ago
      Siri has gone through phases at least twice where it forgot what the word "half" meant in relation to lights, then suddenly understood it again several months later.
      - smt88 4y ago
        I've extensively used Google Assistant and Siri. I've used Alexa a little bit, too. Google is like 4/10, Alexa is 5/10, and Siri is 0/10. Siri is stunningly bad. It understands the command ~50% of the time, and then the action it takes is useful (i.e. prevents me from needing to touch my phone) ~30% of the time.
- astrange 4y ago
  > Imagine you say, "Alexa, turn up the air conditioning," and Alexa interprets this to mean you want to increase the temperature. There are uncountable and unforeseeable failure modes. "up" and "off" sound similar too, and I've had assistants confuse them often.
0cf8612b2e1e 4y ago
Hypothetically, if the voice detection and parsing were superhuman - what capabilities does that enable which the device currently lacks? Voice control is a miserable interface, and there are few applications which I would endeavor to use it. Time, weather, math, and maybe a trivial fact queried from Wikipedia.
- jmathai 4y ago
  Voice control using a central mic that’s always on seems to deliver a poor user experience. I found myself shouting at these assistants while they try to interpret what I say mid sentence. The novelty wore off very quickly. But voice seems like a very powerful interface if you can solve privacy and usability issues. Seems like we haven’t tried very hard at this yet.
makeitdouble 4y ago
What made Alexa miserable to me was the Amazon ecosystem. It is just way too limited to be more useful than a timer, news broadcast and perhaps home automation assistant. In that respect, a better voice interface wouldn´t help. In comparison Google and Apple are competitive in so much more (in particular calendaring, contacts, music/videos). I think Amazon missed the boat in not coming with an open standard and get cosier with independant makers like Sonos or Ikea etc. Basically they could have pulled an android for the voice assistant ecosystem, but preferred to try to build another walled garden.
jmathai 4y ago
I think the point is that there’s plenty of ways in which Alexa could have been useful besides ones requiring LLM and if Amazon couldn’t deliver on these simpler use cases then it’s optimistic to think they’ll be able to do any different with LLMs.

smt88 4y ago

> with the advent of LLMs, amazon has a chance to dramatically upgrade the capabilities of alexa This is a terrible idea. NLP, with or without LLMs, isn't remotely close to being able to interpret commands in someone's house. And who would want a non-deterministic voice assistant anyway? Imagine you say, "Alexa, turn up the air conditioning," and Alexa interprets this to mean you want to increase the temperature. There are uncountable and unforeseeable failure modes. Note that this happens even when talking to humans with near-perfect NLP/brains. The major flaw with voice assistants in general is that they only make sense if they have a pretty narrow and pre-defined universe of accepted commands, which is incredibly confusing to someone who mistakes them for actual NLP. It's just a really, really bad UI. I use mine all the time, but only to set timers and turn lights on or off.

fdgsdfogijq 4y ago
You sound like you have no clue what youre talking about. The limitations of alexa and complaints about alexa are really complaints about the limits of NLP, which have just been expanded.
- smt88 4y ago
  You sound like you have no idea what you're talking about when you say that ChatGPT is a magic fix for Alexa. I gave concrete examples of: - why ChatGPT's style of "convincing bullshit" is a bad fit for a user interface like Alexa - failure modes of NLP in UIs - why NLP still won't save Alexa if it gets to a human level of accuracy because humans aren't accurate enough If you have any actual knowledge or argument other than, "You're ignorant, [technology that doesn't exist] is going to fix Alexa," then let's hear it.
  - fdgsdfogijq 4y ago
    I worked on core ML systems in alexa :) My knowledge is that I know how these systems work, and why its so hard to get them to work properly.
    - smt88 4y ago
      This comment still has no content in it other than "this is how it is" and "trust me bro". Do you have anything substantial to add? Do you have any reason to refute anything I said? It doesn't matter if you're Amazon's ML expert or the world's best surgeon or a barista if all you do is make an unsubstantiated statement and give your credentials.
      - fdgsdfogijq 4y ago
        No because its proprietary, and secondly all of your arguments are made out of thin air. Do you want me to produce a design doc about why getting alexa to work correctly is hard?
        shigawire 4y ago
        At the minimum explain how LLMs would make it much better... At least for the slow kids like me who don't know how LLMs or Alexa work.
at_a_remove 4y ago
I quasi-jokingly have suggested that we will end up with a conlang like lojban (which is designed to be unambiguous) as a kind of "command language" to machines.
- smt88 4y ago
  It's definitely not a joke at all. It's exactly how I use the voice assistants. But unlike a traditional programming language, there's no documentation, it's non-deterministic, and the same command will suddenly stop working. It's insane. So here's my process with Google Asisstant: - guess what it will understand through trial and error - use that command pattern, like "set [something] to [level]%", for a few months - try it again one day, and suddenly it doesn't work It makes you scared to trust it in almost any situation.
  - crooked-v 4y ago
    Siri has gone through phases at least twice where it forgot what the word "half" meant in relation to lights, then suddenly understood it again several months later.
    - smt88 4y ago
      I've extensively used Google Assistant and Siri. I've used Alexa a little bit, too. Google is like 4/10, Alexa is 5/10, and Siri is 0/10. Siri is stunningly bad. It understands the command ~50% of the time, and then the action it takes is useful (i.e. prevents me from needing to touch my phone) ~30% of the time.
astrange 4y ago
> Imagine you say, "Alexa, turn up the air conditioning," and Alexa interprets this to mean you want to increase the temperature. There are uncountable and unforeseeable failure modes. "up" and "off" sound similar too, and I've had assistants confuse them often.

0cf8612b2e1e 4y ago

Hypothetically, if the voice detection and parsing were superhuman - what capabilities does that enable which the device currently lacks? Voice control is a miserable interface, and there are few applications which I would endeavor to use it. Time, weather, math, and maybe a trivial fact queried from Wikipedia.

jmathai 4y ago
Voice control using a central mic that’s always on seems to deliver a poor user experience. I found myself shouting at these assistants while they try to interpret what I say mid sentence. The novelty wore off very quickly. But voice seems like a very powerful interface if you can solve privacy and usability issues. Seems like we haven’t tried very hard at this yet.

makeitdouble 4y ago

What made Alexa miserable to me was the Amazon ecosystem. It is just way too limited to be more useful than a timer, news broadcast and perhaps home automation assistant. In that respect, a better voice interface wouldn´t help. In comparison Google and Apple are competitive in so much more (in particular calendaring, contacts, music/videos). I think Amazon missed the boat in not coming with an open standard and get cosier with independant makers like Sonos or Ikea etc. Basically they could have pulled an android for the voice assistant ecosystem, but preferred to try to build another walled garden.

jmathai 4y ago

I think the point is that there’s plenty of ways in which Alexa could have been useful besides ones requiring LLM and if Amazon couldn’t deliver on these simpler use cases then it’s optimistic to think they’ll be able to do any different with LLMs.