3 ms·

> with the advent of LLMs, amazon has a chance to dramatically upgrade the capabilities of alexa This is a terrible idea. NLP, with or without LLMs, isn't remo

by smt88 4y ago

> with the advent of LLMs, amazon has a chance to dramatically upgrade the capabilities of alexa This is a terrible idea. NLP, with or without LLMs, isn't remotely close to being able to interpret commands in someone's house. And who would want a non-deterministic voice assistant anyway? Imagine you say, "Alexa, turn up the air conditioning," and Alexa interprets this to mean you want to increase the temperature. There are uncountable and unforeseeable failure modes. Note that this happens even when talking to humans with near-perfect NLP/brains. The major flaw with voice assistants in general is that they only make sense if they have a pretty narrow and pre-defined universe of accepted commands, which is incredibly confusing to someone who mistakes them for actual NLP. It's just a really, really bad UI. I use mine all the time, but only to set timers and turn lights on or off.

fdgsdfogijq 4y ago
You sound like you have no clue what youre talking about. The limitations of alexa and complaints about alexa are really complaints about the limits of NLP, which have just been expanded.
- smt88 4y ago
  You sound like you have no idea what you're talking about when you say that ChatGPT is a magic fix for Alexa. I gave concrete examples of: - why ChatGPT's style of "convincing bullshit" is a bad fit for a user interface like Alexa - failure modes of NLP in UIs - why NLP still won't save Alexa if it gets to a human level of accuracy because humans aren't accurate enough If you have any actual knowledge or argument other than, "You're ignorant, [technology that doesn't exist] is going to fix Alexa," then let's hear it.
  - fdgsdfogijq 4y ago
    I worked on core ML systems in alexa :) My knowledge is that I know how these systems work, and why its so hard to get them to work properly.
    - smt88 4y ago
      This comment still has no content in it other than "this is how it is" and "trust me bro". Do you have anything substantial to add? Do you have any reason to refute anything I said? It doesn't matter if you're Amazon's ML expert or the world's best surgeon or a barista if all you do is make an unsubstantiated statement and give your credentials.
      - fdgsdfogijq 4y ago
        No because its proprietary, and secondly all of your arguments are made out of thin air. Do you want me to produce a design doc about why getting alexa to work correctly is hard?
        shigawire 4y ago
        At the minimum explain how LLMs would make it much better... At least for the slow kids like me who don't know how LLMs or Alexa work.
at_a_remove 4y ago
I quasi-jokingly have suggested that we will end up with a conlang like lojban (which is designed to be unambiguous) as a kind of "command language" to machines.
- smt88 4y ago
  It's definitely not a joke at all. It's exactly how I use the voice assistants. But unlike a traditional programming language, there's no documentation, it's non-deterministic, and the same command will suddenly stop working. It's insane. So here's my process with Google Asisstant: - guess what it will understand through trial and error - use that command pattern, like "set [something] to [level]%", for a few months - try it again one day, and suddenly it doesn't work It makes you scared to trust it in almost any situation.
  - crooked-v 4y ago
    Siri has gone through phases at least twice where it forgot what the word "half" meant in relation to lights, then suddenly understood it again several months later.
    - smt88 4y ago
      I've extensively used Google Assistant and Siri. I've used Alexa a little bit, too. Google is like 4/10, Alexa is 5/10, and Siri is 0/10. Siri is stunningly bad. It understands the command ~50% of the time, and then the action it takes is useful (i.e. prevents me from needing to touch my phone) ~30% of the time.
astrange 4y ago
> Imagine you say, "Alexa, turn up the air conditioning," and Alexa interprets this to mean you want to increase the temperature. There are uncountable and unforeseeable failure modes. "up" and "off" sound similar too, and I've had assistants confuse them often.

by smt88 4y ago

fdgsdfogijq 4y ago

You sound like you have no clue what youre talking about. The limitations of alexa and complaints about alexa are really complaints about the limits of NLP, which have just been expanded.

smt88 4y ago
You sound like you have no idea what you're talking about when you say that ChatGPT is a magic fix for Alexa. I gave concrete examples of: - why ChatGPT's style of "convincing bullshit" is a bad fit for a user interface like Alexa - failure modes of NLP in UIs - why NLP still won't save Alexa if it gets to a human level of accuracy because humans aren't accurate enough If you have any actual knowledge or argument other than, "You're ignorant, [technology that doesn't exist] is going to fix Alexa," then let's hear it.
- fdgsdfogijq 4y ago
  I worked on core ML systems in alexa :) My knowledge is that I know how these systems work, and why its so hard to get them to work properly.
  - smt88 4y ago
    This comment still has no content in it other than "this is how it is" and "trust me bro". Do you have anything substantial to add? Do you have any reason to refute anything I said? It doesn't matter if you're Amazon's ML expert or the world's best surgeon or a barista if all you do is make an unsubstantiated statement and give your credentials.
    - fdgsdfogijq 4y ago
      No because its proprietary, and secondly all of your arguments are made out of thin air. Do you want me to produce a design doc about why getting alexa to work correctly is hard?
      - shigawire 4y ago
        At the minimum explain how LLMs would make it much better... At least for the slow kids like me who don't know how LLMs or Alexa work.

at_a_remove 4y ago

I quasi-jokingly have suggested that we will end up with a conlang like lojban (which is designed to be unambiguous) as a kind of "command language" to machines.

smt88 4y ago
It's definitely not a joke at all. It's exactly how I use the voice assistants. But unlike a traditional programming language, there's no documentation, it's non-deterministic, and the same command will suddenly stop working. It's insane. So here's my process with Google Asisstant: - guess what it will understand through trial and error - use that command pattern, like "set [something] to [level]%", for a few months - try it again one day, and suddenly it doesn't work It makes you scared to trust it in almost any situation.
- crooked-v 4y ago
  Siri has gone through phases at least twice where it forgot what the word "half" meant in relation to lights, then suddenly understood it again several months later.
  - smt88 4y ago
    I've extensively used Google Assistant and Siri. I've used Alexa a little bit, too. Google is like 4/10, Alexa is 5/10, and Siri is 0/10. Siri is stunningly bad. It understands the command ~50% of the time, and then the action it takes is useful (i.e. prevents me from needing to touch my phone) ~30% of the time.

astrange 4y ago

> Imagine you say, "Alexa, turn up the air conditioning," and Alexa interprets this to mean you want to increase the temperature. There are uncountable and unforeseeable failure modes. "up" and "off" sound similar too, and I've had assistants confuse them often.