4 ms·

I've interviewed many people on Alexa before. From what I gather, its just a giant switch statement, and each individual "path" takes a bunch of effort to suppo

I've interviewed many people on Alexa before. From what I gather, its just a giant switch statement, and each individual "path" takes a bunch of effort to support and there are thousands of paths for music, ordering, commands, etc. It's peak AI == if statement architecture.

laidoffamazon 2y ago
I think describing the NLU Engine as a switch statement is underselling it a bit. Determining domain and intent alone requires more than that (frequently, at least).
SahAssar 2y ago
Do you mean actual coded if statements as in actually human written code like if (question.match(/^what is (.*)/)) return wikipedia.search(question) or something more automated?
- laidoffamazon 2y ago
  Not like that, but once you get to first party Alexa Skill themselves there's a bunch of match rules that make up a big chunk of the traffic. Don't know exactly how much. The longtail is done through ML means.
ihkasfjdkabnsk 2y ago
Throwaway, used to work at the NLU unit of Alexa about 5 years ago. There is some ML going on but as with all ML projects I have worked on people want control. This means you add rules for the "important" stuff. You also add test cases to make sure the ML works. But if you already have those test cases, why not just match on them directly? There are also advanced techniques for generating examples (FST for example). What this culminated in is a platform where 80% of request, and pretty much 99% of "commands" are served by rules built with a team of linguists.
- laidoffamazon 2y ago
  Later on, there were methods to generate FSTs themselves without manual human curation.
- rtpg 2y ago
  Is this not actually kind of powerful? Having linguists write up a bunch of rules seems a lot more predictable than "rolling a bunch of dice and hoping that some LLM spits out a coherent set of steps". It feels very fractal but on the other hand if Alexa has only a specific gamut of responses it's not exactly a limitless state space right? Very curious about how those rules look like though
  - factormeta 2y ago
    wouldn't that just be some kind of NLP https://en.wikipedia.org/wiki/Natural_language_processing https://en.wikipedia.org/wiki/Natural_language_processing? May just be a long list of if/else and/or switch statements or isomorphism.
  - ihkasfjdkabnsk 2y ago
    Honestly the FSTs themselves were actually really cool, it's very much GOFAI. It automatically creates lots of permutations, i.e. `play taylor swift`, `please play taylor swift`, play taylor swift now`. etc. And once the FST is built it always works deterministically. It's compiled to a graph and an incoming command is pushed through the state machine, if you get to an end state it "matched the fst" and some specific behaviour would be triggered. the rule were really just strings and we had efficient matching against it. I didn't work on that, I would assume some sort of LHS.
    - phito 2y ago
      what do all these acronyms mean
      - 39896880 2y ago
        https://en.wikipedia.org/wiki/Finite-state_transducer https://en.wikipedia.org/wiki/Finite-state_transducer https://medium.com/fetch-ai/classical-roots-of-ai-unpacking-gofai-1afce52de53b#:~:text=The%20term%20GOFAI%2C%20introduced%20by,methods%20to%20mimic%20intelligent%20behavior https://medium.com/fetch-ai/classical-roots-of-ai-unpacking-....
novok 2y ago
IMO with my experience with siri being AI-style unreliable in many ways, like bit flips when saying turn off the lights makes the dimmer go to %100, I think it's better to do the switch statement for the dozen or so query types that probably represent %90 of traffic, like weather, music, home control, unit conversions, etc in exchange for way more reliability.
visarga 2y ago
> I've interviewed many people on Alexa before. I got a recruiting message once for a ML engineer position in Alexa, ignored it.

laidoffamazon 2y ago

I think describing the NLU Engine as a switch statement is underselling it a bit. Determining domain and intent alone requires more than that (frequently, at least).

SahAssar 2y ago

Do you mean actual coded if statements as in actually human written code like if (question.match(/^what is (.*)/)) return wikipedia.search(question) or something more automated?

laidoffamazon 2y ago
Not like that, but once you get to first party Alexa Skill themselves there's a bunch of match rules that make up a big chunk of the traffic. Don't know exactly how much. The longtail is done through ML means.

ihkasfjdkabnsk 2y ago

Throwaway, used to work at the NLU unit of Alexa about 5 years ago. There is some ML going on but as with all ML projects I have worked on people want control. This means you add rules for the "important" stuff. You also add test cases to make sure the ML works. But if you already have those test cases, why not just match on them directly? There are also advanced techniques for generating examples (FST for example). What this culminated in is a platform where 80% of request, and pretty much 99% of "commands" are served by rules built with a team of linguists.

laidoffamazon 2y ago
Later on, there were methods to generate FSTs themselves without manual human curation.
rtpg 2y ago
Is this not actually kind of powerful? Having linguists write up a bunch of rules seems a lot more predictable than "rolling a bunch of dice and hoping that some LLM spits out a coherent set of steps". It feels very fractal but on the other hand if Alexa has only a specific gamut of responses it's not exactly a limitless state space right? Very curious about how those rules look like though
- factormeta 2y ago
  wouldn't that just be some kind of NLP https://en.wikipedia.org/wiki/Natural_language_processing https://en.wikipedia.org/wiki/Natural_language_processing? May just be a long list of if/else and/or switch statements or isomorphism.
- ihkasfjdkabnsk 2y ago
  Honestly the FSTs themselves were actually really cool, it's very much GOFAI. It automatically creates lots of permutations, i.e. `play taylor swift`, `please play taylor swift`, play taylor swift now`. etc. And once the FST is built it always works deterministically. It's compiled to a graph and an incoming command is pushed through the state machine, if you get to an end state it "matched the fst" and some specific behaviour would be triggered. the rule were really just strings and we had efficient matching against it. I didn't work on that, I would assume some sort of LHS.
  - phito 2y ago
    what do all these acronyms mean
    - 39896880 2y ago
      https://en.wikipedia.org/wiki/Finite-state_transducer https://en.wikipedia.org/wiki/Finite-state_transducer https://medium.com/fetch-ai/classical-roots-of-ai-unpacking-gofai-1afce52de53b#:~:text=The%20term%20GOFAI%2C%20introduced%20by,methods%20to%20mimic%20intelligent%20behavior https://medium.com/fetch-ai/classical-roots-of-ai-unpacking-....

novok 2y ago

IMO with my experience with siri being AI-style unreliable in many ways, like bit flips when saying turn off the lights makes the dimmer go to %100, I think it's better to do the switch statement for the dozen or so query types that probably represent %90 of traffic, like weather, music, home control, unit conversions, etc in exchange for way more reliability.

visarga 2y ago

> I've interviewed many people on Alexa before. I got a recruiting message once for a ML engineer position in Alexa, ignored it.