5 ms·

These are all subtly different problems I think, but in general most of these architectures currently assume there is a single intent. > stop music and turn of

by Closi 4y ago

These are all subtly different problems I think, but in general most of these architectures currently assume there is a single intent. > stop music and turn off all the lights This is probably the easiest of the bunch because you are asking it to perform two distinct actions. > turn off the bedroom light in 5 minutes This is much more complex, because you are asking the application to setup some sort of workflow - after it understands what you want it to do, it then has to work out how to execute that, which will be utilising the device API's / services. This is a simple example, but there are lots of permutations of different actions here, for example you might want to say "turn off the sound system once this song finishes playing" which assumes that the assistant has the capability to then understand you want it to create a task specifically waiting for the trigger of a particular song finishing playing, and that it has the ability to setup that trigger. > "keep the lights on” in a motion sensor system Now this is where the orchestration gets tricky - The assistant has to: * Work out that the lights are being affected by a motion sensor system, which is likely outside it's own platform. * Work out that your intent is that you want the assistant to override that. * Understand how to connect to the platform in order to control it. * Work out what parameter it is supposed to alter to achieve this task. * Override the existing users settings, and presumably reinstate the settings after some portion of time.

visarga 4y ago
Language models can generate code from text instructions. It just needs a training set to learn the target APIs. I expect in the next couple of years to see automated desktop operation (RPA) from text commands, generalising access over human interfaces. It's really a shame the good language models are not deployed as voice assistants. It would probably be expensive to offer and they don't have the scale necessary. Just to load one of these models you need a $100K computer.
- Closi 4y ago
  It also depends what's the biggest priority - I would assume there is a bigger 'quick win' from becoming more reliable at single-intent actions from a market/customer experience perspective rather than pursuing highly complex multi-intent statements. >99% of commands will be single intent, and they probably work 80% of the time at the moment, so getting those to 99% will have a much bigger short-term impact than focussing on solving the 1% (with the added benfit that once you have solved the first case of getting single-intent right all the time, solving the second more complex queries will be easier as you will have built a more robust base).
- petra 4y ago
  I'm curious: what's the leading edge in commercial chatbots?
  - speed_spread 4y ago
    Hello, I'm Gretta! I'm here to help. It seems you're looking for information on enterprise chatbots geometry. Answer "yes" to access your account's balance or press # for more options.
omega3 4y ago
> This is much more complex, because you are asking the application to setup some sort of workflow - after it understands what you want it to do, it then has to work out how to execute that, which will be utilising the device API's / services. I don't see conceptually the difference between the first and the second example. You're still executing two distinct actions, first being the waiting for x amount of time?
- dietr1ch 4y ago
  The deferred execution makes this more complex as now the yet to be executed part needs to be queued somewhere It could be issuing a command in the same way you do the first, but probably most lights just support switching them on/off right now, and won't take a delay argument. So, if you can't just do the same you were doing, how do you do it? If there's some kind of local support, you can issue a delay command to some device in the network that will switch the lights on, but you still need to be able to control that action as the user might want to cancel or adjust the delay too. If the execution need to happen remotely because there's no local support, then you don't even know that you'll be able to reach the device in 5 minutes (Internet down, router not letting random connections in). And keeping this request queued needs some infra in the server side too together with the necessary APIs to allow adjusting and cancelling requests.
  - hmryehbut 4y ago
    So it requires things software engineers do all the time. Where is the hard part?
  - NLPlatypus 4y ago
    But you can apparently tell it “remember I put my keys upstairs”, and when you ask where your keys are, it tells you upstairs. There is storage going on already
  - shepherdjerred 4y ago
    > If there's some kind of local support, you can issue a delay command to some device in the network that will switch the lights on, but you still need to be able to control that action as the user might want to cancel or adjust the delay too. I'm pretty sure there is some form of local support, at least for Google. Whne I last used it you had to choose a speaker that automations executed from
avsteele 4y ago
notwithstanding voice->text part: Is it just me or does it seem like these tasks would be not all that hard if you just, you know, programmed them rather than trying to be so fancy with ML?
- Closi 4y ago
  I think the issue is the number of permutations of the tasks. i.e. you can program the use case "turn off the lights in 5 minutes" but that doesn't cover the use cases of "turn off the lights when i leave the house", "turn off the lights and turn on the tv" or "turn on the lights at 10pm tonight" - and there are so many potential scenarios here that it can quickly scale up.
- deleted 4y ago
  [deleted]
- nl 4y ago
  > Is it just me or does it seem like these tasks would be not all that hard if you just, you know, programmed them rather than trying to be so fancy with ML? That's how you do it at the moment. For example, in Google home you have "Routines"[1] which are a list of sequential actions triggered by a key phrase (or optionally something else). [1] https://support.google.com/googlenest/answer/7029585 https://support.google.com/googlenest/answer/7029585?
nomel 4y ago
> turn off the bedroom light in 5 minutes I can already do something similar with Siri. "Remind me to take out the trash in 5 minutes". Seems odd that "turn off the bedroom light" isn't trivial to support.
- sib 4y ago
  The iOS reminders app already has a concept of "remind user <x> at <y> time" so Siri is just creating one of those. The bedroom light may not have such a structured concept already existing (or may not expose the full-featured API to Siri).
  - Gordonjcp 4y ago
    If it can turn lights on and off, and separately play some sort of reminder sound after five minutes, then it can turn the lights off after five minutes.
    - cypress66 4y ago
      It just pipes the command to the appropriate app/service. That's why it doesn't work. Reminder service supports that, but lights service doesn't. There is no "general ai" doing stuff in the background.
      - Gordonjcp 4y ago
        The lights service doesn't have to know anything about the reminder service. The reminder service needs "one line of code (tm)" to go "After five minutes, send <this> to the lights service".
        Closi 4y ago
        The problem is that there are so many different permutations of this that the reminder service (and every other service) would then need to accomodate for hundreds of other scenarios, such as: - When I arrive at home, turn on the lights - After five minutes, play this song - Lock the front door at 10pm And then the reminders service (which is a to-do app!) suddenly becomes 100x more complicated than it was to accoomodate all these strange use-cases that are outside of it's bounded context. It's easy to implement a single use case, it's practically impossible to manually code every possible use case (which would also probably create a huge amount of tech debt).

by Closi 4y ago

visarga 4y ago

Language models can generate code from text instructions. It just needs a training set to learn the target APIs. I expect in the next couple of years to see automated desktop operation (RPA) from text commands, generalising access over human interfaces. It's really a shame the good language models are not deployed as voice assistants. It would probably be expensive to offer and they don't have the scale necessary. Just to load one of these models you need a $100K computer.

Closi 4y ago
It also depends what's the biggest priority - I would assume there is a bigger 'quick win' from becoming more reliable at single-intent actions from a market/customer experience perspective rather than pursuing highly complex multi-intent statements. >99% of commands will be single intent, and they probably work 80% of the time at the moment, so getting those to 99% will have a much bigger short-term impact than focussing on solving the 1% (with the added benfit that once you have solved the first case of getting single-intent right all the time, solving the second more complex queries will be easier as you will have built a more robust base).
petra 4y ago
I'm curious: what's the leading edge in commercial chatbots?
- speed_spread 4y ago
  Hello, I'm Gretta! I'm here to help. It seems you're looking for information on enterprise chatbots geometry. Answer "yes" to access your account's balance or press # for more options.

omega3 4y ago

> This is much more complex, because you are asking the application to setup some sort of workflow - after it understands what you want it to do, it then has to work out how to execute that, which will be utilising the device API's / services. I don't see conceptually the difference between the first and the second example. You're still executing two distinct actions, first being the waiting for x amount of time?

dietr1ch 4y ago
The deferred execution makes this more complex as now the yet to be executed part needs to be queued somewhere It could be issuing a command in the same way you do the first, but probably most lights just support switching them on/off right now, and won't take a delay argument. So, if you can't just do the same you were doing, how do you do it? If there's some kind of local support, you can issue a delay command to some device in the network that will switch the lights on, but you still need to be able to control that action as the user might want to cancel or adjust the delay too. If the execution need to happen remotely because there's no local support, then you don't even know that you'll be able to reach the device in 5 minutes (Internet down, router not letting random connections in). And keeping this request queued needs some infra in the server side too together with the necessary APIs to allow adjusting and cancelling requests.
- hmryehbut 4y ago
  So it requires things software engineers do all the time. Where is the hard part?
- NLPlatypus 4y ago
  But you can apparently tell it “remember I put my keys upstairs”, and when you ask where your keys are, it tells you upstairs. There is storage going on already
- shepherdjerred 4y ago
  > If there's some kind of local support, you can issue a delay command to some device in the network that will switch the lights on, but you still need to be able to control that action as the user might want to cancel or adjust the delay too. I'm pretty sure there is some form of local support, at least for Google. Whne I last used it you had to choose a speaker that automations executed from

avsteele 4y ago

notwithstanding voice->text part: Is it just me or does it seem like these tasks would be not all that hard if you just, you know, programmed them rather than trying to be so fancy with ML?

Closi 4y ago
I think the issue is the number of permutations of the tasks. i.e. you can program the use case "turn off the lights in 5 minutes" but that doesn't cover the use cases of "turn off the lights when i leave the house", "turn off the lights and turn on the tv" or "turn on the lights at 10pm tonight" - and there are so many potential scenarios here that it can quickly scale up.
deleted 4y ago
[deleted]
nl 4y ago
> Is it just me or does it seem like these tasks would be not all that hard if you just, you know, programmed them rather than trying to be so fancy with ML? That's how you do it at the moment. For example, in Google home you have "Routines"[1] which are a list of sequential actions triggered by a key phrase (or optionally something else). [1] https://support.google.com/googlenest/answer/7029585 https://support.google.com/googlenest/answer/7029585?

nomel 4y ago

> turn off the bedroom light in 5 minutes I can already do something similar with Siri. "Remind me to take out the trash in 5 minutes". Seems odd that "turn off the bedroom light" isn't trivial to support.

sib 4y ago
The iOS reminders app already has a concept of "remind user <x> at <y> time" so Siri is just creating one of those. The bedroom light may not have such a structured concept already existing (or may not expose the full-featured API to Siri).
- Gordonjcp 4y ago
  If it can turn lights on and off, and separately play some sort of reminder sound after five minutes, then it can turn the lights off after five minutes.
  - cypress66 4y ago
    It just pipes the command to the appropriate app/service. That's why it doesn't work. Reminder service supports that, but lights service doesn't. There is no "general ai" doing stuff in the background.
    - Gordonjcp 4y ago
      The lights service doesn't have to know anything about the reminder service. The reminder service needs "one line of code (tm)" to go "After five minutes, send <this> to the lights service".
      - Closi 4y ago
        The problem is that there are so many different permutations of this that the reminder service (and every other service) would then need to accomodate for hundreds of other scenarios, such as: - When I arrive at home, turn on the lights - After five minutes, play this song - Lock the front door at 10pm And then the reminders service (which is a to-do app!) suddenly becomes 100x more complicated than it was to accoomodate all these strange use-cases that are outside of it's bounded context. It's easy to implement a single use case, it's practically impossible to manually code every possible use case (which would also probably create a huge amount of tech debt).