4 ms·

I tried the conversational weather one on google assistant on my phone. "Should I take an umbrella tomorrow?" "No..." and shows me tomorrows forecast. "What

by IanCal 9y ago

I tried the conversational weather one on google assistant on my phone. "Should I take an umbrella tomorrow?" "No..." and shows me tomorrows forecast. "What about the day after?" "No..." and shows a forecast that when I look more closely at I notice is for today. Neither of these also spotted that I'm heading to another city tomorrow, which is in my calendar. If I changed it to "do I need to take an umbrella for my trip tomorrow" it just searches google and gives me a search result suggesting I take a small folding umbrella... for a trip to Thailand.

visarga 9y ago
It might be so, but remember that experts were predicting computer will beat top humans in Go in 10 years. Maybe next month there will be a breakthrough with NLP. The amount of research and compute going into this problem is amazing, and we don't know what's around the corner. Also, it might be possible to create a much better assistant today, but it would be too expensive to offer to the public for free. What if it requires 100 TPUs to run?
- IanCal 9y ago
  The thing for me in that example is it's not a particularly complex set of options. It knows I'm asking for the weather, it knows that it should give me the weather in a particular location, and google knows where I'll be tomorrow. The other main mistake was (despite it getting the conversational part right) thinking "the day after" means today. I don't think these things require TPUs or new NLP.
  - visarga 9y ago
    Yes, they managed assistants badly - Google, Apple and Amazon. They could have added more APIs, recognized many more patterns of user requests, and it's just plain old programming, so what's keeping them? For example, Google Assistant doesn't play nice with YouTube. I guess it isn't in Google's interest to have a free agent software serve a free music site when the same service makes money for Amazon (Alexa). They wanted to make their agent profitable, so didn't let it be extended for free. Just my guess.
    - johnsmith21006 9y ago
      Google Home with YouTube is amazing. I say play Gwen Bottle and get Gwen Stefanni sing Message in a bottle. Or say play One Love Peas and get Blach Eye Peas ad One Love concert. What are you talking about?
- marcosdumay 9y ago
  This one problem of NLP is kinda solved¹ for more than 10 years already. What you are seeing here is a much simpler CS problem, but much harder social problem. It is "why can't my applications talk to each other?". I really doubt it will be solved in 10 years. 1 - Humans don't have a perfect solution for it, and machines are still worse, but not that worse.
- opportune 9y ago
  This isn't an NLP problem, it's a coder problem. These solutions already exist, Google/ other assistant providers just need to dedicate the man hours to make it happen.
  - dilemma 9y ago
    Hardcoding isn't a solution
    - resf 9y ago
      Hardcoding is how the sausage is made.
      - IanCal 9y ago
        I agree. Hardcoding might not get me a great general purpose assistant, but then the generic solution is failing at that too. Hardcoding can get me to a useful subset of the functionality now.
  - nl 9y ago
    No, they don't (generally) I do NLP for work, and the edges cases are so frequent and aggregate over processing pipelines. That means the end to end performance of the system is bad, and it isn't clear how to systematically fix it. In this particular case there is the added system integration too, which is "just programming".
    - IanCal 9y ago
      Perhaps, these just feel like such common things to ask and do that I don't get why they're not planned for. The conversational side works for the weather, the remaining problems are: 1. There is an in-built assumption I am always where I currently am. This part isn't anything to do with the NLP. 2. "the day after" is translated to "today", possibly. [edit - see lower, it is in only one case] 3. This is more of an NLP one, it understands that the context of "weather" carries from one question to the next, but not the timing. So asking for the weather tomorrow and then "one day later" gets tomorrows as well. [edit - more complex than this, it's actually working in some areas and not in others] I'd like to see what user-stories it's trying to solve, because apart from setting timers and alarms it's been massively hit and miss for me. I tried to repeat what I'd put in and this time I had: "What's the weather the day after" - translated to tomorrow, with no context, that makes sense. "What's the weather today?" - weather today, followed by "What about the day after?" which gave me results about the film. "What's the weather tomorrow?" - weather tomorrow, followed by "what about the day after?" which then worked. So it works just fine for "weather" but not for asking if I need to take an umbrella. And it doesn't work if I ask for today then the day after, but does for tomorrow and the day after. Why does the context get passed on for the day correctly for "tomorrow" but not for "today"? Why can it get that "the day after" means tomorrow, unless I've asked about an umbrella in which case it means today? At the core of my question is how is it this inconsistent? What tests have they got around this?
jetpacktuxedo 9y ago
I use google assistant for controlling smart lights sometimes with "ok google, turn on/off the lights". At one point I tried "Ok google, turn off the lights in ten minutes" and it just searched it. That seems super simple and like something it shouldn't have had any trouble with, but here we are :/
dilemma 9y ago
Poor AI and these goal posts we keep moving.

by IanCal 9y ago

visarga 9y ago

It might be so, but remember that experts were predicting computer will beat top humans in Go in 10 years. Maybe next month there will be a breakthrough with NLP. The amount of research and compute going into this problem is amazing, and we don't know what's around the corner. Also, it might be possible to create a much better assistant today, but it would be too expensive to offer to the public for free. What if it requires 100 TPUs to run?

IanCal 9y ago
The thing for me in that example is it's not a particularly complex set of options. It knows I'm asking for the weather, it knows that it should give me the weather in a particular location, and google knows where I'll be tomorrow. The other main mistake was (despite it getting the conversational part right) thinking "the day after" means today. I don't think these things require TPUs or new NLP.
- visarga 9y ago
  Yes, they managed assistants badly - Google, Apple and Amazon. They could have added more APIs, recognized many more patterns of user requests, and it's just plain old programming, so what's keeping them? For example, Google Assistant doesn't play nice with YouTube. I guess it isn't in Google's interest to have a free agent software serve a free music site when the same service makes money for Amazon (Alexa). They wanted to make their agent profitable, so didn't let it be extended for free. Just my guess.
  - johnsmith21006 9y ago
    Google Home with YouTube is amazing. I say play Gwen Bottle and get Gwen Stefanni sing Message in a bottle. Or say play One Love Peas and get Blach Eye Peas ad One Love concert. What are you talking about?
marcosdumay 9y ago
This one problem of NLP is kinda solved¹ for more than 10 years already. What you are seeing here is a much simpler CS problem, but much harder social problem. It is "why can't my applications talk to each other?". I really doubt it will be solved in 10 years. 1 - Humans don't have a perfect solution for it, and machines are still worse, but not that worse.
opportune 9y ago
This isn't an NLP problem, it's a coder problem. These solutions already exist, Google/ other assistant providers just need to dedicate the man hours to make it happen.
- dilemma 9y ago
  Hardcoding isn't a solution
  - resf 9y ago
    Hardcoding is how the sausage is made.
    - IanCal 9y ago
      I agree. Hardcoding might not get me a great general purpose assistant, but then the generic solution is failing at that too. Hardcoding can get me to a useful subset of the functionality now.
- nl 9y ago
  No, they don't (generally) I do NLP for work, and the edges cases are so frequent and aggregate over processing pipelines. That means the end to end performance of the system is bad, and it isn't clear how to systematically fix it. In this particular case there is the added system integration too, which is "just programming".
  - IanCal 9y ago
    Perhaps, these just feel like such common things to ask and do that I don't get why they're not planned for. The conversational side works for the weather, the remaining problems are: 1. There is an in-built assumption I am always where I currently am. This part isn't anything to do with the NLP. 2. "the day after" is translated to "today", possibly. [edit - see lower, it is in only one case] 3. This is more of an NLP one, it understands that the context of "weather" carries from one question to the next, but not the timing. So asking for the weather tomorrow and then "one day later" gets tomorrows as well. [edit - more complex than this, it's actually working in some areas and not in others] I'd like to see what user-stories it's trying to solve, because apart from setting timers and alarms it's been massively hit and miss for me. I tried to repeat what I'd put in and this time I had: "What's the weather the day after" - translated to tomorrow, with no context, that makes sense. "What's the weather today?" - weather today, followed by "What about the day after?" which gave me results about the film. "What's the weather tomorrow?" - weather tomorrow, followed by "what about the day after?" which then worked. So it works just fine for "weather" but not for asking if I need to take an umbrella. And it doesn't work if I ask for today then the day after, but does for tomorrow and the day after. Why does the context get passed on for the day correctly for "tomorrow" but not for "today"? Why can it get that "the day after" means tomorrow, unless I've asked about an umbrella in which case it means today? At the core of my question is how is it this inconsistent? What tests have they got around this?

jetpacktuxedo 9y ago

I use google assistant for controlling smart lights sometimes with "ok google, turn on/off the lights". At one point I tried "Ok google, turn off the lights in ten minutes" and it just searched it. That seems super simple and like something it shouldn't have had any trouble with, but here we are :/

dilemma 9y ago

Poor AI and these goal posts we keep moving.