11 ms·

Pretty neat, but from a user's perspective I'm a little disappointed that the current implementation seems to require me to explicitly tell Google what app I wa

by Ajedi32 10y ago

Pretty neat, but from a user's perspective I'm a little disappointed that the current implementation seems to require me to explicitly tell Google what app I want to interact with. I'd rather be able to just ask Google "Find me a movie ticket for Rogue One on Thursday" and have it be able to infer that I want to use the movie tickets app installed on my phone, than for me to have to explicitly say "Let me talk to Fandango" first. That said, it looks like deeper integrations are coming: > We'll continue to add more platform capabilities over time, including the ability to make your integrations available across the various Assistant surfaces like Pixel phones and Google Allo. We'll also enable support for purchases and bookings as well as deeper Assistant integrations across verticals.

IshKebab 10y ago
There are two actions - conversation actions (where you have to say "Tell Uber I want a cab") and direct actions ("get me a cab"). Somewhat understandably the direct actions are limited to certain domains that Google supports. It isn't a free for all.
- Ajedi32 10y ago
  Yeah, I guess considering that there's no installation process for actions that's kind of a necessity. Maybe as the technology matures Google will start getting smarter about being able to infer which provider I want to use for particular types of actions, but I recognise that's likely to be a rather difficult problem to solve.
hirsin 10y ago
The problem with protocol activation is that there are a million different endpoints and you either * Keep a list of every single one that apps can opt into. This is how you end up with "midsized_sedan_carwashing" in an enum list. It's a developer drain for Google/Microsoft/Amazon and creates "wait for the next release for 'largetruck_carwashing' so you can release your app". * Let apps run wild and claim any protocol (in this case, the word "movie ticket") they like. This is how you get malware claiming the .pdf or http:// http:// protocols. * Let certain special apps run wild. This is how you end up with "Buy Tickets With Fandango" advertised on the side of the box while Brown Paper Tickets or another startup is left in the cold/ "ask brown paper tickets for..." No solutions here, it's a hard space if you want to protect users but also enable developers equally. In the first two cases you also need to create confusing UI for the user to select which app they want for movies vs drycleaning.
- TeMPOraL 10y ago
  That's why personally I want a fully configurable system in which I myself do the #3. But this ain't gonna happen, because nobody is building anything for advanced users anymore :<.
- Ajedi32 10y ago
  Maybe Assistant could somehow monitor what kind of things users are asking a particular app to do and use that to determine defaults for specific actions in future interactions. Imagine: "Okay Google, get me movie tickets for Rogue One on Tuesday" "I'm sorry, I don't know how to do that yet. Try telling me to ask another service like Fandango or MovieTickets.com." "Ask Fandango" "Okay. [Switches to Fandango.] What theater do you want to see Rogue One at?" --- Later when I ask for movie tickets again... "Okay Google, get me a movie ticket for Moana." "I'm not sure how to do that. Would you like me to ask Fandango?" "Sure" "Okay. [Switches to Fandango.] What theater do you want to see Moana at?" --- Later still... "Okay Google, get me a movie ticket for Fantastic Beasts and Where to Find Them." "Sure, let me ask Fandango... [Switches to Fandango.]" --- You get the idea. And if a user ever wants to use a different service, they can always ask for it explicitly.
  - sixdimensional 10y ago
    I think the Google voice recognition service (triggered by "Ok Google") on my phone already does something similar. When I used to say to call a particular person, it would ask "which number, home or mobile?", for example. Over time, I usually always ask for the same number for a particular contact (mobile), and it learned that when I ask for the first name of the contact, it usually always means a specific contact. So whereas initially I had to say the full name and pick a number, now all I say is "Call <first name>" and it starts dialing the exact contact I wanted. It also knows that this first name is my wife... so I can even say "call my wife" and it also works. This lends credence to the model you're describing, and maybe that's exactly how it will work in the future. I think machine learning sorts this out quite well.
  - pkulak 10y ago
    That's probably step 2 (or 8). The current situation seems like a good step one, though.

by Ajedi32 10y ago

IshKebab 10y ago

There are two actions - conversation actions (where you have to say "Tell Uber I want a cab") and direct actions ("get me a cab"). Somewhat understandably the direct actions are limited to certain domains that Google supports. It isn't a free for all.

Ajedi32 10y ago
Yeah, I guess considering that there's no installation process for actions that's kind of a necessity. Maybe as the technology matures Google will start getting smarter about being able to infer which provider I want to use for particular types of actions, but I recognise that's likely to be a rather difficult problem to solve.

hirsin 10y ago

The problem with protocol activation is that there are a million different endpoints and you either * Keep a list of every single one that apps can opt into. This is how you end up with "midsized_sedan_carwashing" in an enum list. It's a developer drain for Google/Microsoft/Amazon and creates "wait for the next release for 'largetruck_carwashing' so you can release your app". * Let apps run wild and claim any protocol (in this case, the word "movie ticket") they like. This is how you get malware claiming the .pdf or http:// http:// protocols. * Let certain special apps run wild. This is how you end up with "Buy Tickets With Fandango" advertised on the side of the box while Brown Paper Tickets or another startup is left in the cold/ "ask brown paper tickets for..." No solutions here, it's a hard space if you want to protect users but also enable developers equally. In the first two cases you also need to create confusing UI for the user to select which app they want for movies vs drycleaning.

TeMPOraL 10y ago
That's why personally I want a fully configurable system in which I myself do the #3. But this ain't gonna happen, because nobody is building anything for advanced users anymore :<.
Ajedi32 10y ago
Maybe Assistant could somehow monitor what kind of things users are asking a particular app to do and use that to determine defaults for specific actions in future interactions. Imagine: "Okay Google, get me movie tickets for Rogue One on Tuesday" "I'm sorry, I don't know how to do that yet. Try telling me to ask another service like Fandango or MovieTickets.com." "Ask Fandango" "Okay. [Switches to Fandango.] What theater do you want to see Rogue One at?" --- Later when I ask for movie tickets again... "Okay Google, get me a movie ticket for Moana." "I'm not sure how to do that. Would you like me to ask Fandango?" "Sure" "Okay. [Switches to Fandango.] What theater do you want to see Moana at?" --- Later still... "Okay Google, get me a movie ticket for Fantastic Beasts and Where to Find Them." "Sure, let me ask Fandango... [Switches to Fandango.]" --- You get the idea. And if a user ever wants to use a different service, they can always ask for it explicitly.
- sixdimensional 10y ago
  I think the Google voice recognition service (triggered by "Ok Google") on my phone already does something similar. When I used to say to call a particular person, it would ask "which number, home or mobile?", for example. Over time, I usually always ask for the same number for a particular contact (mobile), and it learned that when I ask for the first name of the contact, it usually always means a specific contact. So whereas initially I had to say the full name and pick a number, now all I say is "Call <first name>" and it starts dialing the exact contact I wanted. It also knows that this first name is my wife... so I can even say "call my wife" and it also works. This lends credence to the model you're describing, and maybe that's exactly how it will work in the future. I think machine learning sorts this out quite well.
- pkulak 10y ago
  That's probably step 2 (or 8). The current situation seems like a good step one, though.