8 ms·

I was an early Alexa PM for Skills back in 2017. Even back then when Alexa had all the hype and first mover advantage, it was extremely difficult to provide any

I was an early Alexa PM for Skills back in 2017. Even back then when Alexa had all the hype and first mover advantage, it was extremely difficult to provide any meaningful traction to “Skills developers” for various reasons (friction in VUI, discoverability issues mainly) Fast forward to 2023 when OpenAI came up with GPT Store, it felt like a deja vu of Alexa skills in some sense. My understanding is that GPTs are also struggling to gain good traction. There are some fundamental reasons why conversational 3rd party platforms are hard. If there’s any interest I will write about it in the Enterprise AI substack (nextword.substack.com)

jeremyjh 3y ago
The problem is not discoverability. The problem is there is nothing there to be discovered that is useful or entertaining enough for people to pay money for. The only thing surprising here is that it took so long for Amazon to admit its a failure.
- nextworddev 3y ago
  You need discoverability to incentivize innovators to add more value, which itself feeds back into the value of the overall ecosystem, etc. But yes, natural language as an interface requires higher activation energy versus GUI which hampers how much value Alexa Skills (and GPT) can provide.
  - criddell 3y ago
    You do need discoverability, but you also need a capable platform and things worth discovering. When I look at Amazon’s skills page, it’s a wasteland. Until there’s something worth surfacing, discoverability doesn’t really matter. That said, I do have two Echos in my house (and one Home Pod) and use them all the time for basic things.
- IshKebab 3y ago
  I disagree. I have used Alexa and Google Assistant a lot and also developed an Alexa skill for controlling Dyson's robot vacuum cleaners, so I'm pretty familiar. The problem is that the set of supported operations are always MUCH smaller than the set of operations people randomly try. You might develop a skill with 200 commands or whatever and think you've covered everything, but people can up with thousands of possible commands just by guessing. This means if people just do "I'll try asking this..." then probably 80% of the time it won't work. That's an incredibly frustrating experience. You quickly give up and just stick to the features that you know work, and never try to find any new ones. But I also disagree that OpenAI has the same problem, because LLMs means you don't need to manually add thousands of possible commands, so any random request that people make is MUCH more likely to work.
  - jeremyjh 3y ago
    Honestly I'm disappointed Amazon hasn't married these two technologies either, but I think it must just be too expensive. I would think you could even prompt a midrange LLM to process the request and generate structured data for the limited list of supported prompts for a skill.
    - IshKebab 3y ago
      Too expensive and they're probably scared about how uncontrollable and unreliable LLMs are. Can't say I blame them really. Every time a company deploys an LLM you get a news story like "hurr durr I tricked it into saying poopy pants" or whatever.
simonw 3y ago
Please do write about that. I've seen so many Alexa projects at hackathons that were exciting at first glance but didn't go anywhere. My favourite is event recommendations: lots of people try building an Alexa thing to help you find events, but it turns out listening to a bot read out a list of 20 things happening this weekend is way less useful than browsing a web page with the same information all on a screen at once.
- nextworddev 3y ago
  The core problem lies in latency and information bandwidth - for which eyes (which sees the “gestalt”) are just far superior to reading or listening to streaming text (which is linear). The one main situation where NL interfaces are superior is when you are mobile (like driving) or hands are tied up. I think this affects GPTs just like it did Alexa. Which means that GPTs aren’t the final UI. The real innovation will be in the right AI UX.
  - ethbr1 3y ago
    Conversational interfaces are superior... when the counterparties have context. In the parent situation, I don't want a GPT spoken interface to give me the top 20 events: I want it to give me 1-2 events that I am most likely to enjoy. In the same way that actual conversations take into account tone, facial expression, etc. to jump straight to important information. I thought that's where Google was going with their "we have all your data because we run all your services", but it seems they Microsoftified before they could get services cooperating for the larger good.
    - hx8 3y ago
      > Conversational interfaces are superior... when the counterparties have context. Why? I find conversational interfaces poor for common data retrieval. I can read faster than you can speak. I can type faster than I can speak. I'm staring at a screen 14 hours a day anyway. Just show me the list of 20 events and sort it by what I am most likely to enjoy. Provide links for more information. Show me visual promotional materials. If I need to cross reference it with my calendar it's easier if all the information is visual.
meowtimemania 3y ago
Aren't the GPT's consumed in a text based UI? Lots of avenues for discoverability there
- vineyardmike 3y ago
  How? “Discoverability” here means the ability for the user to learn the systems abilities. Historically, you could read a menu or list of app icons. A chat box is empty - it tells the user nothing about the systems abilities, just its interface. A smart speaker is even worse - there’s minimal UX hints. Asking a chat box “what can you do” is not likely to be exhaustive, and and will likely require a series of “can you do X” queries.
  - ZiiS 3y ago
    We need to get to the point the very least likely response is "I am sorry Dave I can't do that". Obviously, I am not going to ask Alexa to take out the rubbish; but any reasonable request needs to have a >>50% chance of working; rather than a <<1% chance.
  - meowtimemania 3y ago
    Show suggested messages that mention specific gpt’s. After each message show suggested follow-ups
    - vineyardmike 2y ago
      > After each message show suggested follow-ups Ask people how they feel about Alexa's "follow up" suggestions ;) It's not easy to do discoverability in general. Especially not through an intentionally limited modality. People study this stuff, many businesses and product researchers have spent years workshopping ideas. If it was a quick and easy idea like "just tell the user" then it wouldn't be a challenge in 2024. The problem with "show messages... suggest followups" is that you can't teach the user about new features that are unrelated to their current interaction because it feels like advertising, and its distracting to their current task at hand.

jeremyjh 3y ago

The problem is not discoverability. The problem is there is nothing there to be discovered that is useful or entertaining enough for people to pay money for. The only thing surprising here is that it took so long for Amazon to admit its a failure.

nextworddev 3y ago
You need discoverability to incentivize innovators to add more value, which itself feeds back into the value of the overall ecosystem, etc. But yes, natural language as an interface requires higher activation energy versus GUI which hampers how much value Alexa Skills (and GPT) can provide.
- criddell 3y ago
  You do need discoverability, but you also need a capable platform and things worth discovering. When I look at Amazon’s skills page, it’s a wasteland. Until there’s something worth surfacing, discoverability doesn’t really matter. That said, I do have two Echos in my house (and one Home Pod) and use them all the time for basic things.
IshKebab 3y ago
I disagree. I have used Alexa and Google Assistant a lot and also developed an Alexa skill for controlling Dyson's robot vacuum cleaners, so I'm pretty familiar. The problem is that the set of supported operations are always MUCH smaller than the set of operations people randomly try. You might develop a skill with 200 commands or whatever and think you've covered everything, but people can up with thousands of possible commands just by guessing. This means if people just do "I'll try asking this..." then probably 80% of the time it won't work. That's an incredibly frustrating experience. You quickly give up and just stick to the features that you know work, and never try to find any new ones. But I also disagree that OpenAI has the same problem, because LLMs means you don't need to manually add thousands of possible commands, so any random request that people make is MUCH more likely to work.
- jeremyjh 3y ago
  Honestly I'm disappointed Amazon hasn't married these two technologies either, but I think it must just be too expensive. I would think you could even prompt a midrange LLM to process the request and generate structured data for the limited list of supported prompts for a skill.
  - IshKebab 3y ago
    Too expensive and they're probably scared about how uncontrollable and unreliable LLMs are. Can't say I blame them really. Every time a company deploys an LLM you get a news story like "hurr durr I tricked it into saying poopy pants" or whatever.

simonw 3y ago

Please do write about that. I've seen so many Alexa projects at hackathons that were exciting at first glance but didn't go anywhere. My favourite is event recommendations: lots of people try building an Alexa thing to help you find events, but it turns out listening to a bot read out a list of 20 things happening this weekend is way less useful than browsing a web page with the same information all on a screen at once.

nextworddev 3y ago
The core problem lies in latency and information bandwidth - for which eyes (which sees the “gestalt”) are just far superior to reading or listening to streaming text (which is linear). The one main situation where NL interfaces are superior is when you are mobile (like driving) or hands are tied up. I think this affects GPTs just like it did Alexa. Which means that GPTs aren’t the final UI. The real innovation will be in the right AI UX.
- ethbr1 3y ago
  Conversational interfaces are superior... when the counterparties have context. In the parent situation, I don't want a GPT spoken interface to give me the top 20 events: I want it to give me 1-2 events that I am most likely to enjoy. In the same way that actual conversations take into account tone, facial expression, etc. to jump straight to important information. I thought that's where Google was going with their "we have all your data because we run all your services", but it seems they Microsoftified before they could get services cooperating for the larger good.
  - hx8 3y ago
    > Conversational interfaces are superior... when the counterparties have context. Why? I find conversational interfaces poor for common data retrieval. I can read faster than you can speak. I can type faster than I can speak. I'm staring at a screen 14 hours a day anyway. Just show me the list of 20 events and sort it by what I am most likely to enjoy. Provide links for more information. Show me visual promotional materials. If I need to cross reference it with my calendar it's easier if all the information is visual.

meowtimemania 3y ago

Aren't the GPT's consumed in a text based UI? Lots of avenues for discoverability there

vineyardmike 3y ago
How? “Discoverability” here means the ability for the user to learn the systems abilities. Historically, you could read a menu or list of app icons. A chat box is empty - it tells the user nothing about the systems abilities, just its interface. A smart speaker is even worse - there’s minimal UX hints. Asking a chat box “what can you do” is not likely to be exhaustive, and and will likely require a series of “can you do X” queries.
- ZiiS 3y ago
  We need to get to the point the very least likely response is "I am sorry Dave I can't do that". Obviously, I am not going to ask Alexa to take out the rubbish; but any reasonable request needs to have a >>50% chance of working; rather than a <<1% chance.
- meowtimemania 3y ago
  Show suggested messages that mention specific gpt’s. After each message show suggested follow-ups
  - vineyardmike 2y ago
    > After each message show suggested follow-ups Ask people how they feel about Alexa's "follow up" suggestions ;) It's not easy to do discoverability in general. Especially not through an intentionally limited modality. People study this stuff, many businesses and product researchers have spent years workshopping ideas. If it was a quick and easy idea like "just tell the user" then it wouldn't be a challenge in 2024. The problem with "show messages... suggest followups" is that you can't teach the user about new features that are unrelated to their current interaction because it feels like advertising, and its distracting to their current task at hand.