9 ms·

Startup idea: Zapier for consumer apps

42 points by indieept 3y ago

I've been wondering why there haven't been any really useful 'digital personal assistants' developed over the last few months. There are several recurring use cases for which it could be incredibly useful (ordering Uber/Uber Eats, booking restaurants, booking tennis courts, etc). Wouldn't it be nice to just write, 'Book me a tennis court at @club today at 6 pm for 1 hour' or 'schedule UberX for today at 3 pm, pickup at my place, drop off at the airport'? Can't one just get access to these consumer apps' APIs and develop this? Cheers from Portugal

dbish 3y ago
This is basically every virtual assistant’s pitch since forever, including Cortana, Siri, and Alexa. They’ve all tried some version of this, with almost no customer usage (most famous 3rd kart friendly one being Alexa skills).
- indieept 3y ago
  Why do you think it never got traction? I believe the key here should be to be OS-agnostic and to focus on specific tasks that users perform on a recurring/weekly basis and that are simple to automate
  - ToJans 3y ago
    My best bet would be to find a tiny, single, super-annoying use case that is instantly fixed with the tool, and grow an audience from there...
  - dbish 3y ago
    I don’t think people actually want this even though they talk about it unless it’s basically perfect which the tech still isn’t there for. It’s like the “book a flight” use case that has been used or eons, unless you’re rich (in which case you probably have an assistant), you’re going to double check and second guess anything that is booked for you so you might as well do it yourself. There are very few things that people do weekly at this point that aren’t already somewhat automatable if they cared to.
louison11 3y ago
This is the dream of "ubiquitous" computing that Silicon Valley has been aiming at for two decades. Computing so easy it's almost invisible. You just say what you want to happen and it happens nearly instantly. The biggest problem is that APIs tend to be very restricted. So most assistants end up limited in what they can achieve. Cheers from Portugal, too.
- indieept 3y ago
  Agreed, these requests tend to be complex (many variables) and APIs are somewhat restricted. But big tech companies should be able to work towards a common API/ecosystem with these players - I would be a user from day one.
  - fragmede 3y ago
    the problem is regular people don't want to program aka use zapier, and just want things done for them, and programmers aren't willing to pay for things. It's different for work, because the company pays for zapier, and people get paid to do work aka use zapier.
  - louison11 3y ago
    That’s already what’s going on when Alexa/Siri offer standardized SDKs for app publishers to plug their API into the assistants. The problem is kind of chicken and egg however where many companies don’t want to bother creating these connections, thus these assistants are pretty useless, and people don’t use them much. This however may just be a matter of time until the incentives align and all of this becomes possible. Eventually, it is my conviction that the web will be conversational first - and typing/clicking will be a secondary mode of action for more advanced queries.
sokoloff 3y ago
I’m struggling to see how ordering an Uber would be better via a text interface to an app not owned by Uber. By that, I mean ordering an Uber on the Uber app is pretty quick, informative, and some of the interaction is needed. Similar for social things. I need to coordinate movie tickets, dinner reservations, or tennis courts not just with the supplier but with my friends. A generalized text interface for consumers would have to be not only as good, but noticeably better in order to create some monetizable advantage. If this arises, I’d expect it to come from an existing body of users (Siri/Alexa) encouraging suppliers (Uber, OpenTable, DoorDash, movie theaters) to build towards a common API. Without that pump priming with tens of thousands of local users already on the platform with money to spend, it’ll be hard to gain traction I think.
- indieept 3y ago
  I believe a text interface can be a lot quicker in many cases where you already know what you want and you don't need to spend time browsing the app (choosing a restaurant, picking what to eat, etc). For social things, this is obviously after you coordinate everything with your friends - it's a lot easier to book the court with one text message than opening an app, finding your club, selecting time, etc. In regards to the last sentence, 100% - I think this will eventually have to come from one of the big tech
  - sokoloff 3y ago
    I don’t know your club’s scheduling pressure, but here, we’re trying to find spots on personal calendars that work with the few openings for courts rather than coordinating our schedules and afterwards informing the club when we’ll take a court.
    - mrmanner 3y ago
      I don’t get the current hype around text (or voice) interfaces. This problem, for example, is much better solved by a calendar interface where everyone including the club can share availability. If I could choose I’d much rather book by sending a calendar than a text message
jsemrau 3y ago
Because scheduling is not a pain point. The pain point of Uber is "It's 12:05 and I quickly need to my appointment". At this point you are willing to pay almost any price for the service. Is there a similar point in digital PAs?
- thisgoesnowhere 3y ago
  Also trust. If this thing screws up now I have 2 problems, Im paying for a thing that doesn't work and I didn't get what I want. So it's more stressful and time consuming than just doing it myself
onion2k 3y ago
This is what getmagic.com started out trying to do. The original getmagic.com was a service you could send an SMS to and their team of people would get it done for you. It was mostly focused on buying things and delivering them to you but I think the idea was that they'd be able to achieve anything, including booking an Uber presumably. It was a bit expensive so they pivoted to serving businesses. I vaguely recall some discussion on HN about it, mostly focused on the 'why' question. There's company in YC23 called Real Magic that I think are having another go.
TruthWillHurt 3y ago
With AI, the solution might be browser automation. Maybe try https://harpa.ai https://harpa.ai . It seems the most advanced / flexible browser extension in a survey I did. Not affiliated in any way.
dalemhurley 3y ago
Zapier is great for repeating the same task over and over again e.g. log new email to sales as a new lead. These requests seem a lot less frequent and more complicated. Context matters a lot. Which makes it very difficult. What tennis court? What club? What if it is booked? What about payment? What about agreeing to waivers? Who would build the integration, the club? The app? What would be the workflow? 1. Join consumer zapier, 2. join tennis club, 3. oAuth them, 4. then type my command?
yellow_lead 3y ago
My opinion is: by the time you've accurately described the task to the AI, and confirmed all the details, you may as well have done the thing yourself.
- maaaaattttt 3y ago
  For one service alone yes, but if well implemented, the real use case would be: “find me a tennis court where I could play tonight after 6pm and before 9pm for one hour. Make sure I have an Uber on time at my door. Report when you’ve ordered the Uber and where the tennis court will be” This would take quite some time to organise manually and such a service would make sense
  - sokoloff 3y ago
    That only has value if you can simplify to “I’d like to play tennis between 6 and 9 tonight; please arrange that.” Having to write/speak the assembly language of each component of that every time in chat is not going to win in the market.
    - axlee 3y ago
      How could the system even guess that you'd be departing from home? That you'll need transport to and from the court? That you want to play for one hour? I don't see these statements as "assembly language", they are questions a human assistant would ask as well.
      - sokoloff 3y ago
        The same way it would know where “my door” is in your example: From previous context, other calendar integrations, and information/settings you’ve given it (or given it access to). “There’s a court available 150 miles away; the Uber will be at your door at 3PM to get you to the court by 6” is a legal but non-sensical solution in your version of the interaction. You have to have some kind of implicit rules and some kind of explicit configuration whether that’s “when I say my door, I mean X” or “when I play tennis, the default duration is an hour and I prefer club Y over town court T1 over town court T2.” And an implicit “people don’t want to spent more time in transit than at an activity, so first constrain search and issue a notice to the user if a solution chosen violates this”.
kristianc 3y ago
> ‘Schedule UberX for today at 3 pm, pickup at my place, drop off at the airport' I guess one reason is that you can literally do this yourself within the Uber app in 30 seconds without the need for an intermediary.
- nerbert 3y ago
  You’d be amazed by the number of things people could do themselves but need an app for.
  - Terretta 3y ago
    That's sort of the point, Uber is an app and they have the app for that. And their phone OS is an app containing apps-for-that. Arguably Android or iOS is already an app with buttons for tasks, and you press the button of the task you want like "get a car" or "bring me food" and it happens. The cool part is each button can bring up a UI designed to work best for that particular task. This is why "everything apps" might not be the next big thing -- they're already here. An everything app is just centralizing and bottlenecking what already exists: Android or iOS as the everything app with buttons for tasks.
- agilob 3y ago
  >Book me a tennis court at @club today at 6 pm for 1 hour' Ok, but can I use my phone to message my tennis court reception to book a session?
  - smallerfish 3y ago
    In certain countries, yeah. Everything is done over whatsapp where I live in latin america. Book a doctor, ask your electrician if he can find a part, schedule a session with the lawyer, ask the pharmacy if they have something in stock, reserve a restaurant, order a meal... As much as I hate Meta there's a ton of value in a universal messaging app. Seems like WeChat is one step further. There's a reason why Musk is attempting to take X in that direction.
  - rco8786 3y ago
notaigenerated 3y ago
Not everything should be a chat interface.
Brajeshwar 3y ago
You might want to check out IFTTT[1], which was pretty popular. Here are a few more automation services to explore - https://docs.inboxstartup.com/build/automation https://docs.inboxstartup.com/build/automation 1. https://ifttt.com https://ifttt.com
- 1letterunixname 3y ago
  It was popular until they slammed on the monetization brakes and everyone left. It's better to start with a workable business model and promise at the beginning rather than scam users with a bait-and-switch.
  - robbiemitchell 3y ago
    It is workable. It’s like $6/month. The real problem is that most people don’t want to pay for things.
    - dzikimarian 3y ago
      I used IFTTT for a while to manage smart sockets. Then I moved to the Home Assistant. Works better and doesn't require monthly payment. Just one time purchase of hardware. It was unnecessarily complex to setup tho, because producers lock down their hardware to their closed cloud. Then they demand monthly payments because they have "ongoing cost", that shouldn't exist in the first place (IFTTT mentioned api fees of various producers as main reason for more aggressive monetization). I'm happy to pay for a product. I'm not going to pay company that constructs closed ecosystem just because it allows to extract monthly fee from me.
      - user_7832 3y ago
        I think HA combined with something like Node Red would actually solve OP's question. Only thing being it's still a more techy thing than a polished "drop-in" thing.
raybb 3y ago
Now that speech to text is quite amazing and LLMs are pretty good at fixing things I've been really wondering when there will be an easy way to go from LLM to my own self hosted apps like creating calendar events (especially in the past which AFAIK none of the assistants can do) or adding things to my nectcloud todolist via caldav or just adding notes to my daily note in obsidian. I feel like someone must be making a easy tool to point llm at your own api but I haven't seen it yet.
- phillipsmg 3y ago
  How do you think the LLM -> API tool should work?
- toomuchtodo 3y ago
  Gorilla? https://gorilla.cs.berkeley.edu/ https://gorilla.cs.berkeley.edu/
antasvara 3y ago
Pricing is the main issue. API's usually aren't free, especially as you grow your business. Most consumers don't have a problem with their main apps; if you can't create this service for free or a super low cost, it's hard to get buy-in. Most apps also want to make it easy for you to do repetitive tasks quickly. Doordash has a "recent orders" section where I can tap to reorder. That's pretty seamless. This general idea holds across most large consumer apps: if something is done repetitively, the app has a way to make that process faster. As such, a personal assistant only becomes useful for edge cases like "ordering from a new restaurant where you already know what you want and know that it's on Doordash." Even in your example of "book a tennis court at 6," you might run into issues. What if the court is booked at 6? I might have secondary times in mind, or I might be willing to settle for a 45 minute session if it's available. All of this is easier to deal with using a calendar interface. And what if different times cost different amounts of money? I wouldn't necessarily know what amount that hour at 6 would cost, and that may matter to me. Poimt being that these apps work well for one thing: tasks that you do repetitively many times, that aren't sped up by the original app itself, and where things won't change. That ends up being a pretty limited usecase, even if you can keep the service cheap.
- Libcat99 3y ago
  The reasons why I want api for my consumer apps have more to do with my own automation desires than with any personal assistant. Our kids use a kid friendly debit card product where we can fund their accounts on demand. I also have a chore schedule system I built where they enter their chores each day. I'd love to be able to automate allowance based on chore completion but there's no way to integrate the two. Every 2 weeks I enter all my accounts, credit cards, etc into a spreadsheet for graphing and tracking. I'd love if there was an api for these services to fetch the data instead of manually logging in.
  - yoav 3y ago
    Just make them fill out a timesheet in google sheets or something. And then their chore “paycheck” is every two weeks. Like simulated work / prep for the real world.
RileyJames 3y ago
There was an app called “workflows” on iOS (I think it was eventually acquired by Apple) that felt very similar to a Zapier, but on my phone for controlling/automating my apps. I only used it a few times and found it too clunky. But I loved the idea. Worth researching what happened to them.
- rrdharan 3y ago
  It still exists as an iOS feature called Shortcuts and plenty of folks use it for exactly the use cases requested in this post.
laurentlassalle 3y ago
Facebook's M comes to mind
1letterunixname 3y ago
Alexa, Siri/Shortcuts, IFTTT, and more already do this. There's nothing original or defensible about it. Sorry. What needs to happen is an AI-improved Alexa that doesn't talk like a damn robot, doesn't only respond to fixed script text patterns, and ties into all of the home automation, cloud services, and IoT things. Voice agents shouldn't just do things by guessing but should seek a bit of follow-up prompting to clarify ambiguous commands rather than picking the wrong one like Alexa does.
jacky2wong 3y ago
I'm building this right now actually! Launching this week - if you're interested in talking further - happy to share it. You can ping me on jacky@floatlabs.io :)
- jacky2wong 3y ago
  Quietly launching now on langvoice.dev :)
gnoy 3y ago
You are looking for Induced AI
- kannishk 3y ago
  They don't do for consumers
kannishk 3y ago
It is in-general very difficult. ->Lack of APIs ->Interface on Phones ->Access to every app which should be explicity given by the user in case of no apis ->A lot of companies have forked chromium and deployed it on cloud to automate the tasks. Basically a bot doing things on a browser in cloud with access to credentials. ->iOS is not open sourced. ->Running a hyper-optimised AOSP is difficult(Andorid) It is good in case, for an app that does everything already(a super-app), they can release a voice assistant which is confined to that app itself.
charlesischuck 3y ago
There's a startup in my coworking space that is doing exactly this. They are based in menlo park, exactly as you describe it they are doing. Not sure how good it is but they're trying. If anyone cares about the name reply and I'll get it tomorrow when I go in.
- indieept 3y ago
  Would be great to know their name! Thanks in advance
  - charlesischuck 3y ago
    https://www.multion.ai/ https://www.multion.ai/ Sorry for late reply
javalavaklava 3y ago
[dead]
jumploops 3y ago
Shameless plug: this is what our end goal is, and we have a tool that does most of this already. Rather than relying on an LLM every time, our tool uses LLMs to help you automate your task (with code) and then lets you call that “Magic Loop” via text, email, or webhook. People are using it for surf notifications, triggering lights during touchdowns, notifying about low Tesla batteries, and more. We have a few “first-class” APIs for scraping, weather, finance, etc. but you can also just add your own API keys/tokens (e.g. for a Slack webhook). You can try it today: https://magicloops.dev https://magicloops.dev
- deleted 3y ago
  [deleted]
haswell 3y ago
Shortcuts in iOS is what currently fills this gap for me, and I think it’s probably the most viable approach for the time being. Building a single production integration connector is difficult and expensive (API licensing/T&C stuff will eat a LOT of your time, APIs across products vary wildly in terms of maturity and standards compliance, dealing with versions/deprecation, and don’t get me started on handling auth). Building a library of them significantly more difficult and expensive. And this is why the products that do well tend to cost a fair amount of money, which consumers typically aren’t going to be interested in spending. Open source tools like n8n are promising, but still clunky and unreliable. Too fiddly for consumers. (Was a PM for an integration product, and have been intimately involved in this space).
solardev 3y ago
Didn't the Google Assistant do this for a while? It could interact with certain popular services (Uber, Doordash, Amazon, etc.). There was a minor splash when that feature launched a few years ago, but haven't heard much about it since then. In my experience of having tried it, it only worked right about 60% of the time, which wasn't worth the hassle of having to correct it all the time. It was easier just to pull up the app and see the immediate visual feedback every step of the way.
lbrunson 3y ago
I really think there should be an app like this, but it also needs to have full context awareness for the rest of your life as well (to-dos, calendar, preferences etc.) in order to accurately make those bookings or orders and to be truly useful. It's possible, just a challenging problem to accomplish at a reasonable price to be a consumer app. Currently in the early stages of building something like that (https://www.flexibly.ai https://www.flexibly.ai)
nonlogical 3y ago
This is something very spiritually close to a concept I recently learned about from Cory Doctorow’s talk at DEFCON called Adversarial Interoperability: https://www.eff.org/deeplinks/2019/10/adversarial-interoperability https://www.eff.org/deeplinks/2019/10/adversarial-interopera... As many have noted the reason this functionality is not available is because it invariably leads to commodification of services. Competitors will quickly jump in and use those APIs to interoperate to bootstrap themselves up. Most likely for consumers that is a huge win, but for companies that is a dangerous filter that filers out companies that stagnate. This used to be common back in the day and was keeping companies from stagnation, that was until per Cory’s arguments there have been an explosion of laws that basically outlawed Adversarial Interoperability. Which in turn led to too big to fail software giants we have today.
- ekanes 3y ago
  > As many have noted the reason this functionality is not available is because it invariably leads to commodification of services. Exactly right, and that's why when AI-based voice assistants become wildly capable someday, some of the more brand-based companies will lose a lot of value. I'll be asking Alexa/Siri/Google Assistant for that ride/groceries/etc and as long as it's good, I won't care where it comes from. The (perceived) value will be with the voice assistant, not the fulfilling service.
- jkestner 3y ago
  Yeah, Doctorow's discussion of that is what got me thinking about the snow job of ecosystems, and specifically how Apple scared/seduced many industries into this: https://blog.supermechanical.com/2023/10/31/the-latest-and-greatest-in-moats/ https://blog.supermechanical.com/2023/10/31/the-latest-and-g... There's probably a subtle business case for interoperability that you can make from within a company, but I only offer that because it's all we have the power to do ourselves, while we're waiting for regulation to change the economic motivations.
scottazro47 3y ago
[dead]
r9295 3y ago
This won't happen as you can then easily build alternative front-ends for applications with ads or exploitative design approaches. Why go through several hoops consisting of dark patterns trying to squeeze extra money or personal information from you when you can just make an API call from a community developed front-end?
vkaku 3y ago
We need an events w/ providers framework. If you can afford to put your providers credentials and build all these integrations, it will work. You need to provide reasonable abstractions between them (Uber/Lyft use the same abstraction) Part of the problem also remains that many providers do not have a straightforward way of giving out keys, and allowing users to log on to a third party app. Businesses won't like it. (See other reply on Adversarial Interoperability) They'll keep breaking these flows often.
axelthegerman 3y ago
Actually most of these companies don't offer any 3P API access - or even better only to a few selected partners. Not one you mentioned but Airbnb is a prime example where folks want to automate things but nothing is accessible to hosts, guests or small scale developers trying to build something. Very frustrating but if you're not happy, use one of the services they partnered with or use something else (oh wait they already got rid of all the competition)
Mrirazak1 3y ago
I think the problem might be for consumer apps how they create their APIs since lots of consumer apps are startups with limited resources and second is pricing. They need to make money somehow?

42 points by indieept 3y ago

dbish 3y ago

This is basically every virtual assistant’s pitch since forever, including Cortana, Siri, and Alexa. They’ve all tried some version of this, with almost no customer usage (most famous 3rd kart friendly one being Alexa skills).

indieept 3y ago
Why do you think it never got traction? I believe the key here should be to be OS-agnostic and to focus on specific tasks that users perform on a recurring/weekly basis and that are simple to automate
- ToJans 3y ago
  My best bet would be to find a tiny, single, super-annoying use case that is instantly fixed with the tool, and grow an audience from there...
- dbish 3y ago
  I don’t think people actually want this even though they talk about it unless it’s basically perfect which the tech still isn’t there for. It’s like the “book a flight” use case that has been used or eons, unless you’re rich (in which case you probably have an assistant), you’re going to double check and second guess anything that is booked for you so you might as well do it yourself. There are very few things that people do weekly at this point that aren’t already somewhat automatable if they cared to.

louison11 3y ago

This is the dream of "ubiquitous" computing that Silicon Valley has been aiming at for two decades. Computing so easy it's almost invisible. You just say what you want to happen and it happens nearly instantly. The biggest problem is that APIs tend to be very restricted. So most assistants end up limited in what they can achieve. Cheers from Portugal, too.

indieept 3y ago
Agreed, these requests tend to be complex (many variables) and APIs are somewhat restricted. But big tech companies should be able to work towards a common API/ecosystem with these players - I would be a user from day one.
- fragmede 3y ago
  the problem is regular people don't want to program aka use zapier, and just want things done for them, and programmers aren't willing to pay for things. It's different for work, because the company pays for zapier, and people get paid to do work aka use zapier.
- louison11 3y ago
  That’s already what’s going on when Alexa/Siri offer standardized SDKs for app publishers to plug their API into the assistants. The problem is kind of chicken and egg however where many companies don’t want to bother creating these connections, thus these assistants are pretty useless, and people don’t use them much. This however may just be a matter of time until the incentives align and all of this becomes possible. Eventually, it is my conviction that the web will be conversational first - and typing/clicking will be a secondary mode of action for more advanced queries.

sokoloff 3y ago

I’m struggling to see how ordering an Uber would be better via a text interface to an app not owned by Uber. By that, I mean ordering an Uber on the Uber app is pretty quick, informative, and some of the interaction is needed. Similar for social things. I need to coordinate movie tickets, dinner reservations, or tennis courts not just with the supplier but with my friends. A generalized text interface for consumers would have to be not only as good, but noticeably better in order to create some monetizable advantage. If this arises, I’d expect it to come from an existing body of users (Siri/Alexa) encouraging suppliers (Uber, OpenTable, DoorDash, movie theaters) to build towards a common API. Without that pump priming with tens of thousands of local users already on the platform with money to spend, it’ll be hard to gain traction I think.

indieept 3y ago
I believe a text interface can be a lot quicker in many cases where you already know what you want and you don't need to spend time browsing the app (choosing a restaurant, picking what to eat, etc). For social things, this is obviously after you coordinate everything with your friends - it's a lot easier to book the court with one text message than opening an app, finding your club, selecting time, etc. In regards to the last sentence, 100% - I think this will eventually have to come from one of the big tech
- sokoloff 3y ago
  I don’t know your club’s scheduling pressure, but here, we’re trying to find spots on personal calendars that work with the few openings for courts rather than coordinating our schedules and afterwards informing the club when we’ll take a court.
  - mrmanner 3y ago
    I don’t get the current hype around text (or voice) interfaces. This problem, for example, is much better solved by a calendar interface where everyone including the club can share availability. If I could choose I’d much rather book by sending a calendar than a text message

jsemrau 3y ago

Because scheduling is not a pain point. The pain point of Uber is "It's 12:05 and I quickly need to my appointment". At this point you are willing to pay almost any price for the service. Is there a similar point in digital PAs?

thisgoesnowhere 3y ago
Also trust. If this thing screws up now I have 2 problems, Im paying for a thing that doesn't work and I didn't get what I want. So it's more stressful and time consuming than just doing it myself

onion2k 3y ago

This is what getmagic.com started out trying to do. The original getmagic.com was a service you could send an SMS to and their team of people would get it done for you. It was mostly focused on buying things and delivering them to you but I think the idea was that they'd be able to achieve anything, including booking an Uber presumably. It was a bit expensive so they pivoted to serving businesses. I vaguely recall some discussion on HN about it, mostly focused on the 'why' question. There's company in YC23 called Real Magic that I think are having another go.

TruthWillHurt 3y ago

With AI, the solution might be browser automation. Maybe try https://harpa.ai https://harpa.ai . It seems the most advanced / flexible browser extension in a survey I did. Not affiliated in any way.

dalemhurley 3y ago

Zapier is great for repeating the same task over and over again e.g. log new email to sales as a new lead. These requests seem a lot less frequent and more complicated. Context matters a lot. Which makes it very difficult. What tennis court? What club? What if it is booked? What about payment? What about agreeing to waivers? Who would build the integration, the club? The app? What would be the workflow? 1. Join consumer zapier, 2. join tennis club, 3. oAuth them, 4. then type my command?

yellow_lead 3y ago

My opinion is: by the time you've accurately described the task to the AI, and confirmed all the details, you may as well have done the thing yourself.

maaaaattttt 3y ago
For one service alone yes, but if well implemented, the real use case would be: “find me a tennis court where I could play tonight after 6pm and before 9pm for one hour. Make sure I have an Uber on time at my door. Report when you’ve ordered the Uber and where the tennis court will be” This would take quite some time to organise manually and such a service would make sense
- sokoloff 3y ago
  That only has value if you can simplify to “I’d like to play tennis between 6 and 9 tonight; please arrange that.” Having to write/speak the assembly language of each component of that every time in chat is not going to win in the market.
  - axlee 3y ago
    How could the system even guess that you'd be departing from home? That you'll need transport to and from the court? That you want to play for one hour? I don't see these statements as "assembly language", they are questions a human assistant would ask as well.
    - sokoloff 3y ago
      The same way it would know where “my door” is in your example: From previous context, other calendar integrations, and information/settings you’ve given it (or given it access to). “There’s a court available 150 miles away; the Uber will be at your door at 3PM to get you to the court by 6” is a legal but non-sensical solution in your version of the interaction. You have to have some kind of implicit rules and some kind of explicit configuration whether that’s “when I say my door, I mean X” or “when I play tennis, the default duration is an hour and I prefer club Y over town court T1 over town court T2.” And an implicit “people don’t want to spent more time in transit than at an activity, so first constrain search and issue a notice to the user if a solution chosen violates this”.

kristianc 3y ago

> ‘Schedule UberX for today at 3 pm, pickup at my place, drop off at the airport' I guess one reason is that you can literally do this yourself within the Uber app in 30 seconds without the need for an intermediary.

nerbert 3y ago
You’d be amazed by the number of things people could do themselves but need an app for.
- Terretta 3y ago
  That's sort of the point, Uber is an app and they have the app for that. And their phone OS is an app containing apps-for-that. Arguably Android or iOS is already an app with buttons for tasks, and you press the button of the task you want like "get a car" or "bring me food" and it happens. The cool part is each button can bring up a UI designed to work best for that particular task. This is why "everything apps" might not be the next big thing -- they're already here. An everything app is just centralizing and bottlenecking what already exists: Android or iOS as the everything app with buttons for tasks.
agilob 3y ago
>Book me a tennis court at @club today at 6 pm for 1 hour' Ok, but can I use my phone to message my tennis court reception to book a session?
- smallerfish 3y ago
  In certain countries, yeah. Everything is done over whatsapp where I live in latin america. Book a doctor, ask your electrician if he can find a part, schedule a session with the lawyer, ask the pharmacy if they have something in stock, reserve a restaurant, order a meal... As much as I hate Meta there's a ton of value in a universal messaging app. Seems like WeChat is one step further. There's a reason why Musk is attempting to take X in that direction.
- rco8786 3y ago

notaigenerated 3y ago

Not everything should be a chat interface.

Brajeshwar 3y ago

You might want to check out IFTTT[1], which was pretty popular. Here are a few more automation services to explore - https://docs.inboxstartup.com/build/automation https://docs.inboxstartup.com/build/automation 1. https://ifttt.com https://ifttt.com

1letterunixname 3y ago
It was popular until they slammed on the monetization brakes and everyone left. It's better to start with a workable business model and promise at the beginning rather than scam users with a bait-and-switch.
- robbiemitchell 3y ago
  It is workable. It’s like $6/month. The real problem is that most people don’t want to pay for things.
  - dzikimarian 3y ago
    I used IFTTT for a while to manage smart sockets. Then I moved to the Home Assistant. Works better and doesn't require monthly payment. Just one time purchase of hardware. It was unnecessarily complex to setup tho, because producers lock down their hardware to their closed cloud. Then they demand monthly payments because they have "ongoing cost", that shouldn't exist in the first place (IFTTT mentioned api fees of various producers as main reason for more aggressive monetization). I'm happy to pay for a product. I'm not going to pay company that constructs closed ecosystem just because it allows to extract monthly fee from me.
    - user_7832 3y ago
      I think HA combined with something like Node Red would actually solve OP's question. Only thing being it's still a more techy thing than a polished "drop-in" thing.

raybb 3y ago

Now that speech to text is quite amazing and LLMs are pretty good at fixing things I've been really wondering when there will be an easy way to go from LLM to my own self hosted apps like creating calendar events (especially in the past which AFAIK none of the assistants can do) or adding things to my nectcloud todolist via caldav or just adding notes to my daily note in obsidian. I feel like someone must be making a easy tool to point llm at your own api but I haven't seen it yet.

phillipsmg 3y ago
How do you think the LLM -> API tool should work?
toomuchtodo 3y ago
Gorilla? https://gorilla.cs.berkeley.edu/ https://gorilla.cs.berkeley.edu/

antasvara 3y ago

Pricing is the main issue. API's usually aren't free, especially as you grow your business. Most consumers don't have a problem with their main apps; if you can't create this service for free or a super low cost, it's hard to get buy-in. Most apps also want to make it easy for you to do repetitive tasks quickly. Doordash has a "recent orders" section where I can tap to reorder. That's pretty seamless. This general idea holds across most large consumer apps: if something is done repetitively, the app has a way to make that process faster. As such, a personal assistant only becomes useful for edge cases like "ordering from a new restaurant where you already know what you want and know that it's on Doordash." Even in your example of "book a tennis court at 6," you might run into issues. What if the court is booked at 6? I might have secondary times in mind, or I might be willing to settle for a 45 minute session if it's available. All of this is easier to deal with using a calendar interface. And what if different times cost different amounts of money? I wouldn't necessarily know what amount that hour at 6 would cost, and that may matter to me. Poimt being that these apps work well for one thing: tasks that you do repetitively many times, that aren't sped up by the original app itself, and where things won't change. That ends up being a pretty limited usecase, even if you can keep the service cheap.

Libcat99 3y ago
The reasons why I want api for my consumer apps have more to do with my own automation desires than with any personal assistant. Our kids use a kid friendly debit card product where we can fund their accounts on demand. I also have a chore schedule system I built where they enter their chores each day. I'd love to be able to automate allowance based on chore completion but there's no way to integrate the two. Every 2 weeks I enter all my accounts, credit cards, etc into a spreadsheet for graphing and tracking. I'd love if there was an api for these services to fetch the data instead of manually logging in.
- yoav 3y ago
  Just make them fill out a timesheet in google sheets or something. And then their chore “paycheck” is every two weeks. Like simulated work / prep for the real world.

RileyJames 3y ago

There was an app called “workflows” on iOS (I think it was eventually acquired by Apple) that felt very similar to a Zapier, but on my phone for controlling/automating my apps. I only used it a few times and found it too clunky. But I loved the idea. Worth researching what happened to them.

rrdharan 3y ago
It still exists as an iOS feature called Shortcuts and plenty of folks use it for exactly the use cases requested in this post.

laurentlassalle 3y ago

Facebook's M comes to mind

1letterunixname 3y ago

Alexa, Siri/Shortcuts, IFTTT, and more already do this. There's nothing original or defensible about it. Sorry. What needs to happen is an AI-improved Alexa that doesn't talk like a damn robot, doesn't only respond to fixed script text patterns, and ties into all of the home automation, cloud services, and IoT things. Voice agents shouldn't just do things by guessing but should seek a bit of follow-up prompting to clarify ambiguous commands rather than picking the wrong one like Alexa does.

jacky2wong 3y ago

I'm building this right now actually! Launching this week - if you're interested in talking further - happy to share it. You can ping me on jacky@floatlabs.io :)

jacky2wong 3y ago
Quietly launching now on langvoice.dev :)

gnoy 3y ago

You are looking for Induced AI

kannishk 3y ago
They don't do for consumers

kannishk 3y ago

It is in-general very difficult. ->Lack of APIs ->Interface on Phones ->Access to every app which should be explicity given by the user in case of no apis ->A lot of companies have forked chromium and deployed it on cloud to automate the tasks. Basically a bot doing things on a browser in cloud with access to credentials. ->iOS is not open sourced. ->Running a hyper-optimised AOSP is difficult(Andorid) It is good in case, for an app that does everything already(a super-app), they can release a voice assistant which is confined to that app itself.

charlesischuck 3y ago

There's a startup in my coworking space that is doing exactly this. They are based in menlo park, exactly as you describe it they are doing. Not sure how good it is but they're trying. If anyone cares about the name reply and I'll get it tomorrow when I go in.

indieept 3y ago
Would be great to know their name! Thanks in advance
- charlesischuck 3y ago
  https://www.multion.ai/ https://www.multion.ai/ Sorry for late reply

javalavaklava 3y ago

[dead]

jumploops 3y ago

Shameless plug: this is what our end goal is, and we have a tool that does most of this already. Rather than relying on an LLM every time, our tool uses LLMs to help you automate your task (with code) and then lets you call that “Magic Loop” via text, email, or webhook. People are using it for surf notifications, triggering lights during touchdowns, notifying about low Tesla batteries, and more. We have a few “first-class” APIs for scraping, weather, finance, etc. but you can also just add your own API keys/tokens (e.g. for a Slack webhook). You can try it today: https://magicloops.dev https://magicloops.dev

deleted 3y ago
[deleted]

haswell 3y ago

Shortcuts in iOS is what currently fills this gap for me, and I think it’s probably the most viable approach for the time being. Building a single production integration connector is difficult and expensive (API licensing/T&C stuff will eat a LOT of your time, APIs across products vary wildly in terms of maturity and standards compliance, dealing with versions/deprecation, and don’t get me started on handling auth). Building a library of them significantly more difficult and expensive. And this is why the products that do well tend to cost a fair amount of money, which consumers typically aren’t going to be interested in spending. Open source tools like n8n are promising, but still clunky and unreliable. Too fiddly for consumers. (Was a PM for an integration product, and have been intimately involved in this space).

solardev 3y ago

Didn't the Google Assistant do this for a while? It could interact with certain popular services (Uber, Doordash, Amazon, etc.). There was a minor splash when that feature launched a few years ago, but haven't heard much about it since then. In my experience of having tried it, it only worked right about 60% of the time, which wasn't worth the hassle of having to correct it all the time. It was easier just to pull up the app and see the immediate visual feedback every step of the way.

lbrunson 3y ago

I really think there should be an app like this, but it also needs to have full context awareness for the rest of your life as well (to-dos, calendar, preferences etc.) in order to accurately make those bookings or orders and to be truly useful. It's possible, just a challenging problem to accomplish at a reasonable price to be a consumer app. Currently in the early stages of building something like that (https://www.flexibly.ai https://www.flexibly.ai)

nonlogical 3y ago

This is something very spiritually close to a concept I recently learned about from Cory Doctorow’s talk at DEFCON called Adversarial Interoperability: https://www.eff.org/deeplinks/2019/10/adversarial-interoperability https://www.eff.org/deeplinks/2019/10/adversarial-interopera... As many have noted the reason this functionality is not available is because it invariably leads to commodification of services. Competitors will quickly jump in and use those APIs to interoperate to bootstrap themselves up. Most likely for consumers that is a huge win, but for companies that is a dangerous filter that filers out companies that stagnate. This used to be common back in the day and was keeping companies from stagnation, that was until per Cory’s arguments there have been an explosion of laws that basically outlawed Adversarial Interoperability. Which in turn led to too big to fail software giants we have today.

ekanes 3y ago
> As many have noted the reason this functionality is not available is because it invariably leads to commodification of services. Exactly right, and that's why when AI-based voice assistants become wildly capable someday, some of the more brand-based companies will lose a lot of value. I'll be asking Alexa/Siri/Google Assistant for that ride/groceries/etc and as long as it's good, I won't care where it comes from. The (perceived) value will be with the voice assistant, not the fulfilling service.
jkestner 3y ago
Yeah, Doctorow's discussion of that is what got me thinking about the snow job of ecosystems, and specifically how Apple scared/seduced many industries into this: https://blog.supermechanical.com/2023/10/31/the-latest-and-greatest-in-moats/ https://blog.supermechanical.com/2023/10/31/the-latest-and-g... There's probably a subtle business case for interoperability that you can make from within a company, but I only offer that because it's all we have the power to do ourselves, while we're waiting for regulation to change the economic motivations.

scottazro47 3y ago

[dead]

r9295 3y ago

This won't happen as you can then easily build alternative front-ends for applications with ads or exploitative design approaches. Why go through several hoops consisting of dark patterns trying to squeeze extra money or personal information from you when you can just make an API call from a community developed front-end?

vkaku 3y ago

We need an events w/ providers framework. If you can afford to put your providers credentials and build all these integrations, it will work. You need to provide reasonable abstractions between them (Uber/Lyft use the same abstraction) Part of the problem also remains that many providers do not have a straightforward way of giving out keys, and allowing users to log on to a third party app. Businesses won't like it. (See other reply on Adversarial Interoperability) They'll keep breaking these flows often.

axelthegerman 3y ago

Actually most of these companies don't offer any 3P API access - or even better only to a few selected partners. Not one you mentioned but Airbnb is a prime example where folks want to automate things but nothing is accessible to hosts, guests or small scale developers trying to build something. Very frustrating but if you're not happy, use one of the services they partnered with or use something else (oh wait they already got rid of all the competition)

Mrirazak1 3y ago

I think the problem might be for consumer apps how they create their APIs since lots of consumer apps are startups with limited resources and second is pricing. They need to make money somehow?