5 ms·

What is fascinating about this announcement is if you look into future after considerable improvements in product and the model, we will be just chatting with C

What is fascinating about this announcement is if you look into future after considerable improvements in product and the model, we will be just chatting with ChatGPT to book dinner tables, flights, buy groceries and do all sort of mundane and hugely boring things we do on the web, just by talking to the agents. I'd definitely love that.

TeMPOraL 2y ago
I don't. Chat interface sucks; for most of these things, a more direct interface could be much more ergonomic, and easier to operate and integrate. The only reason we don't have those interfaces is because neither restaurants, nor airlines, nor online stores, nor any other businesses actually want us to have them. To a business, the user interface isn't there to help the user achieve their goals - it's a platform for milking the users as much as possible. To a lesser or greater extent, almost every site actively defeats attempts at interoperability. Denying interoperability is so culturally ingrained at this point, that it got pretty much baked into entire web stack. The only force currently countering this is accessibility - screen readers are pretty much an interoperability backdoor with legal backing in some situations, so not every company gets to ignore it. No, we'll have to settle for "chat agents" powered by multimodal LLMs working as general-purpose web scrappers, because those models are the ultimate form of adversarial interoperability, and chat agents are the cheapest, least-effort way to let users operate them.
- gordon_freeman 2y ago
  I also do not like Chat interface. What I meant by above comment was actually talking and having natural conversations with Operator agent while driving car or just going for a walk or whenever and wherever something comes to my mind which requires me to go to browser and fill out forms etc. That would get us closer to using chatGPT as a universal AI agent to get those things done. (This is what Siri was supposed to be one day when Steve Jobs introduced it on that stage but unfortunately that day never arrived.)
  - TeMPOraL 2y ago
    > This is what Siri was supposed to be one day when Steve Jobs introduced it on that stage but unfortunately that day never arrived. The irony is, the reason neither Siri nor Alexa nor Google Assistant/Now/${whatever they call it these days} nor Cortana achieved this isn't the voice side of the equation. That one sucks too, when you realize that 20 years ago Microsoft Speech API could do better, fully locally, on cheap consumer hardware, but the real problem is the integration approach. Doing interop by agreements between vendors only ever led to commercial entities exposing minimal, trivial functionality of their services, which were activated by voice commands in the form of "{Brand Wake word}, {verb} {Brand 1} to {verb} {Brand 2}" etc. This is not an ergonomic user interface, it's merely making people constantly read ads themselves. "Okay Google, play some Taylor Swift on Spotify" is literally three brand ads in eight words you just spoke out loud. No, all the magical voice experience you describe is enabled[0] by having multimodal LLMs that can be sicced on any website and beat it into submission, whether the website vendor likes it or not. Hopefully they won't screw it up (again[1]) trying to commercialize it by offering third parties control over what LLMs can do. If, in this new reality, I have to utter the word "Spotify" to have my phone start playing music, this is going to be a double regression relative to MS Speech API in the mid 2000s. -- [0] - Actually, it was possible ever since OpenAI added function calling, which was like over a good year ago - if you exposed stuff you care about as functions on your own. As it is, currently the smartphone voice assistant that's closest to Star Trek experience is actually free and easy to set up - it's Home Assistant with its mobile app (for the phone assistant side) and server-side integrations (mostly, but not limited to, IoT hardware). [1] - Like OpenAI did with "GPTs". They've tried to package a system prompt and function call configuration into a digital product and build a marketplace around it. This delayed their release of the functionality to the official ChatGPT app/website for about half a year, leading to an absurd situation where, for those 6+ months, anyone with API access could use a much better implementation of "GPTs" via third-party frontends like TypingMind.
- sky2224 2y ago
  I think the chat interface is bad, but for certain things it could honestly streamline a lot of mundane things as the poster you're replying two stated. For example, McDonald's has heavily shifted away from cashiers taking orders and instead is using the kiosks to have customers order. The downside of this is 1) it's incredibly unsanitary and 2) customers are so goddamn slow at tapping on that god awful screen. An AI agent could actually take orders with surprisingly good accuracy. Now, whether we want that in the world is a whole different debate.
  - krapp 2y ago
    McDonald's already tried having AI take orders and stopped when the AI did things like randomly add $250 of McNuggets or mistake ketchup for butter. Note - because this is something which needs to be pointed out in any discussion of AI now - even though human beings also make mistakes this is still markedly less accurate than the average human employee.
    - ItsMattyG 2y ago
      For now
      - TeMPOraL 2y ago
        Indeed. I think a GPT-4o class model, properly prompted, would work just fine today. The trick is, unlike a human, the computer is free to just say "no" without consequences. The model could be aggressively prompted to detect and refuse weird orders. Having to escalate to a human supervisor (who conveniently is always busy doing other things and will come to you in a minute or three) should be sufficient at discouraging pranksters and fraudsters, while not annoying enough to deter normal customer. (I say model, but for this problem I'd consider a pipeline where the powerful model is just parsing orders and formulating replies, while being sanity-checked by a cheaper model and some old-school logic to detect excessive amounts or unusual combinations. I'd also consider using "open source" model in place of GPT-4o, as open models allow doing "alignment" shenanigans in the latent space, instead of just in the prompts.)
  - 2y ago
- csharpminor 2y ago
  Why assume that chat will be the interface? Multimodal and dynamically generated seems more likely.
  - TeMPOraL 2y ago
    Voice chat with LLMs is a complete interface, and it's one that already works and can be slotted right into the product. You can prototype voice chat-based ordering app via no-code tools today, and without much effort going into it. Dynamically generated interactive UIs are something people are barely beginning to experiment with; we don't know if current models can do them reliably for realistic problems, and how effort has to go into setting them up for any particular product. At this point, they're an expensive, conceptual solution, that doesn't scale.
- thatjoeoverthr 2y ago
  Yes. Chat is absolutely bad, because it is opaque. It perfectly reproduces what used to be called "hunt the verb" in gaming, for the same reason. The simple truth is you're interacting with a piece of software, with features and subroutines. GUIs are great at surfacing features, affordances, changing with context. A chat interface invites you to guess.
CaptainFever 2y ago
I would really love for Apple Knowledge Navigator to be real: https://www.youtube.com/watch?v=umJsITGzXd0 https://www.youtube.com/watch?v=umJsITGzXd0 and I'm surprised that people don't bring this visualisation up more often.
- freediver 2y ago
  This is something we'd like to build. It requires owning both hardware and software - you can not build this in the world of platforms with permissionless apps.
windowlessmonad 2y ago
Are our attention spans so shot that we consider booking a reservation at a restaurant or buying groceries "hugely boring"? And do we value convenience so much that we're willing to sacrifice a huge breadth of options for whatever sponsor du jour OpenAI wants to serve us just to save less than 10 minutes? And would this company spend billions of dollars for this infinitesimally small increase in convenience? No, of course not; you are not the real customer here. Consider reading between the lines and thinking about what you are sacrificing just for the sake of minor convenience.
- snakeyjake 2y ago
  The potential of x-Models (x=ll, transformer, tts, etc), which are not AI, to perfect the flooding of social media with bullshit to increase the sales of drop-shipped garbage to hundreds of millions of people is so great that there is a near-infinite stream of money available to be spent on useless shit like this. Talking to an x-Model (still not AI), just like talking to a human, has never been, is not now, and will never be faster than looking at an information-dense table of data. x-Models (will never be AI) will eat the world though, long after the dream of talking to a computer to reserve a table has died, because they are so good at flooding social media with bullshit to facilitate the sales of drop-shipped garbage to hundreds of millions of people. That being said, it is highly likely that is an extremely large group of people who are so braindead that they need a robot to click through TripAdvisor links for them to create a boring, sterile, assembly-line one-day tour of Rome. Whether or not those people have enough money to be extracted from them to make running such a service profitable remains to be seen.
- dougb5 2y ago
  I'm reminded of Kurt Vonnegut's famous story about buying postage stamps: https://www.insidehook.com/wellness/kurt-vonnegut-advice https://www.insidehook.com/wellness/kurt-vonnegut-advice "I stamp the envelope and mail it in a mailbox in front of the post office, and I go home. And I’ve had a hell of a good time. And I tell you, we are here on Earth to fart around, and don’t let anybody tell you any different...How beautiful it is to get up and go do something."
  - 2y ago
n144q 2y ago
Not until ChatGPT can do these things as reliably as concierge service, and provide full refund for any situation it messes up. I am not looking forward to a trip booked for wrong dates with the hotel name confused/hallucinated for a different one.
- drewbeck 2y ago
  Yeah the failure states are really an issue. Happy path looks magical but there are so many ways that it can go wrong. And you don’t have the fallback of an actual human you’re talking to to clear it up.
melvinmelih 2y ago
After many years of dealing with chat bots, I think we can all agree that we don't want chat-based interfaces to order our pizza (clicking buttons and scrolling through lists of options is way way faster). I can't think of many other things I'd like to accomplish by chat that I wouldn't want to do through a website or an app. My eyes bleed watching the AI crawl tediously slow to place a pizza order for me. But… what if I told you that AI could generate an context-specific user interface on the fly to accomplish the specific task at hand. This way we don't have to deal with the random (and often hostile) user interfaces from random websites but still enjoy the convenience of it. I think this will be the future.
tmvphil 2y ago
Reserving dinner and booking flights is like .01% of my time. Really just negligible, and they are easy enough. Groceries are more time, but I don't really want groceries delivered, I enjoy going to the store and seeing what is available and what looks good. Maybe it could read HN for me and tell me if there is anything I'd find interesting. But then how would I slack off?
whartung 2y ago
At the moment, I'm looking forward to it. Let the bot deal with the ads, the cookie banners, the upsells, "newsletters" and all of the other web BS we deal with. The bot clicks through the front door of the website, just like us. No APIs, no keys, no nothing. "Hey Siri, grab me a bottle of slow release 500mg Vitamin C from either Amazon or Walmart, whichever has the best deal. Kthx"
adamanonymous 2y ago
The internet optimized away things like concierge services and travel agents by giving us the power to book reservations and plan trips on our own, without dealing with a cumbersome and expensive middleman. Now with the power of AI we have added back in that middle man to countless more services!
lm28469 2y ago
I just booked a restaurant table, it took me maybe 10s on opentable. Booking flights are well under a minute now. Grocery shopping is a 15m stop on my daily walk around the block. If these are your pain points in life, and they're worth spending $500b to solve, you must live in an insane bubble.

TeMPOraL 2y ago

I don't. Chat interface sucks; for most of these things, a more direct interface could be much more ergonomic, and easier to operate and integrate. The only reason we don't have those interfaces is because neither restaurants, nor airlines, nor online stores, nor any other businesses actually want us to have them. To a business, the user interface isn't there to help the user achieve their goals - it's a platform for milking the users as much as possible. To a lesser or greater extent, almost every site actively defeats attempts at interoperability. Denying interoperability is so culturally ingrained at this point, that it got pretty much baked into entire web stack. The only force currently countering this is accessibility - screen readers are pretty much an interoperability backdoor with legal backing in some situations, so not every company gets to ignore it. No, we'll have to settle for "chat agents" powered by multimodal LLMs working as general-purpose web scrappers, because those models are the ultimate form of adversarial interoperability, and chat agents are the cheapest, least-effort way to let users operate them.

gordon_freeman 2y ago
I also do not like Chat interface. What I meant by above comment was actually talking and having natural conversations with Operator agent while driving car or just going for a walk or whenever and wherever something comes to my mind which requires me to go to browser and fill out forms etc. That would get us closer to using chatGPT as a universal AI agent to get those things done. (This is what Siri was supposed to be one day when Steve Jobs introduced it on that stage but unfortunately that day never arrived.)
- TeMPOraL 2y ago
  > This is what Siri was supposed to be one day when Steve Jobs introduced it on that stage but unfortunately that day never arrived. The irony is, the reason neither Siri nor Alexa nor Google Assistant/Now/${whatever they call it these days} nor Cortana achieved this isn't the voice side of the equation. That one sucks too, when you realize that 20 years ago Microsoft Speech API could do better, fully locally, on cheap consumer hardware, but the real problem is the integration approach. Doing interop by agreements between vendors only ever led to commercial entities exposing minimal, trivial functionality of their services, which were activated by voice commands in the form of "{Brand Wake word}, {verb} {Brand 1} to {verb} {Brand 2}" etc. This is not an ergonomic user interface, it's merely making people constantly read ads themselves. "Okay Google, play some Taylor Swift on Spotify" is literally three brand ads in eight words you just spoke out loud. No, all the magical voice experience you describe is enabled[0] by having multimodal LLMs that can be sicced on any website and beat it into submission, whether the website vendor likes it or not. Hopefully they won't screw it up (again[1]) trying to commercialize it by offering third parties control over what LLMs can do. If, in this new reality, I have to utter the word "Spotify" to have my phone start playing music, this is going to be a double regression relative to MS Speech API in the mid 2000s. -- [0] - Actually, it was possible ever since OpenAI added function calling, which was like over a good year ago - if you exposed stuff you care about as functions on your own. As it is, currently the smartphone voice assistant that's closest to Star Trek experience is actually free and easy to set up - it's Home Assistant with its mobile app (for the phone assistant side) and server-side integrations (mostly, but not limited to, IoT hardware). [1] - Like OpenAI did with "GPTs". They've tried to package a system prompt and function call configuration into a digital product and build a marketplace around it. This delayed their release of the functionality to the official ChatGPT app/website for about half a year, leading to an absurd situation where, for those 6+ months, anyone with API access could use a much better implementation of "GPTs" via third-party frontends like TypingMind.
sky2224 2y ago
I think the chat interface is bad, but for certain things it could honestly streamline a lot of mundane things as the poster you're replying two stated. For example, McDonald's has heavily shifted away from cashiers taking orders and instead is using the kiosks to have customers order. The downside of this is 1) it's incredibly unsanitary and 2) customers are so goddamn slow at tapping on that god awful screen. An AI agent could actually take orders with surprisingly good accuracy. Now, whether we want that in the world is a whole different debate.
- krapp 2y ago
  McDonald's already tried having AI take orders and stopped when the AI did things like randomly add $250 of McNuggets or mistake ketchup for butter. Note - because this is something which needs to be pointed out in any discussion of AI now - even though human beings also make mistakes this is still markedly less accurate than the average human employee.
  - ItsMattyG 2y ago
    For now
    - TeMPOraL 2y ago
      Indeed. I think a GPT-4o class model, properly prompted, would work just fine today. The trick is, unlike a human, the computer is free to just say "no" without consequences. The model could be aggressively prompted to detect and refuse weird orders. Having to escalate to a human supervisor (who conveniently is always busy doing other things and will come to you in a minute or three) should be sufficient at discouraging pranksters and fraudsters, while not annoying enough to deter normal customer. (I say model, but for this problem I'd consider a pipeline where the powerful model is just parsing orders and formulating replies, while being sanity-checked by a cheaper model and some old-school logic to detect excessive amounts or unusual combinations. I'd also consider using "open source" model in place of GPT-4o, as open models allow doing "alignment" shenanigans in the latent space, instead of just in the prompts.)
- 2y ago
csharpminor 2y ago
Why assume that chat will be the interface? Multimodal and dynamically generated seems more likely.
- TeMPOraL 2y ago
  Voice chat with LLMs is a complete interface, and it's one that already works and can be slotted right into the product. You can prototype voice chat-based ordering app via no-code tools today, and without much effort going into it. Dynamically generated interactive UIs are something people are barely beginning to experiment with; we don't know if current models can do them reliably for realistic problems, and how effort has to go into setting them up for any particular product. At this point, they're an expensive, conceptual solution, that doesn't scale.
thatjoeoverthr 2y ago
Yes. Chat is absolutely bad, because it is opaque. It perfectly reproduces what used to be called "hunt the verb" in gaming, for the same reason. The simple truth is you're interacting with a piece of software, with features and subroutines. GUIs are great at surfacing features, affordances, changing with context. A chat interface invites you to guess.

CaptainFever 2y ago

I would really love for Apple Knowledge Navigator to be real: https://www.youtube.com/watch?v=umJsITGzXd0 https://www.youtube.com/watch?v=umJsITGzXd0 and I'm surprised that people don't bring this visualisation up more often.

freediver 2y ago
This is something we'd like to build. It requires owning both hardware and software - you can not build this in the world of platforms with permissionless apps.

windowlessmonad 2y ago

Are our attention spans so shot that we consider booking a reservation at a restaurant or buying groceries "hugely boring"? And do we value convenience so much that we're willing to sacrifice a huge breadth of options for whatever sponsor du jour OpenAI wants to serve us just to save less than 10 minutes? And would this company spend billions of dollars for this infinitesimally small increase in convenience? No, of course not; you are not the real customer here. Consider reading between the lines and thinking about what you are sacrificing just for the sake of minor convenience.

snakeyjake 2y ago
The potential of x-Models (x=ll, transformer, tts, etc), which are not AI, to perfect the flooding of social media with bullshit to increase the sales of drop-shipped garbage to hundreds of millions of people is so great that there is a near-infinite stream of money available to be spent on useless shit like this. Talking to an x-Model (still not AI), just like talking to a human, has never been, is not now, and will never be faster than looking at an information-dense table of data. x-Models (will never be AI) will eat the world though, long after the dream of talking to a computer to reserve a table has died, because they are so good at flooding social media with bullshit to facilitate the sales of drop-shipped garbage to hundreds of millions of people. That being said, it is highly likely that is an extremely large group of people who are so braindead that they need a robot to click through TripAdvisor links for them to create a boring, sterile, assembly-line one-day tour of Rome. Whether or not those people have enough money to be extracted from them to make running such a service profitable remains to be seen.
dougb5 2y ago
I'm reminded of Kurt Vonnegut's famous story about buying postage stamps: https://www.insidehook.com/wellness/kurt-vonnegut-advice https://www.insidehook.com/wellness/kurt-vonnegut-advice "I stamp the envelope and mail it in a mailbox in front of the post office, and I go home. And I’ve had a hell of a good time. And I tell you, we are here on Earth to fart around, and don’t let anybody tell you any different...How beautiful it is to get up and go do something."
- 2y ago

n144q 2y ago

Not until ChatGPT can do these things as reliably as concierge service, and provide full refund for any situation it messes up. I am not looking forward to a trip booked for wrong dates with the hotel name confused/hallucinated for a different one.

drewbeck 2y ago
Yeah the failure states are really an issue. Happy path looks magical but there are so many ways that it can go wrong. And you don’t have the fallback of an actual human you’re talking to to clear it up.

melvinmelih 2y ago

After many years of dealing with chat bots, I think we can all agree that we don't want chat-based interfaces to order our pizza (clicking buttons and scrolling through lists of options is way way faster). I can't think of many other things I'd like to accomplish by chat that I wouldn't want to do through a website or an app. My eyes bleed watching the AI crawl tediously slow to place a pizza order for me. But… what if I told you that AI could generate an context-specific user interface on the fly to accomplish the specific task at hand. This way we don't have to deal with the random (and often hostile) user interfaces from random websites but still enjoy the convenience of it. I think this will be the future.

tmvphil 2y ago

Reserving dinner and booking flights is like .01% of my time. Really just negligible, and they are easy enough. Groceries are more time, but I don't really want groceries delivered, I enjoy going to the store and seeing what is available and what looks good. Maybe it could read HN for me and tell me if there is anything I'd find interesting. But then how would I slack off?

whartung 2y ago

At the moment, I'm looking forward to it. Let the bot deal with the ads, the cookie banners, the upsells, "newsletters" and all of the other web BS we deal with. The bot clicks through the front door of the website, just like us. No APIs, no keys, no nothing. "Hey Siri, grab me a bottle of slow release 500mg Vitamin C from either Amazon or Walmart, whichever has the best deal. Kthx"

adamanonymous 2y ago

The internet optimized away things like concierge services and travel agents by giving us the power to book reservations and plan trips on our own, without dealing with a cumbersome and expensive middleman. Now with the power of AI we have added back in that middle man to countless more services!

lm28469 2y ago

I just booked a restaurant table, it took me maybe 10s on opentable. Booking flights are well under a minute now. Grocery shopping is a 15m stop on my daily walk around the block. If these are your pain points in life, and they're worth spending $500b to solve, you must live in an insane bubble.