3 ms·

I believe we are almost at peak GUI. The endgame here is that all these crappy GUIs that are getting worse every year will be relegated to a role of being APIs

I believe we are almost at peak GUI. The endgame here is that all these crappy GUIs that are getting worse every year will be relegated to a role of being APIs for AI agents. Instead of clicking around and filling out forms and waiting for loading spinners all the time, we'll just tell a large language model what we want to do in English, and it will go off and screen-scrape a bunch of apps and websites, do all the clicking for us, and summarize the results in a much simpler UI designed to actually be fast and useful, vs. designed to optimize the business metrics of some company as interpreted by a gaggle of product managers. This isn't unprecedented. Plaid screen scrapes terrible bank websites and turns them into APIs, though without AI. Google Duplex uses AI to turn restaurant phone numbers into an API for making reservations. DeepMind's Sparrow[1], just announced today, answers factual questions posed in plain English by performing Google searches and summarizing the results. But it's going to be a revolution when it becomes much more general and able to take actions rather than just summarize information. It isn't far off! https://adept.ai https://adept.ai is pretty much exactly what I'm talking about, and I expect there are a lot more people working on similar things that are still in stealth mode. [1] https://www.deepmind.com/blog/building-safer-dialogue-agents https://www.deepmind.com/blog/building-safer-dialogue-agents

bretbernhoft 4y ago
Your point about more people working on similar projects in stealth mode is likely an underappreciated notion. And thank you for introducing me to Adept; looking forward to keeping my eyes on that company's progress.
deleted 4y ago
[deleted]
systemvoltage 4y ago
> we'll just tell a large language model what we want to do in English Language is an imprecise tool, it has inherent ambiguity. Language is also laborious and one dimensional (stream of bits over temporal dimension). A tool such as the one you describe would be extremely frustrating to use.
- modeless 4y ago
  If you imagine asking Siri or Google Assistant to do things for you, of course that would be incredibly frustrating; they suck. That style of HCI is a dead end for sure. However, asking a human personal assistant to do things for you wouldn't be so frustrating. Especially if they know you well. That is the kind of experience I'm talking about. Yes, I do believe it is possible in the not too distant future. Maybe 10 years or so. It won't be as good as a human assistant at first, but it will not be like the speech interfaces you're used to.
throwaway0asd 4y ago
I have a web app that is essentially a full OS GUI in the browser. The load time in Chrome with state restoration is almost exactly 0.3 seconds, which includes asynchronous gathering of file system artifacts and rendering display thereof. At the same many primarily text sites struggle, such as news and social media, struggle to load in 15 seconds. My connection measures about 920mbps down so 15 seconds is really ridiculously slow. This is certainly not a technology problem evidenced by my own app that is doing so much more is such shorter time.
- modeless 4y ago
  You are right. It is not a technology problem. It is a problem of incentives. The people writing the software don't have the right incentives to make it pleasant to use.
  - mod 4y ago
    Some of them do, and then we end up with social media that doesn't take 15 seconds to load. This site, for instance.
    - renonce 4y ago
      It's not the incentives that make this site load in 15 seconds. The site just doesn't have the incentives found in the majority of sites to make it any more complex than it has to be.
nathias 4y ago
how naive, we have long since moved from peak GUIs now the game is infesting them and rearanging them around the need to serve adds
jl6 4y ago
Using a LLM as an agent will be comical, like a supervillain trying to explain a menial chore to a dimwitted henchman: “No, you fool, that’s not what I meant when I said bring me the head of R&D!” Then, maybe LLMs will get more intelligent and it will be less comical, and more like having an actual slave. An agent intelligent enough to do all these things is probably intelligent enough make me queasy at the thought of being its master.
makeitdouble 4y ago
You mention it in your response, but this whole paradigm looks like an extension of what we have with Google Search. And Google Search is so utterly weak. Not just because of the neutering of search options or the weird priority conflict inside Google, but just because even with litteraly all the data in the world about a specific user it doesn't seems like it can wrangle what a request actually means. It can tell me the time in Chicago, but not what computer would actually be the best for my work. That search will only be spam, irrelevant popular results and paid reviews. Same if I asked for a _good_ pizza recipe, it would probably not understand what that actually means for me. The whole model of "throwing a request in the box and expecting a result" seems broken to me, I mean even between humans it doesn't work that way, why would it work with an advanced AI ? PS: even with more back and forth, I'm imagining what we have now with customer support over chat, and while more efficient than by phone, it's definitely not the interface I want by default
civilized 4y ago
They won't go down without a fight. They want real human eyeballs. But I wonder if there will come a point where captchas and "Not a Robot" checkboxes no longer work.
- modeless 4y ago
  If AI can't solve the captchas, we can just hire people to do it. Services for that already exist and they are fairly cheap.
fuzzyset 4y ago
Somewhat related, but I recently discovered the Calibre "Fetch News" function. It parses the articles (similarly to archive.org I assume in that it also bypasses paywalls) and puts them into a nice EPUB format. I run it in server mode on a RPi so every morning I can just load up the Calibre web reader on my phone and read the news in pure text form.

bretbernhoft 4y ago

Your point about more people working on similar projects in stealth mode is likely an underappreciated notion. And thank you for introducing me to Adept; looking forward to keeping my eyes on that company's progress.

deleted 4y ago

[deleted]

systemvoltage 4y ago

> we'll just tell a large language model what we want to do in English Language is an imprecise tool, it has inherent ambiguity. Language is also laborious and one dimensional (stream of bits over temporal dimension). A tool such as the one you describe would be extremely frustrating to use.

modeless 4y ago
If you imagine asking Siri or Google Assistant to do things for you, of course that would be incredibly frustrating; they suck. That style of HCI is a dead end for sure. However, asking a human personal assistant to do things for you wouldn't be so frustrating. Especially if they know you well. That is the kind of experience I'm talking about. Yes, I do believe it is possible in the not too distant future. Maybe 10 years or so. It won't be as good as a human assistant at first, but it will not be like the speech interfaces you're used to.

throwaway0asd 4y ago

I have a web app that is essentially a full OS GUI in the browser. The load time in Chrome with state restoration is almost exactly 0.3 seconds, which includes asynchronous gathering of file system artifacts and rendering display thereof. At the same many primarily text sites struggle, such as news and social media, struggle to load in 15 seconds. My connection measures about 920mbps down so 15 seconds is really ridiculously slow. This is certainly not a technology problem evidenced by my own app that is doing so much more is such shorter time.

modeless 4y ago
You are right. It is not a technology problem. It is a problem of incentives. The people writing the software don't have the right incentives to make it pleasant to use.
- mod 4y ago
  Some of them do, and then we end up with social media that doesn't take 15 seconds to load. This site, for instance.
  - renonce 4y ago
    It's not the incentives that make this site load in 15 seconds. The site just doesn't have the incentives found in the majority of sites to make it any more complex than it has to be.

nathias 4y ago

how naive, we have long since moved from peak GUIs now the game is infesting them and rearanging them around the need to serve adds

jl6 4y ago

Using a LLM as an agent will be comical, like a supervillain trying to explain a menial chore to a dimwitted henchman: “No, you fool, that’s not what I meant when I said bring me the head of R&D!” Then, maybe LLMs will get more intelligent and it will be less comical, and more like having an actual slave. An agent intelligent enough to do all these things is probably intelligent enough make me queasy at the thought of being its master.

makeitdouble 4y ago

You mention it in your response, but this whole paradigm looks like an extension of what we have with Google Search. And Google Search is so utterly weak. Not just because of the neutering of search options or the weird priority conflict inside Google, but just because even with litteraly all the data in the world about a specific user it doesn't seems like it can wrangle what a request actually means. It can tell me the time in Chicago, but not what computer would actually be the best for my work. That search will only be spam, irrelevant popular results and paid reviews. Same if I asked for a _good_ pizza recipe, it would probably not understand what that actually means for me. The whole model of "throwing a request in the box and expecting a result" seems broken to me, I mean even between humans it doesn't work that way, why would it work with an advanced AI ? PS: even with more back and forth, I'm imagining what we have now with customer support over chat, and while more efficient than by phone, it's definitely not the interface I want by default

civilized 4y ago

They won't go down without a fight. They want real human eyeballs. But I wonder if there will come a point where captchas and "Not a Robot" checkboxes no longer work.

modeless 4y ago
If AI can't solve the captchas, we can just hire people to do it. Services for that already exist and they are fairly cheap.

fuzzyset 4y ago

Somewhat related, but I recently discovered the Calibre "Fetch News" function. It parses the articles (similarly to archive.org I assume in that it also bypasses paywalls) and puts them into a nice EPUB format. I run it in server mode on a RPi so every morning I can just load up the Calibre web reader on my phone and read the news in pure text form.