4 ms·

We focused so much on how that we didn’t stop to ask why. We already have an interface for agents, we call that an API. Why do we need to have AI click buttons

We focused so much on how that we didn’t stop to ask why. We already have an interface for agents, we call that an API. Why do we need to have AI click buttons, that eventually call an API. Just skip the middle layer and go straight to an API that does everything that system is capable of doing.

deleted 2y ago
[deleted]
solidasparagus 2y ago
People know this. But most actually useful software built for humans doesn't have a good programmatic API. So computer use is the only way to connect AI agents to that software. And that's where a lot of the true (medium-term) financial gain will be - automating tasks on tools that are poorly built or just old but are still valuable.
bberenberg 2y ago
I think everyone agrees with you. The issue is who is building API first interfaces? The value an agent provides here is that it moves control from the publisher to the consumer (in the technical sense). “Twitter doesn’t want to let me access via an API? That’s fine, my browser/agent will automatically automate what I want it to do for me in the UI.” Is a major shift in how we interface with other services.
- rixed 2y ago
  Come especially handy to work around the increasingly user hostile captchas, ironically. I expect this arm race to spiral down quickly into some form of mandatory biometric check, the dream come true of our security obsessed insecure rulers.
  - marxisttemp 2y ago
    Altman already spiraled down there: https://en.m.wikipedia.org/wiki/World_(blockchain) https://en.m.wikipedia.org/wiki/World_(blockchain) > World Network, originally Worldcoin, is a for-profit cryptocurrency project that uses iris biometrics developed by San Francisco- and Berlin-based Tools for Humanity. Founded in 2019 by OpenAI chief executive Sam Altman, Max Novendstern, and Alex Blania, it has accumulated $250 million in funding from venture capital firms Andreessen Horowitz and Khosla Ventures, as well as from Reid Hoffman.
- gorgoiler 2y ago
  And of course every user interface designed with anccessibility in mind will automatically become an API endpoint, or at the very least an interface that is much easier for machines to use. This goes for aria- labelled forms, but also for button operated doors, tactile surfaces, and information printed in plain and clear English. Robots must love public buildings! Many doors are labelled with a bright green sign using the human word for door (“exit”), most signs and switches are labelled in human binary (braille) and the designers of the building go out of their way to replace lots of pesky steps with nice ramps instead.
ants_everywhere 2y ago
I would think that a lot of apis are designed for human users and we may need new apis designed for a usage patterns more natural to bots.
jackvalentine 2y ago
Unfortunately LOB apps often don’t have a useful API, nor do they have a vendor/support that is capable of supporting you in a timely manner even if they do. RPA bridged the gap a bit for us.
preommr 2y ago
Because money. End-user interfaces can be served ads, are intrinsically rate limited, and are often the main money source. APIs make no ad revenue, can easily rack up expensive server costs, and are often a single third-party service that offer services to the rest of the customer base (bonus points for making you look bad if they implement a feature better).
- necovek 2y ago
  It's trivial to insert ads into an API response. The fact that people don't is simply because we were never really API-first. But it's also why ad-blockers generally work.

deleted 2y ago

[deleted]

solidasparagus 2y ago

People know this. But most actually useful software built for humans doesn't have a good programmatic API. So computer use is the only way to connect AI agents to that software. And that's where a lot of the true (medium-term) financial gain will be - automating tasks on tools that are poorly built or just old but are still valuable.

bberenberg 2y ago

I think everyone agrees with you. The issue is who is building API first interfaces? The value an agent provides here is that it moves control from the publisher to the consumer (in the technical sense). “Twitter doesn’t want to let me access via an API? That’s fine, my browser/agent will automatically automate what I want it to do for me in the UI.” Is a major shift in how we interface with other services.

rixed 2y ago
Come especially handy to work around the increasingly user hostile captchas, ironically. I expect this arm race to spiral down quickly into some form of mandatory biometric check, the dream come true of our security obsessed insecure rulers.
- marxisttemp 2y ago
  Altman already spiraled down there: https://en.m.wikipedia.org/wiki/World_(blockchain) https://en.m.wikipedia.org/wiki/World_(blockchain) > World Network, originally Worldcoin, is a for-profit cryptocurrency project that uses iris biometrics developed by San Francisco- and Berlin-based Tools for Humanity. Founded in 2019 by OpenAI chief executive Sam Altman, Max Novendstern, and Alex Blania, it has accumulated $250 million in funding from venture capital firms Andreessen Horowitz and Khosla Ventures, as well as from Reid Hoffman.
gorgoiler 2y ago
And of course every user interface designed with anccessibility in mind will automatically become an API endpoint, or at the very least an interface that is much easier for machines to use. This goes for aria- labelled forms, but also for button operated doors, tactile surfaces, and information printed in plain and clear English. Robots must love public buildings! Many doors are labelled with a bright green sign using the human word for door (“exit”), most signs and switches are labelled in human binary (braille) and the designers of the building go out of their way to replace lots of pesky steps with nice ramps instead.

ants_everywhere 2y ago

I would think that a lot of apis are designed for human users and we may need new apis designed for a usage patterns more natural to bots.

jackvalentine 2y ago

Unfortunately LOB apps often don’t have a useful API, nor do they have a vendor/support that is capable of supporting you in a timely manner even if they do. RPA bridged the gap a bit for us.

preommr 2y ago

Because money. End-user interfaces can be served ads, are intrinsically rate limited, and are often the main money source. APIs make no ad revenue, can easily rack up expensive server costs, and are often a single third-party service that offer services to the rest of the customer base (bonus points for making you look bad if they implement a feature better).

necovek 2y ago
It's trivial to insert ads into an API response. The fact that people don't is simply because we were never really API-first. But it's also why ad-blockers generally work.