4 ms·

Maybe for a transition period. There's no reason for somebody to create a website, pay for resources, and hope for some sort of revenue if their visitors are m

Maybe for a transition period. There's no reason for somebody to create a website, pay for resources, and hope for some sort of revenue if their visitors are mostly AI. So why bother creating a UI? Instead it would make more sense to close the website and offer the same information as a paid API service. Any sort of website that needs to validate human visitors will be plastered with DRM. Rendering these web browsing LLMs useless. And good riddance as well. Using an LLM to browse the internet feels like a huge waste of resources. Instead it would make more sense to have a wikipedia-like for AIs to crawl via embeddings.

suchintan 3y ago
I suspect that web traffic will encapsulate both. Many websites (government ones in particular) aren't interested in API-based access patterns. This kind of pattern makes it so you can serve both users and agents with a single interface
- MattDaEskimo 3y ago
  This would be ideal. The only issue here is trust. If my website relies on advertising then of course I would prefer to serve more content to a human visitor. So what? I bot protect my site, redirecting the AI to a minimalistic part that most likely expects some sort of value given? People will just breach this trust, like OP and abuse tools like Selenium (as they always have) to imitate being a human.
  - suchintan 3y ago
    I think this is pretty interesting -- I wonder if websites could allow agents to self-identify, and not count them towards advertising CPM to prevent dilution in the advertising metrics Perhaps a similar thing as robots.txt is in order (agents.txt?)
Spivak 3y ago
I mean what kind of websites are we talking about here? The kinds of websites where all the value can be extracted by via a LLM are just content farms. And yeah, that sucks for content farms but putting up content and getting nothing in return is already how ad blockers work and it hasn't destroyed the them. I seriously doubt that AI traffic will even put a dent 1/1000th of the traffic loss of Google snippets.

suchintan 3y ago

I suspect that web traffic will encapsulate both. Many websites (government ones in particular) aren't interested in API-based access patterns. This kind of pattern makes it so you can serve both users and agents with a single interface

MattDaEskimo 3y ago
This would be ideal. The only issue here is trust. If my website relies on advertising then of course I would prefer to serve more content to a human visitor. So what? I bot protect my site, redirecting the AI to a minimalistic part that most likely expects some sort of value given? People will just breach this trust, like OP and abuse tools like Selenium (as they always have) to imitate being a human.
- suchintan 3y ago
  I think this is pretty interesting -- I wonder if websites could allow agents to self-identify, and not count them towards advertising CPM to prevent dilution in the advertising metrics Perhaps a similar thing as robots.txt is in order (agents.txt?)

Spivak 3y ago

I mean what kind of websites are we talking about here? The kinds of websites where all the value can be extracted by via a LLM are just content farms. And yeah, that sucks for content farms but putting up content and getting nothing in return is already how ad blockers work and it hasn't destroyed the them. I seriously doubt that AI traffic will even put a dent 1/1000th of the traffic loss of Google snippets.