4 ms·

I don't see this is a permanent problem. Right now there must be 1000s of well-funded AI companies trying to scrape the entire internet. Eventually the AI equit

I don't see this is a permanent problem. Right now there must be 1000s of well-funded AI companies trying to scrape the entire internet. Eventually the AI equity bubble will pop and there will be consolidation. If every player left has already scanned the web, will they need to keep constantly scanning it? Seems like no. Even if they do, there will be a lot less of them.

kdheiwns 7mo ago
The current trend is that it's getting cheaper and easier to roll out your own AI on your own computer, so more and more people will do it as a hobby. Even if the big players die out, some dude with a decent gaming PC could decide to start scraping everything pertaining to their interests just for the hell of it. Every government with a budget and someone capable of doing the job will surely get in on it as well.
- overfeed 7mo ago
  > some dude with a decent gaming PC could decide to start scraping everything pertaining to their interests just for the hell of it. Not from their single residential IP, they are not. If they do succeed[1] - it is not going to be at hundreds or thousands of requests per second that the current AI scrapers bombard servers with. Some dude at home will, at best, be putting 4-6 orders of magnitude less strain on a limited set of servers. 1. Scraping is an arms race: if you're just "some dude" at the skill floor - you're going to have a bad time whether you're scraping, or defending against scrapers.