3 ms·
Because the bot requests are consuming significant amounts of bandwidth, memory, CPU, and disk space. Like the intro says, it's just rude, and there's no reason
by phyzome 1y ago
Because the bot requests are consuming significant amounts of bandwidth, memory, CPU, and disk space. Like the intro says, it's just rude, and there's no reason to serve traffic to harvesters like that.
Google also runs an AI scraper, which might be what you saw represented there?
- globie 1y agoFrom the article it's sure starting to seem like people across the internet are just starting to realize what happens when you don't have just 3-4 search engines responsible for crawling for data anymore. When data becomes truly democratized, its access increases dramatically, and we can either adjust or shelter ourselves while the world moves on without us. Did Google never ever scrape individual commits from Gitea?
- HumanOstrich 1y ago> When data becomes truly democratized... That is not at all what is happening.
- globie 1y agoI know, we're locking everything down behind WAFs and repeating captchas so only attested identities can get access in the end.
- HumanOstrich 1y agoYep, everyone is building their own little walled gardens instead of adapting.
- itsafarqueue 1y agoFriend, this IS them adapting.
- HumanOstrich 1y agoMore like maladapting.
- BLKNSLVR 1y agoCan you please provide an explanation of what you consider 'adapting' to be? What's described in the article did my personal definition pretty well.
- hooverd 1y agoOk, and, what your solution?