3 ms·
Do you feel the same way about Google spidering for their commercial search engine?
by influx 2y ago
Do you feel the same way about Google spidering for their commercial search engine?
- BadHumans 2y agoGoogle sent traffic to your website. You were at least getting something in return.
- ToucanLoucan 2y agoEmphasis on were, since they've made their search such utter shit in the quest for ad revenue that they're now going to have their own AI sum up your results (badly) instead to attempt to solve the problem they created.
- hmry 2y agoYeah, and when Google stopped sending traffic, people sued them, as they should.
- deleted 2y ago[deleted]
- LukeShu 2y agoI don't. Just 3 AI spiders put more load on our servers than all search engine spiders and all human traffic combined. Some numbers I have handy from before I blocked the bots: ClaudeBot drove more requests through our Redmine in a month than it saw in the combined 5 years prior to ClaudeBot. Bytespider accounted for 59% of the total traffic to our Git server. Amazonbot accounted for 21% of the total traffic to our Git server. Google has never even been close to breaking out of the single-digit-percentages of any metric.
- DamonHD 2y agoGenerally Googlebot is well behaved and efficient these days, though I have discovered that it is currently horribly broken around 429 / 503 response codes... And pays no attention to Retry-After either... Also Google-Podcast which is meant to have been turned off!
- o11c 2y agoSomeone needs to start adding all these AI's homepages to the browser "malware" lists.
- Saris 2y agoNot originally since they just sent people to the site. But these days where they just rip content from the site to give people as answers, completely depriving the site of traffic, yeah that seems basically just as bad as the AI bots.