4 ms·
Meta isn’t going to fix this because they have your mindset. Meanwhile, my website with 48M pages over 8 domains is getting hammered with over 200 req/s 24/7 f
by spiderfarmer 9mo ago
Meta isn’t going to fix this because they have your mindset.
Meanwhile, my website with 48M pages over 8 domains is getting hammered with over 200 req/s 24/7 from AI bots in addition to the regular search engine bots. It seems like every day new bots appear that all want to download every single one of my URL’s.
To me it’s not background noise. It’s a problem. It simply requires a lot of CPU power and traffic. I could do with 95% less resources and have faster response times for my actual users if these bots would just bugger off.