3 ms·
Or optimize the website. A knowledge base that isn't being actively updated shouldn't need to hit PHP to serve unauthenticated requests. It doesn't excuse the
by account42 3mo ago
Or optimize the website. A knowledge base that isn't being actively updated shouldn't need to hit PHP to serve unauthenticated requests.
It doesn't excuse the scrapers but a lot of websites are just needlessly wasteful.
- edg5000 3mo agoFrom what I hear it's misconfigured bots that can cause issues. This could happen for dynamic content, where the bot thinks there are a lot of unique pages and it just keeps scraping with slightly different query params for example, sometimes not rate limited nor limited to pages/day/IP. That's my read from the various posts I've read from web admins.