> When you detect a scraper - through what ever means - you don't block it, as that would tip it off that you are on to it. Having written bespoke scraping sys

> When you detect a scraper - through what ever means - you don't block it, as that would tip it off that you are on to it. Having written bespoke scraping systems professionally, I think you overestimate the applicability of this technique. For one thing, detecting that a scraper is a scraper is the problem, not the prelude to the problem. You might as well block them at that point, if you feel you can reliably detect them. If nothing else it's more resource efficient than sending them fake data. Second, and more importantly, "poisoning the well" is not going to work against a sophisticated scraper. I used to use massive amounts of crawled web data to accurately forecast earnings announcements months in advance. I've also consulted with various companies building distributed crawling systems or looking for ways to develop integrations without public APIs. My colleagues would know very quickly if something was wrong with the data because our model would suddenly be extremely out of whack in ways that could be traced back to the website's behavior. We used to specifically look for this sort of thing and basically eyeball the data on a daily basis. We even had tools in place that measured the volume, response time and type of data being received, and would alert us if the ingested data went more than one standard deviation outside of the expectation in any of these metrics. You might succeed in screwing up whatever the data is being used to inform for a little while, but you will, with near certainty, show your hand by doing this, and scrapers will react in the usual cat and mouse game. Modern scraping systems are extremely sophisticated and success stories are prone to confirmation bias, because you're mostly unaware when it's happening successfully on your website. For a while I was experimenting with borrowing (non-malicious) methods from remote timing attacks to identify when servers were treating automated requests differently from manual requests instead of simply dropping them. The rabbit hole of complexity is sufficiently deep that you could saturate a full research time with work to do in this area. If you want to productively block scrapers, you should consider using a captcha-based system at the application layer, preferably a captcha that hasn't been broken yet and which can't be outsourced to a mechanical turk-based API. If nothing else, doing that will introduce at least 10 - 20 seconds of latency per request, which might be intolerable for many scrapers even if they're quite sophisticated.