You can't expect a data scraper that can access the same publicly available information as a regular visitor to somehow magically know the difference. Misconfi

You can't expect a data scraper that can access the same publicly available information as a regular visitor to somehow magically know the difference. Misconfigured servers are 100% the fault of the admin.