3 ms·

It is great that you care and I guess others are already provided some examples, but I'll add my own 2c here. Obvious problem is that centralized service like C

by SXX 5y ago

It is great that you care and I guess others are already provided some examples, but I'll add my own 2c here. Obvious problem is that centralized service like CloudFlare do create entry barrier and make large players on search and data mining markets even more entrenched than ever. Recently your company announced partnership with Internet Archive, but if CloudFlare want to continue play a role as behevolent party everyone should have equal access to this data. Yeah it means that some bad actors will be able to easily scrap the web too, but... CloudFlare service can't prevent scrapping anyway. There are shady residential proxy networks, services to bypass captcha and scrapping software like Zennoposter. It's possible to make scrapping more expensive, but bad actors don't care because they have money. Unfortunately enthusiasts, open source projects and small companies don't have enough resources to do the same.

dannyw 5y ago
Making scraping harder definitely reduces scraping. Some bad actors will get through, but others will be deterred. I think you might not understand that it's site owners like me who want to stop scraping. It usually comes from specific bad incidents, like copycat sites stealing our content and work. Cloudflare wouldn't block scraping if website owners didn't want it. And website owners can easily disable this protection.
- SXX 5y ago
  Scrapping protection is not a problem: defaults that CloudFlare promote are. Saying that website owners can disable it is akin saying website owners should go and whitelist Tor nodes. Most of website owners don't understand either of issues and they never gonna opt-out. Also I'm talking from experience because I been on both sides of the fence: doing scrapping and implementing protection. So yeah your CloudFlare protection will deter 10% of bad actors, but will also cut off 99% of enthusiast / research efforts or users of niche software or browsers. Still anyone with $1000+ budget will scrap whatever they want.

by SXX 5y ago

dannyw 5y ago

Making scraping harder definitely reduces scraping. Some bad actors will get through, but others will be deterred. I think you might not understand that it's site owners like me who want to stop scraping. It usually comes from specific bad incidents, like copycat sites stealing our content and work. Cloudflare wouldn't block scraping if website owners didn't want it. And website owners can easily disable this protection.

SXX 5y ago
Scrapping protection is not a problem: defaults that CloudFlare promote are. Saying that website owners can disable it is akin saying website owners should go and whitelist Tor nodes. Most of website owners don't understand either of issues and they never gonna opt-out. Also I'm talking from experience because I been on both sides of the fence: doing scrapping and implementing protection. So yeah your CloudFlare protection will deter 10% of bad actors, but will also cut off 99% of enthusiast / research efforts or users of niche software or browsers. Still anyone with $1000+ budget will scrap whatever they want.