2 ms·
1. Are there techniques to prevent web scraping? Yes. Are they "Efficient", and even more importantly, effective? Debatable 2. However, more to the point, what
by NikolaNovak 5y ago
1. Are there techniques to prevent web scraping? Yes. Are they "Efficient", and even more importantly, effective? Debatable
2. However, more to the point, what's the end game there? If there are 10,000 outlets out there; and arbitrary number, say 100 of them block scrapes; now the other 9,900 will show up in Google/Facebook search and shares, and those who implemented scraping protection will quietly fade into non-existence.
It's a damned-if-you-do:damned-if-you-don't choice of modern media consumption model. I assume that most outlets have decided that the exposure, trickle of users and monetization they DO get is superior to the influx of users drying out. It's consequence of the oft-quoted platitude that these days, for most people Google and Facebook ARE "The Internet" :-/