3 ms·
I would prefer if people would not submit sites to HN that are using anti-scraping tactics such as DataDome that block ordinary web users making a single HTTP r
by 1vuio0pswjnm7 3y ago
I would prefer if people would not submit sites to HN that are using anti-scraping tactics such as DataDome that block ordinary web users making a single HTTP request using a non-popular, smaller, simpler client.
One example is www.reuters.com. It makes no sense because the site works fine without Javascript but Javascript is required as a result of the use of DataDome. See below for example demonstration.
For anyone who is doing the scraping that causes these websites to use hacks like DataDome: Does your scraping solution get blocked by DataDome. I suspect many will answer no, indicating to me that DataDome is not effective at anything more than blocking non-popular clients. To be more specific, there seems to be a blurring of the line between blocking non-popular clients and preventing "scraping". If scraping can be accomplished with the gigantic, complex popular clients, then why block the smaller, simpler non-popular clients that make a single HTTP request.
To browse www.reuters.com text-only in a gigantic, complex, popular browser
1. Clear all cookies
2. Allow Javascript in Settings for the site
ct.captcha-delivery.com
3. Block Javascript for the site
www.reuters.com
4. Block images for the site
www.reuters.com
First try browsing www.reuters.com with these settings. Two cookies will be stored. One from reuters.com. This one is the DataDome cookie. And another one from www.reuters.com. This second cookie can be deleted with no effect on browsing.
NB. No ad blocker is needed.
Then clear the cookies, remove the above settings and try browsing www.reuters.com with Javascript enabled for all sites and again without an ad blocker. This is what DataDome and Reuters ask web users to do:
"Please enable JS and disable any ad blocker."
Following this instruction from some anonymous web developer totally locks up the computer I am using. The user experience is unbearable.
Whereas with the above settings I used for the demonstration, browsing and reading is fast.