3 ms·
The point is that the scrapers can easily bypass this if they cared to do so
by valicord 1y ago
The point is that the scrapers can easily bypass this if they cared to do so
- uqers 1y agoHow so?
- tecoholic 1y agoHmm… by setting the verified=1 cookie on every request to the website? Am I missing something here? All this does is set an unencrypted cookie and reload the page right?
- notpushkin 1y agoThey could, but if this is slightly different from site to site, they’ll have to either do this for every site (annoying but possible if your site is important enough), or go ahead and run JS (which... I thought they do already, with plenty of sites still being SPAs?)
- rezonant 1y agoI would be highly surprised if most of these bots are already running JavaScript, I'm confused by this unquestioned notion that they don't.
- valicord 1y agoThe parent comment was "The author of that site assumes that scrapers will keep track of the access tokens for a week, but most internet-wide scrapers don't do so.". There's no technical reason why they wouldn't reuse those tokens, they don't do that today because they don't care. If anubis gets enough adoption to cause meaningful inconvenience, the scrapers would just start caching the tokens to amortize the cost. The point of the article is that if the scraper is sufficiently motivated, Anubis is not going to do much anyway, and if the scraper doesn't care, same result can be achieved without annoying your actual users.