4 ms·
By default, Anubis will not block Lynx and some other browsers that do not implement JavaScripts, but will block scrapers that claim to be Mozilla-based browser
by zzo38computer 2y ago
By default, Anubis will not block Lynx and some other browsers that do not implement JavaScripts, but will block scrapers that claim to be Mozilla-based browsers, and many of the badly behaving ones do claim to be Mozilla based browsers, so this helps. (I do not have a browser compatible with the Anubis software, and Anubis does not bother me.)
If necessary, it would also be possible to do what powxy does which is displaying an explanation of how the proof of work is working in case you want to implement your own.
Having alternative access of some files using other protocols, might also help, too.
- ranger_danger 2y agoSo any bot author that reads your comment and switches their user agents to curl can now bypass anubis? That doesn't seem very well thought-out to me.
- areyourllySorry 2y agoit's open source, they always could. anubis is just not used by enough sites to be important. ai crawlers have bigger fish to fry, like cloudflare.
- SR2Z 2y agoThe point of Anubis checking the UA string is speed. The main gate is still the proof-of-work scheme, which makes large-scale scraping impractically expensive.