4 ms·
A key aspect [...] Anubis is implemented is that it triggers challenges for every request with a User-Agent containing "Mozilla". Nearly all AI scrapers (and
by keyle 2y ago
A key aspect
[...] Anubis is implemented is that it triggers challenges for every request with a User-Agent containing "Mozilla". Nearly all AI scrapers (and browsers) use a User-Agent string that includes "Mozilla" in it. This means that Anubis is able to block nearly all AI scrapers without any configuration.
Seems like a little loosey-goosey approach, if all it takes is removing Mozilla from the User-Agent string, don't you think that'll be the first thing they do?
- xena 2y agoProbably, but if they do that, then that means that I win because I forced them to make changes to their configuration and code. It means that I would have successfully caused them to waste their time. It would be pretty fun to see that happen.