5 ms·
> Arch Wiki is a high value target for scraping so they'll just solve the anubis challenge once a week. It's not going to stop them. The goal of Anubis isn't t
by pynappo 1y ago
> Arch Wiki is a high value target for scraping so they'll just solve the anubis challenge once a week. It's not going to stop them.
The goal of Anubis isn't to stop them from scraping entirely, but rather to slow down aggressive scraping (e.g. sites with lots of pages being scraped every 6 hours[1]) so that the scraping doesn't impact the backend nearly as much
[1] https://pod.geraspora.de/posts/17342163 https://pod.geraspora.de/posts/17342163, which was linked as an example in the original blog post describing the motivation for anubis[2]
[2]: https://xeiaso.net/blog/2025/anubis/ https://xeiaso.net/blog/2025/anubis/
- jillyboel 1y agoThe point of a static cache is that your backend isn't impacted at all.