3 ms·
Anubis is not free either, it is a matter of how much it costs to run Anubis vs to let bots in. When I see LKML using Anubis when the pages it serves are tens
by GuB-42 26d ago
Anubis is not free either, it is a matter of how much it costs to run Anubis vs to let bots in.
When I see LKML using Anubis when the pages it serves are tens of kB, all presumably static, I wonder if they do it out of spite rather than to protect their servers.
- Gracana 26d agoIt’s done out of necessity, you can read about why here: https://people.kernel.org/monsieuricon/creepy-crawlies https://people.kernel.org/monsieuricon/creepy-crawlies As a user/reader/viewer I absolutely hate Anubis and usually turn around when I see it pop up (at least on my phone where it takes ages to compute), but with stats like that, I get why a site operator would resort to using it.
- glub 26d ago> But no, let's in fact choose the stupidest possible way of doing it — by rendering everything as HTML commit by commit and then parsing it. This drives me crazy with so-called SOTA LLMs that have "achieved AGI". Fable, Sol, Astra, will start by trying to reverse engineer a binary to figure out how something works when software is open source and one search query away. You let them know it's open source, and they will start using github API instead of just cloning and grepping.
- alexey-salmin 25d agoI think the kernel.org post proves the parent point rather than contradicts it. > At any one time, across 5 geo-distributed nodes, there are 14 CPU cores doing nothing but rendering git commits as html. 14 CPU cores total for running a website like kernel.org is laughable. This is not worth burning cycles in Anubis on client's devices, this is not worth the time of the engineer who worked on it. Provisioning more hardware would have been literally better for everyone.
- Gracana 25d agoI can’t say I really disagree, and as a visitor of the site that’s the solution I would prefer.