4 ms·

It doesn't detect "real browsers". It simply adds some friction and is niche enough that AI scrapers have not bothered to bypass it yet.

leros 3mo ago
Yeah this is one of those things that hurts some real users and AI scrapers have figured out how to bypass. Not worth it IMO.
- mostlysimilar 3mo ago
  How do you figure it hurts real users? The amount of compute/energy used on the proof of work is pretty minimal. You're using more when you watch a YouTube video or browse a JS-heavy web app. Of course a sophisticated scraper can "figure out" how to bypass. It isn't trying to be foolproof, it's adding an extra cost to deter massive amounts of bot traffic. I put it in front of my hobby project because I can't afford to serve hundreds of thousands of bot requests from residential proxies all across the world, and I didn't want to route all of my traffic through a third party company like Cloudflare. I've been happy with Anubis.
  - leros 3mo ago
    There are false positives. Every time I've used a tool like this, I get reports from my users about it blocking their access. It's especially bad for users who use privacy oriented browsers like Brave or use VPNs. For a personal site where you don't care about 0.1% of your users getting blocked, it's probably fine. For a business, blocking some of your paying customers isn't acceptable.
    - mostlysimilar 3mo ago
      > For a business, blocking some of your paying customers isn't acceptable. That depends on your business and the costs you're incurring from the bots. It might be justifiable to lose 0.1% of your customers if it means saving server costs from serving useless bot traffic.
      - leros 3mo ago
        I suppose it would depend, yeah. I find having paying customers being blocked to be completely unacceptable. Not only is a bad customer experience, but they tend to be very vocal about it. Posting online, leaving bad reviews, etc.
        ted_dunning 3mo ago
        Having to serve 1000x more traffic from bots is completely unacceptable for most site owners as well. The fact that this trade-off isn't obvious indicates you haven't tried to run a site at scale and make a living off of it.
        leros 3mo ago
        You are quite incorrect. I make a living off a website and 99.9% of my traffic is from bots. Literally hundreds of millions of requests from bots. The cost of bots is worth the cost of not harming legit users.
  - basilikum 3mo ago
    1. It makes the web more annoying and time wasting. 2. I'm a real user and I have to enable JavaScript and run arbitrary programs from untrusted websites to access them. One browser that I use doesn't even support JavaScript (w3m). Fortunately it is unaffected because it doesn't have the string "Mozilla" in its UA and Anubis lets every user without that pass to not hurt legitimate bots like search engine crawlers. As said Anubis is trivially bypassed. The only reason it isn't bypassed is because it's simply too irrelevant to the AI companies.
    - mostlysimilar 3mo ago
      I empathize, I hate the bloated JS-heavy web and avoid it when I can. I just don't see a better solution. The options for me are: sacrifice some legit users by using Anubis, or sacrifice all users by shutting down my service because I can't afford to serve the bot traffic.
    - timbit42 3mo ago
      > 1. It's less annoying than having to click a checkbox.
  - 1718627440 3mo ago
    Over a slow mobile connection Anubis doesn't load at all, even if the actual website would load just fine in seconds. Hence the user is locked out from the website.
axus 3mo ago
There is enough visceral hatred for the Anubis branding, I am surprised an AI skill for bypassing it hasn't been broadcast yet.
randomblock1 3mo ago
Most of the friction is just JS overhead for the computations, a compiled solver is like 1000x faster. If Anubis ever gets popular enough that scrapers care, it would be trivial to defeat. And last I checked you could bypass it by just modifying the user agent
nektro 3mo ago
> and is niche enough that AI scrapers have not bothered to bypass it yet. it is not niche. it is state of the art and widely deployed on major websites.
- throwawaypath 3mo ago
  >it is state of the art and widely deployed on major websites. No it's not.

leros 3mo ago

Yeah this is one of those things that hurts some real users and AI scrapers have figured out how to bypass. Not worth it IMO.

mostlysimilar 3mo ago
How do you figure it hurts real users? The amount of compute/energy used on the proof of work is pretty minimal. You're using more when you watch a YouTube video or browse a JS-heavy web app. Of course a sophisticated scraper can "figure out" how to bypass. It isn't trying to be foolproof, it's adding an extra cost to deter massive amounts of bot traffic. I put it in front of my hobby project because I can't afford to serve hundreds of thousands of bot requests from residential proxies all across the world, and I didn't want to route all of my traffic through a third party company like Cloudflare. I've been happy with Anubis.
- leros 3mo ago
  There are false positives. Every time I've used a tool like this, I get reports from my users about it blocking their access. It's especially bad for users who use privacy oriented browsers like Brave or use VPNs. For a personal site where you don't care about 0.1% of your users getting blocked, it's probably fine. For a business, blocking some of your paying customers isn't acceptable.
  - mostlysimilar 3mo ago
    > For a business, blocking some of your paying customers isn't acceptable. That depends on your business and the costs you're incurring from the bots. It might be justifiable to lose 0.1% of your customers if it means saving server costs from serving useless bot traffic.
    - leros 3mo ago
      I suppose it would depend, yeah. I find having paying customers being blocked to be completely unacceptable. Not only is a bad customer experience, but they tend to be very vocal about it. Posting online, leaving bad reviews, etc.
      - ted_dunning 3mo ago
        Having to serve 1000x more traffic from bots is completely unacceptable for most site owners as well. The fact that this trade-off isn't obvious indicates you haven't tried to run a site at scale and make a living off of it.
        leros 3mo ago
        You are quite incorrect. I make a living off a website and 99.9% of my traffic is from bots. Literally hundreds of millions of requests from bots. The cost of bots is worth the cost of not harming legit users.
- basilikum 3mo ago
  1. It makes the web more annoying and time wasting. 2. I'm a real user and I have to enable JavaScript and run arbitrary programs from untrusted websites to access them. One browser that I use doesn't even support JavaScript (w3m). Fortunately it is unaffected because it doesn't have the string "Mozilla" in its UA and Anubis lets every user without that pass to not hurt legitimate bots like search engine crawlers. As said Anubis is trivially bypassed. The only reason it isn't bypassed is because it's simply too irrelevant to the AI companies.
  - mostlysimilar 3mo ago
    I empathize, I hate the bloated JS-heavy web and avoid it when I can. I just don't see a better solution. The options for me are: sacrifice some legit users by using Anubis, or sacrifice all users by shutting down my service because I can't afford to serve the bot traffic.
  - timbit42 3mo ago
    > 1. It's less annoying than having to click a checkbox.
- 1718627440 3mo ago
  Over a slow mobile connection Anubis doesn't load at all, even if the actual website would load just fine in seconds. Hence the user is locked out from the website.

axus 3mo ago

There is enough visceral hatred for the Anubis branding, I am surprised an AI skill for bypassing it hasn't been broadcast yet.

randomblock1 3mo ago

Most of the friction is just JS overhead for the computations, a compiled solver is like 1000x faster. If Anubis ever gets popular enough that scrapers care, it would be trivial to defeat. And last I checked you could bypass it by just modifying the user agent

nektro 3mo ago

> and is niche enough that AI scrapers have not bothered to bypass it yet. it is not niche. it is state of the art and widely deployed on major websites.

throwawaypath 3mo ago
>it is state of the art and widely deployed on major websites. No it's not.