4 ms·

Not sure why Anubis is getting so much hype on HN, but honestly, it is not the solution. A real solution would use behavioral modeling. Most browser fingerprint

by m00dy 3mo ago

Not sure why Anubis is getting so much hype on HN, but honestly, it is not the solution. A real solution would use behavioral modeling. Most browser fingerprinting issues are already largely solved anyway.

fc417fc802 3mo ago
PoW is a reasonable solution as a fallback when other metrics flag a client.
- microtonal 3mo ago
  How? First, they can solve the Anubis challenge with native code, so they can solve them faster than genuine users. Second, the cost is nothing compared to training LLMs, plus they will just move the work to the residential proxies that they have access to (so, someone is paying through their TV's electricity bill). Anubis only work(s|ed) great for a while when crawlers were not prepared for these challenges. Security through obscurity.
  - fc417fc802 3mo ago
    > they will just move the work to the residential proxies that they have access to The proxies I am familiar with do not offer arbitrary code execution. I think you're thinking of a botnet. Regarding native code, the current crop of solutions seem to work well enough for now. Ultimately a challenge response protocol should be standardized and browsers should ship a native implementation. In the meantime WASM likely gets you close enough to native.
    - microtonal 3mo ago
      Regarding native code, the current crop of solutions seem to work well enough for now. I just quoted a toot in another submission, adding it here since it is relevant: We apologize for a period of extreme slowness today. The army of AI crawlers just leveled up and hit us very badly. [...] It seems like the AI crawlers learned how to solve the Anubis challenges. [...] However, we can confirm that at least Huawei networks now send the challenge responses and they actually do seem to take a few seconds to actually compute the answers. It looks plausible, so we assume that AI crawlers leveled up their computing power to emulate more of real browser behaviour to bypass the diversity of challenges that platform enabled to avoid the bot army. https://social.anoxinon.de/@Codeberg/115033790447125787 https://social.anoxinon.de/@Codeberg/115033790447125787
bornfreddy 3mo ago
As someone who is regularly blocked by CF and G - no, they are not. Even worse, they should not be, because that would mean the complete end of privacy online (not that we are far from that). Anubis works because at scale it wastes bots' resources (time mainly).
- inigyou 3mo ago
  it works because the global ddos adversary doesn't run javascript
- m00dy 3mo ago
  I think you’re confusing bots that crawl a website with bots used to launch DDoS attacks.
stephen_cagle 3mo ago
I'm genuinely baffled that this is your lived experience. I wonder if you only look at a very restricted set of websites?

by m00dy 3mo ago

fc417fc802 3mo ago

PoW is a reasonable solution as a fallback when other metrics flag a client.

microtonal 3mo ago
How? First, they can solve the Anubis challenge with native code, so they can solve them faster than genuine users. Second, the cost is nothing compared to training LLMs, plus they will just move the work to the residential proxies that they have access to (so, someone is paying through their TV's electricity bill). Anubis only work(s|ed) great for a while when crawlers were not prepared for these challenges. Security through obscurity.
- fc417fc802 3mo ago
  > they will just move the work to the residential proxies that they have access to The proxies I am familiar with do not offer arbitrary code execution. I think you're thinking of a botnet. Regarding native code, the current crop of solutions seem to work well enough for now. Ultimately a challenge response protocol should be standardized and browsers should ship a native implementation. In the meantime WASM likely gets you close enough to native.
  - microtonal 3mo ago
    Regarding native code, the current crop of solutions seem to work well enough for now. I just quoted a toot in another submission, adding it here since it is relevant: We apologize for a period of extreme slowness today. The army of AI crawlers just leveled up and hit us very badly. [...] It seems like the AI crawlers learned how to solve the Anubis challenges. [...] However, we can confirm that at least Huawei networks now send the challenge responses and they actually do seem to take a few seconds to actually compute the answers. It looks plausible, so we assume that AI crawlers leveled up their computing power to emulate more of real browser behaviour to bypass the diversity of challenges that platform enabled to avoid the bot army. https://social.anoxinon.de/@Codeberg/115033790447125787 https://social.anoxinon.de/@Codeberg/115033790447125787

bornfreddy 3mo ago

As someone who is regularly blocked by CF and G - no, they are not. Even worse, they should not be, because that would mean the complete end of privacy online (not that we are far from that). Anubis works because at scale it wastes bots' resources (time mainly).

inigyou 3mo ago
it works because the global ddos adversary doesn't run javascript
m00dy 3mo ago
I think you’re confusing bots that crawl a website with bots used to launch DDoS attacks.

stephen_cagle 3mo ago

I'm genuinely baffled that this is your lived experience. I wonder if you only look at a very restricted set of websites?