3 ms·

PoW schemes like Anubis don't work. Increasingly bots are using headless browsers and are basically able to solve captchas, proof-of-work(s) and basically bypas

PoW schemes like Anubis don't work. Increasingly bots are using headless browsers and are basically able to solve captchas, proof-of-work(s) and basically bypass all any any attempts to block them. It's becoming impossible to stop bots from hammering your sites/services for unwanted traffic.

m00dy 3mo ago
The most recent captchas that involve basic reasoning can block majority of bots today.
- prologic 3mo ago
  Name one, that an LLM can't solve. The only real mechanism that continues to still work (but doesn't scale) is "Human in the loop".
fc417fc802 3mo ago
You are asserting that they don't work without explanation or evidence. Meanwhile it isn't clear why they wouldn't and indeed they appear to accomplish the stated goal of severely rate limiting scrapers.
- prologic 3mo ago
  Surely I don't need to explain how headless browsers work, bot proxies and distributed crawlers to you do I? If you haven't experienced this first-hand, I can understand. Maybe it warrants a blog post. But I can assure you, these don't work at scale, they might only stop the "less sophisticated" bots and maybe (just maybe) some unwanted spam.
  - fc417fc802 3mo ago
    What do the browsers being headless or the traffic proxied have to do with anything? Each client still has to solve a PoW challenge regardless. That mitigates the (possibly unintentional) DoS by making it exorbitantly expensive.
- gib444 3mo ago
  > You are asserting that they don't work without explanation or evidence. Oh, no, that isn't how it works — you made a claim first (albeit indirectly), with no evidence. It's on you
  - inigyou 3mo ago
    Evidence: they solved many people's problems
  - fc417fc802 3mo ago
    That's not how it works. There are cases where established norms or common sense dictate a certain default assumption. This is one of them. Still, I'll humor your absurd request by reminding you of the many success stories that have repeatedly made the front page of HN. Beyond that we have cryptocurrencies. If you have knowledge of a generalized solution for defeating PoW schemes then why are you posting here instead of making yourself a billionaire?
- codethief 3mo ago
  IIRC there was a blog post here on HN recently explaining in detail why Anubis doesn't keep out AI bots.
  - fc417fc802 2mo ago
    Yes the default config can be trivially bypassed. When it becomes a problem the UA exceptions can be removed. It's no different than how difficulty can be scaled up and down as circumstance demands.
inigyou 3mo ago
There's one or a small number of actors who are DDoSing the internet right now, and those are the ones you care about. What's the purpose for blocking non-DDoS bots?
stephen_cagle 3mo ago
Seems like there is no POW "solving" if the POW in question is literally solving some sort of mathematical puzzle. It has a bounds and we can just scale the bounds depending on how much abuse we are seeing. I've never understood why that isn't more common honestly. It would be cool if I could actually transfer POW from one domain to another as well. Probably wouldn't stop scrapers, but it would likely stop low margin fraud type things.

m00dy 3mo ago

The most recent captchas that involve basic reasoning can block majority of bots today.

prologic 3mo ago
Name one, that an LLM can't solve. The only real mechanism that continues to still work (but doesn't scale) is "Human in the loop".

fc417fc802 3mo ago

You are asserting that they don't work without explanation or evidence. Meanwhile it isn't clear why they wouldn't and indeed they appear to accomplish the stated goal of severely rate limiting scrapers.

prologic 3mo ago
Surely I don't need to explain how headless browsers work, bot proxies and distributed crawlers to you do I? If you haven't experienced this first-hand, I can understand. Maybe it warrants a blog post. But I can assure you, these don't work at scale, they might only stop the "less sophisticated" bots and maybe (just maybe) some unwanted spam.
- fc417fc802 3mo ago
  What do the browsers being headless or the traffic proxied have to do with anything? Each client still has to solve a PoW challenge regardless. That mitigates the (possibly unintentional) DoS by making it exorbitantly expensive.
gib444 3mo ago
> You are asserting that they don't work without explanation or evidence. Oh, no, that isn't how it works — you made a claim first (albeit indirectly), with no evidence. It's on you
- inigyou 3mo ago
  Evidence: they solved many people's problems
- fc417fc802 3mo ago
  That's not how it works. There are cases where established norms or common sense dictate a certain default assumption. This is one of them. Still, I'll humor your absurd request by reminding you of the many success stories that have repeatedly made the front page of HN. Beyond that we have cryptocurrencies. If you have knowledge of a generalized solution for defeating PoW schemes then why are you posting here instead of making yourself a billionaire?
codethief 3mo ago
IIRC there was a blog post here on HN recently explaining in detail why Anubis doesn't keep out AI bots.
- fc417fc802 2mo ago
  Yes the default config can be trivially bypassed. When it becomes a problem the UA exceptions can be removed. It's no different than how difficulty can be scaled up and down as circumstance demands.

inigyou 3mo ago

There's one or a small number of actors who are DDoSing the internet right now, and those are the ones you care about. What's the purpose for blocking non-DDoS bots?

stephen_cagle 3mo ago

Seems like there is no POW "solving" if the POW in question is literally solving some sort of mathematical puzzle. It has a bounds and we can just scale the bounds depending on how much abuse we are seeing. I've never understood why that isn't more common honestly. It would be cool if I could actually transfer POW from one domain to another as well. Probably wouldn't stop scrapers, but it would likely stop low margin fraud type things.