3 ms·

Monero is fine with making people wait, that's literally the whole basis of mining blocks in "crypto currency". It's tuned to the time required. Anubis wants "

by chr15m 1mo ago

Monero is fine with making people wait, that's literally the whole basis of mining blocks in "crypto currency". It's tuned to the time required. Anubis wants "slow for bots but fast for humans" which is not possible if somebody has a farm of devices that costs them basically zero.

eru 1mo ago
> [...] which is not possible if somebody has a farm of devices that costs them basically zero. Opportunity costs are real.
margalabargala 1mo ago
One possible way to do this: rather than simply burning hashes for nothing, Anubis can burn hashes to mine crypto. This way, the bots pay for the infrastructure they consume, at least in part. Combine with heuristics like in the article, so you can crank up the difficulty only for likely bot traffic, and you've got a bot supported site.
- arc-in-space 1mo ago
  Disappointed at the lack of replies here, I've had a similar thought before and I'd be curious to hear about the technical viability of this
  - supriyo-biswas 1mo ago
    This was done in the past[1] and was widely regarded as a "cryptojacking" scheme serving "malicious" content to users. [1] https://krebsonsecurity.com/2018/03/who-and-what-is-coinhive/ https://krebsonsecurity.com/2018/03/who-and-what-is-coinhive...
    - GoblinSlayer 1mo ago
      If it was for profit. Here it's in response to abuse by AI companies.
- crote 1mo ago
  CPU crypto mining stopped being viable over a decade ago. If you want a client to pay for the infra cost of their request, they're probably going to need to work for hours to do so.
fc417fc802 1mo ago
> Anubis wants "slow for bots but fast for humans" That's not true. The goal is to raise the cost for bulk scrapers to the extent possible while minimizing any inconvenience to regular users. It's not about being slow for bots but rather reducing the asymmetry of the problem (from the perspective of the server) by imposing a (minor) cost on anyone requesting a page.
- Dylan16807 1mo ago
  That's a better way to put the goal. But does it actually raise the cost or reduce asymmetry in a meaningful way? If it makes crawling 0-5% slower then it's really not worth the annoyance. I know having the Anubis javascript at all gets in the way of some bots. But is the calculation helping enough?
  - fc417fc802 1mo ago
    > does it actually raise the cost or reduce asymmetry in a meaningful way? It's complicated. I think anubis currently only collects low hanging fruit so to speak but in practice that's the bulk of what was causing problems so it's good enough. PoW in general absolutely yes - I think monero clearly demonstrates this. Last I checked it remained viable to mine monero on at least somewhat low end and at least somewhat outdated consumer CPUs. It seems to me that fact implies that a well constructed PoW scheme can in practice squash the impact of any performance differences to within at most a couple orders of magnitude.
    - Dylan16807 1mo ago
      But a couple orders of magnitude is still a ton of asymmetry. And even if you can guarantee the same CPU time on bots and legitimate requests, you still have a bunch of asymmetry in favor of the bots.
      - fc417fc802 1mo ago
        Sure. Even if PoW in general is sound, when applied to scraping it's always going to be a matter of what the client is willing to "pay". It's entirely within the realm of possibility that a scraper could be willing to "outbid" the user of a 5 year old tablet. But it's also true that charging anything is going to be expected to reduce server load, that the cheaper something is the more carelessly people will tend to make use of it, and that at least in general bot operators are expected to be more cost sensitive than regular visitors on account of scale. To that last point consider that a regular visitor will generally have an idle CPU and multiple gigabytes of free RAM. Meanwhile a competent scraper will generally be optimizing request parallelism per cheap VPS while an incompetent one will just point and shoot (and proceed to blindly visit every cgit url). The latter is by far the most troublesome and conveniently also the most likely to be impacted by even a poorly implemented PoW scheme. I keep seeing comments along the lines of anubis only working because it bounces bots running lightweight setups without js but stop and ask yourself why are they running such lightweight setups to begin with?

by chr15m 1mo ago

eru 1mo ago

> [...] which is not possible if somebody has a farm of devices that costs them basically zero. Opportunity costs are real.

margalabargala 1mo ago

One possible way to do this: rather than simply burning hashes for nothing, Anubis can burn hashes to mine crypto. This way, the bots pay for the infrastructure they consume, at least in part. Combine with heuristics like in the article, so you can crank up the difficulty only for likely bot traffic, and you've got a bot supported site.

arc-in-space 1mo ago
Disappointed at the lack of replies here, I've had a similar thought before and I'd be curious to hear about the technical viability of this
- supriyo-biswas 1mo ago
  This was done in the past[1] and was widely regarded as a "cryptojacking" scheme serving "malicious" content to users. [1] https://krebsonsecurity.com/2018/03/who-and-what-is-coinhive/ https://krebsonsecurity.com/2018/03/who-and-what-is-coinhive...
  - GoblinSlayer 1mo ago
    If it was for profit. Here it's in response to abuse by AI companies.
crote 1mo ago
CPU crypto mining stopped being viable over a decade ago. If you want a client to pay for the infra cost of their request, they're probably going to need to work for hours to do so.

fc417fc802 1mo ago

> Anubis wants "slow for bots but fast for humans" That's not true. The goal is to raise the cost for bulk scrapers to the extent possible while minimizing any inconvenience to regular users. It's not about being slow for bots but rather reducing the asymmetry of the problem (from the perspective of the server) by imposing a (minor) cost on anyone requesting a page.

Dylan16807 1mo ago
That's a better way to put the goal. But does it actually raise the cost or reduce asymmetry in a meaningful way? If it makes crawling 0-5% slower then it's really not worth the annoyance. I know having the Anubis javascript at all gets in the way of some bots. But is the calculation helping enough?
- fc417fc802 1mo ago
  > does it actually raise the cost or reduce asymmetry in a meaningful way? It's complicated. I think anubis currently only collects low hanging fruit so to speak but in practice that's the bulk of what was causing problems so it's good enough. PoW in general absolutely yes - I think monero clearly demonstrates this. Last I checked it remained viable to mine monero on at least somewhat low end and at least somewhat outdated consumer CPUs. It seems to me that fact implies that a well constructed PoW scheme can in practice squash the impact of any performance differences to within at most a couple orders of magnitude.
  - Dylan16807 1mo ago
    But a couple orders of magnitude is still a ton of asymmetry. And even if you can guarantee the same CPU time on bots and legitimate requests, you still have a bunch of asymmetry in favor of the bots.
    - fc417fc802 1mo ago
      Sure. Even if PoW in general is sound, when applied to scraping it's always going to be a matter of what the client is willing to "pay". It's entirely within the realm of possibility that a scraper could be willing to "outbid" the user of a 5 year old tablet. But it's also true that charging anything is going to be expected to reduce server load, that the cheaper something is the more carelessly people will tend to make use of it, and that at least in general bot operators are expected to be more cost sensitive than regular visitors on account of scale. To that last point consider that a regular visitor will generally have an idle CPU and multiple gigabytes of free RAM. Meanwhile a competent scraper will generally be optimizing request parallelism per cheap VPS while an incompetent one will just point and shoot (and proceed to blindly visit every cgit url). The latter is by far the most troublesome and conveniently also the most likely to be impacted by even a poorly implemented PoW scheme. I keep seeing comments along the lines of anubis only working because it bounces bots running lightweight setups without js but stop and ask yourself why are they running such lightweight setups to begin with?