4 ms·

How does proof of work stop bots?

stephantul 5mo ago
Because it destroys the economics of scraping. It’s too expensive with proof of work, or at least not as economically viable
- gruez 5mo ago
  Depends on what type of scraping you're trying to stop. For the dumb scrapers that would try to scrape every page on a git forge (for which there are a bazillion pages for a modest project, because of how the site works), yeah it might deter them enough to stop. For anything high value (eg. reddit comments or retail prices), 10s of cpu time isn't going to stop them.
  - pmontra 5mo ago
    It will not scare away bots but 10 seconds of wait (CPU or only a sleep) will turn away many real users. "This site is so slow, I'll use something else." A kind of reverse captcha.
    - deleted 5mo ago
      [deleted]
    - Hnrobert42 5mo ago
      Maybe, the proof of work can run in the background.
      - btown 5mo ago
        Or it can run as part of a checkout wizard's "verifying your browser and processing your payment, don't close your tab" step.
        mattstir 5mo ago
        At which point it exists solely to punish real human users? What scraper bot is going through checkout?
        BenjiWiebe 5mo ago
        The credit card tester bots go through the checkout process. PoW wouldn't be a big issue for them though since their volume is much lower.
  - stephantul 5mo ago
    Sure, the whole premise is exactly that proof of work reduces the value of scraping, while having negligible impact on users. If the data is so valuable that bot operators are willing to pay 10s of cpu, then other measures are necessary. Nevertheless even for these high value cases, you can still argue that it disincentivizes the business model, it becomes less efficient.
  - thayne 5mo ago
    If it's high value, there isn't really much you can do that will be completely effective. Traditional captchas can often be beaten by AI, or by "captcha farms" where impoverished people are paid pennies to complete captchas. Fingerprinting can be beaten by using a full browser to make the requests. Basically anything you do is just a matter of making it more expensive for bots to access it.
    - arbol 5mo ago
      Beating fingerprinting and beating traditional captcha is far more expensive than solving pow. Pow doesn't stop anyone, not even the most novice bot operators
  - tobyhinloopen 5mo ago
    You can just download all of Reddit from torrent sites
- arbol 5mo ago
  Except it doesn't
  - deleted 5mo ago
    [deleted]
- ranger_danger 5mo ago
  5W load for 2 seconds is 0.002Wh, I think we'll be fine
ray_v 5mo ago
If it gets too expensive/time-consuming to scrape then it won't happen at scale (as much)?

Because it destroys the economics of scraping. It’s too expensive with proof of work, or at least not as economically viable

gruez 5mo ago
Depends on what type of scraping you're trying to stop. For the dumb scrapers that would try to scrape every page on a git forge (for which there are a bazillion pages for a modest project, because of how the site works), yeah it might deter them enough to stop. For anything high value (eg. reddit comments or retail prices), 10s of cpu time isn't going to stop them.
- pmontra 5mo ago
  It will not scare away bots but 10 seconds of wait (CPU or only a sleep) will turn away many real users. "This site is so slow, I'll use something else." A kind of reverse captcha.
  - deleted 5mo ago
    [deleted]
  - Hnrobert42 5mo ago
    Maybe, the proof of work can run in the background.
    - btown 5mo ago
      Or it can run as part of a checkout wizard's "verifying your browser and processing your payment, don't close your tab" step.
      - mattstir 5mo ago
        At which point it exists solely to punish real human users? What scraper bot is going through checkout?
        BenjiWiebe 5mo ago
        The credit card tester bots go through the checkout process. PoW wouldn't be a big issue for them though since their volume is much lower.
- stephantul 5mo ago
  Sure, the whole premise is exactly that proof of work reduces the value of scraping, while having negligible impact on users. If the data is so valuable that bot operators are willing to pay 10s of cpu, then other measures are necessary. Nevertheless even for these high value cases, you can still argue that it disincentivizes the business model, it becomes less efficient.
- thayne 5mo ago
  If it's high value, there isn't really much you can do that will be completely effective. Traditional captchas can often be beaten by AI, or by "captcha farms" where impoverished people are paid pennies to complete captchas. Fingerprinting can be beaten by using a full browser to make the requests. Basically anything you do is just a matter of making it more expensive for bots to access it.
  - arbol 5mo ago
    Beating fingerprinting and beating traditional captcha is far more expensive than solving pow. Pow doesn't stop anyone, not even the most novice bot operators
- tobyhinloopen 5mo ago
  You can just download all of Reddit from torrent sites
arbol 5mo ago
Except it doesn't
- deleted 5mo ago
  [deleted]
ranger_danger 5mo ago
5W load for 2 seconds is 0.002Wh, I think we'll be fine

ray_v 5mo ago

If it gets too expensive/time-consuming to scrape then it won't happen at scale (as much)?