6 ms·

Because it destroys the economics of scraping. It’s too expensive with proof of work, or at least not as economically viable

gruez 4mo ago
Depends on what type of scraping you're trying to stop. For the dumb scrapers that would try to scrape every page on a git forge (for which there are a bazillion pages for a modest project, because of how the site works), yeah it might deter them enough to stop. For anything high value (eg. reddit comments or retail prices), 10s of cpu time isn't going to stop them.
- pmontra 4mo ago
  It will not scare away bots but 10 seconds of wait (CPU or only a sleep) will turn away many real users. "This site is so slow, I'll use something else." A kind of reverse captcha.
  - deleted 4mo ago
    [deleted]
  - Hnrobert42 4mo ago
    Maybe, the proof of work can run in the background.
    - btown 4mo ago
      Or it can run as part of a checkout wizard's "verifying your browser and processing your payment, don't close your tab" step.
      - mattstir 4mo ago
        At which point it exists solely to punish real human users? What scraper bot is going through checkout?
        BenjiWiebe 4mo ago
        The credit card tester bots go through the checkout process. PoW wouldn't be a big issue for them though since their volume is much lower.
- stephantul 4mo ago
  Sure, the whole premise is exactly that proof of work reduces the value of scraping, while having negligible impact on users. If the data is so valuable that bot operators are willing to pay 10s of cpu, then other measures are necessary. Nevertheless even for these high value cases, you can still argue that it disincentivizes the business model, it becomes less efficient.
- thayne 4mo ago
  If it's high value, there isn't really much you can do that will be completely effective. Traditional captchas can often be beaten by AI, or by "captcha farms" where impoverished people are paid pennies to complete captchas. Fingerprinting can be beaten by using a full browser to make the requests. Basically anything you do is just a matter of making it more expensive for bots to access it.
  - arbol 4mo ago
    Beating fingerprinting and beating traditional captcha is far more expensive than solving pow. Pow doesn't stop anyone, not even the most novice bot operators
- tobyhinloopen 4mo ago
  You can just download all of Reddit from torrent sites
arbol 4mo ago
Except it doesn't
- deleted 4mo ago
  [deleted]
ranger_danger 4mo ago
5W load for 2 seconds is 0.002Wh, I think we'll be fine

gruez 4mo ago

Depends on what type of scraping you're trying to stop. For the dumb scrapers that would try to scrape every page on a git forge (for which there are a bazillion pages for a modest project, because of how the site works), yeah it might deter them enough to stop. For anything high value (eg. reddit comments or retail prices), 10s of cpu time isn't going to stop them.

pmontra 4mo ago
It will not scare away bots but 10 seconds of wait (CPU or only a sleep) will turn away many real users. "This site is so slow, I'll use something else." A kind of reverse captcha.
- deleted 4mo ago
  [deleted]
- Hnrobert42 4mo ago
  Maybe, the proof of work can run in the background.
  - btown 4mo ago
    Or it can run as part of a checkout wizard's "verifying your browser and processing your payment, don't close your tab" step.
    - mattstir 4mo ago
      At which point it exists solely to punish real human users? What scraper bot is going through checkout?
      - BenjiWiebe 4mo ago
        The credit card tester bots go through the checkout process. PoW wouldn't be a big issue for them though since their volume is much lower.
stephantul 4mo ago
Sure, the whole premise is exactly that proof of work reduces the value of scraping, while having negligible impact on users. If the data is so valuable that bot operators are willing to pay 10s of cpu, then other measures are necessary. Nevertheless even for these high value cases, you can still argue that it disincentivizes the business model, it becomes less efficient.
thayne 4mo ago
If it's high value, there isn't really much you can do that will be completely effective. Traditional captchas can often be beaten by AI, or by "captcha farms" where impoverished people are paid pennies to complete captchas. Fingerprinting can be beaten by using a full browser to make the requests. Basically anything you do is just a matter of making it more expensive for bots to access it.
- arbol 4mo ago
  Beating fingerprinting and beating traditional captcha is far more expensive than solving pow. Pow doesn't stop anyone, not even the most novice bot operators
tobyhinloopen 4mo ago
You can just download all of Reddit from torrent sites

arbol 4mo ago

Except it doesn't

deleted 4mo ago
[deleted]

ranger_danger 4mo ago

5W load for 2 seconds is 0.002Wh, I think we'll be fine