3 ms·

Use Anubis - https://anubis.techaro.lol/ https://anubis.techaro.lol/.

• JohnFen 1y ago
  There's very little actual information about Anubis there. At least, I couldn't find much, but perhaps I missed some docs. I'm skeptical about the effectiveness of PoW solutions for this sort of thing, but I'm open to learning about new approaches. How would PoW be effective when the adversary is the user's browser itself and the user is already authenticated?
  • xena 1y ago
    Most scrapers don't run browsers. Making the scrapers run real browsers changes the economics of scraping and makes it less cost-effective. I am working on making it allow more traffic by default and then applying challenges based on request pressure or other factors like system load. I also need to finish the WebAssembly PR and a few other important things. It's a work in progress, but it's used by the United Nations so it can't be that bad :)
    • JohnFen 1y ago
      > Most scrapers don't run browsers. The specific risk I was talking about was that if OpenAI buys Chrome, they could (and I think it's likely they would) use the contents of whatever pages the users browse to as training for their models. Basically, turning the browser into a disguised crawler that would be immune from the usual anti-crawling methods, including putting up a login page.
      • ArinaS 1y ago
        So literally spyware. Don't get me wrong, current Chrome is also spyware, but sending every page you visit to a third-party is just a whole new level.