4 ms·
This has me wondering what it would take to do a bcrypt style slow hashing requirement to retrieve data from a site. Something fast enough that a single mobile
by emseetech 2y ago
This has me wondering what it would take to do a bcrypt style slow hashing requirement to retrieve data from a site. Something fast enough that a single mobile client for a user wouldn't really feel the difference. But an automated scraper would get bogged down in the calculations.
Data is presented to the user with multiple layers of encryption that they use their personal key to decrypt. This might add an extra 200ms to decrypt. Degrades the user experience slightly but creates a bottleneck for large-scale bots.
- koakuma-chan 2y agoCompanies running those bots have more than enough resources
- emseetech 2y agoNobody has unlimited resources. Everything is a cost-benefit analysis. For highly valuable information, they might throw the GDP of a small country at scraping your site. But most information isn't worth that. And there are a lot of bad actors who don't have the resources you're thinking of that are trying to compete with the big guys on a budget. This would cut them out of the equation.
- timewizard 2y agoMake all websites intentionally waste energy as a strategy to defeat unscrupulous operators has negative costs and marginal benefits.
- emseetech 2y agoResources applied to prevent bad actors from degrading or destroying the commons has always been the cost of civilization.
- noosphr 2y agoThen use it to mine monero or similar. The idea that you should pay for content shouldn't be an insane pipedream. It should be the default on the internet. Maybe then we wouldn't be in the situation where getting new users is an existential threat to the majority of websites.
- 01HNNWZ0MV43FF 2y agoI'll add a link on my site recommending that visitors petition their elected officials for a pollution tax
- kevin_thibedeau 2y agoThe problem is that the server also has to do the work. Fine for an infrequent auth challenge. Not so fine for every single data request.
- emseetech 2y agoRight it would need an algorithm with widely different encryption speeds vs decryption speeds. Lattice-based cryptography maybe?
- Retr0id 2y agoHash functions are all you need.
- Dylan16807 2y agoYeah, searching for hashes with some prefix is easy to set up.
- saganus 2y agoMaybe there is a way for the server to ask the client to do the work? Something similar to proof-of-work but on a much smaller scale than Bitcoin.
- mrheosuper 2y agojust add some delay to your response, we don't have to waste any more energy on meaningless calculation.
- 01HNNWZ0MV43FF 2y agoAdding delay means you have to keep more connections open at a single time. Parallelism doesn't favor a server if your problem is already a small server getting hit by a big scraper
- tschwimmer 2y agoCheck out Anubis - it's not quite what you're suggesting but similar in concept: https://anubis.techaro.lol/ https://anubis.techaro.lol/
- emseetech 2y agoInteresting! Not how I'd approach it but certainly thinking along the same lines.
- userbinator 2y agoI just hit a site with this --- and hit the back button immediately.
- Retr0id 2y agoBetter than a 500 error
- fc417fc802 2y agoAlso better than a third party service IMO because of the privacy implications. You could even potentially give users the choice (complete overkill but technically you could do it).
- joeblubaugh 2y agoMaybe once you can use something more professional as an interstitial page
- puchatek 2y agoIf we are able to detect AI scrapers then I would welcome a more strategic solution: feed them garbage data instead of the real content. If enough sites did that then the inference quality would take a hit and eventually the perpetrators, too. But of course this is the more expensive option that can't really be asked of sites that already provide public services (even if those are paid for by ads).
- brookst 2y agoI really don’t think we have such a lack of misinformation that we need to invest in creating more of it, no matter the motive.
- baq 2y agoSee also https://datatracker.ietf.org/wg/privacypass/about/ https://datatracker.ietf.org/wg/privacypass/about/ and https://developers.cloudflare.com/waf/tools/privacy-pass/ https://developers.cloudflare.com/waf/tools/privacy-pass/