4 ms·

The tool "Anubis" uses proof of work instead

timpera 5mo ago
Anubis often takes more than 60 seconds to complete on low-end devices (especially old smartphones). It seems like there's no good solution.
- ImPostingOnHN 5mo ago
  There's not an easy, perfect solution, for sure. Newer phones get faster, but spammer compute gets cheaper. Some sort of decentralized trust web seems like another option, though less viable.
  - WesolyKubeczek 5mo ago
    One of unexpected outcomes from AI-induced hardware shortage may be that, in fact, compute won’t be getting cheaper and may in fact get more expensive…
- dangus 5mo ago
  That must be really low end then. I’ve never seen it complete in a timeframe that was slower than “I can’t even read the page before it redirects”
  - titularcomment 5mo ago
    My guess is its an implementation error, not an hardware limitation. I have two 10-year-old devices and one passes instantaneously while the other halts for a good half minute every time.
- QuantumNomad_ 5mo ago
  But after you’ve completed the Anubis PoW challenge for a site, it remains valid for some amount of time. So it’s not quite as horrible as it sounds. I have setting up Anubis for my own sites on my todo list. And I wish more people did it too. I don’t really mind waiting a little bit extra every now and then before the page loads. What I do mind is ReCaptcha asking me to click all the pictures with buses in them etc. And especially when I have to do it several times over before it’s happy. I’d rather wait a minute for a page to load than to ever solve a ReCaptcha again, if given the choice.
  - mattstir 4mo ago
    > So it’s not quite as horrible as it sounds. I don't know about you, but if a random webpage takes 60+ seconds to load, I just close it and choose to never interact with that site again (unless it's my bank, which is a real and annoying occurrence).
- toastal 4mo ago
  It also requires JavaScript. I like to have JS off by default since running code on my machine is a privilege—one that I opt into, not the the site owner’s choice. This is frustrating since these blockers don’t let me know if the site is trustworthy first before needing to solve a Sudoku for Cloudflare or calculating useless hashes for Anubis.
phoronixrly 5mo ago
How does Anubis stop bots?
- xena 5mo ago
  Bots don't execute JavaScript or follow complicated redirects.
  - pwg 5mo ago
    Bots don't [currently] execute JavaScript or follow complicated redirects. They don't now, but enough "high value to the bots" pages turning on JS or complicated redirects will simply result in the bot authors adding JS execution or redirect following so they can continue "botting" the sites they want to scrape. It's a hole with no bottom. Each one-up on the anti-bot side will eventually be handled on the bot side.
  - ExpertAdvisor01 5mo ago
    That's not true . A lot of bots are just headless chrome instances .
  - ranger_danger 5mo ago
    They have been doing it for years: https://roundproxies.com/blog/bypass-bot-detection/ https://roundproxies.com/blog/bypass-bot-detection/
  - account42 4mo ago
    This level of (mis)undertanding perfectly explains that crapware.
- basilikum 5mo ago
  The real answer is that it makes sites behave different requiring the bots to make slight adjustments. And there are just not enough sites using Anubis for the people and companies running the bots to care to do that. If you do care bypassing Anubis is trivial.
- redwall_hp 5mo ago
BetterThanSober 5mo ago
With a tuned cool down period this isn't a problem, especially if you frequent the sites. OpenWRT uses Anubis and usually when I need to peruse their site I'm on a very low-end device. I prefer waiting much more over finding Waldos But in principle I agree that there's no good answer to this, scraping _is_ useful and I bet most of us here had scraped something, it is AI company and their use of human's material for training without consent and return that led us to this (I know botting exists in forum since forum is a thing but it is easily solved by human moderators and keyword filter)

Anubis often takes more than 60 seconds to complete on low-end devices (especially old smartphones). It seems like there's no good solution.

ImPostingOnHN 5mo ago
There's not an easy, perfect solution, for sure. Newer phones get faster, but spammer compute gets cheaper. Some sort of decentralized trust web seems like another option, though less viable.
- WesolyKubeczek 5mo ago
  One of unexpected outcomes from AI-induced hardware shortage may be that, in fact, compute won’t be getting cheaper and may in fact get more expensive…
dangus 5mo ago
That must be really low end then. I’ve never seen it complete in a timeframe that was slower than “I can’t even read the page before it redirects”
- titularcomment 5mo ago
  My guess is its an implementation error, not an hardware limitation. I have two 10-year-old devices and one passes instantaneously while the other halts for a good half minute every time.
QuantumNomad_ 5mo ago
But after you’ve completed the Anubis PoW challenge for a site, it remains valid for some amount of time. So it’s not quite as horrible as it sounds. I have setting up Anubis for my own sites on my todo list. And I wish more people did it too. I don’t really mind waiting a little bit extra every now and then before the page loads. What I do mind is ReCaptcha asking me to click all the pictures with buses in them etc. And especially when I have to do it several times over before it’s happy. I’d rather wait a minute for a page to load than to ever solve a ReCaptcha again, if given the choice.
- mattstir 4mo ago
  > So it’s not quite as horrible as it sounds. I don't know about you, but if a random webpage takes 60+ seconds to load, I just close it and choose to never interact with that site again (unless it's my bank, which is a real and annoying occurrence).
toastal 4mo ago
It also requires JavaScript. I like to have JS off by default since running code on my machine is a privilege—one that I opt into, not the the site owner’s choice. This is frustrating since these blockers don’t let me know if the site is trustworthy first before needing to solve a Sudoku for Cloudflare or calculating useless hashes for Anubis.

phoronixrly 5mo ago

How does Anubis stop bots?

xena 5mo ago
Bots don't execute JavaScript or follow complicated redirects.
- pwg 5mo ago
  Bots don't [currently] execute JavaScript or follow complicated redirects. They don't now, but enough "high value to the bots" pages turning on JS or complicated redirects will simply result in the bot authors adding JS execution or redirect following so they can continue "botting" the sites they want to scrape. It's a hole with no bottom. Each one-up on the anti-bot side will eventually be handled on the bot side.
- ExpertAdvisor01 5mo ago
  That's not true . A lot of bots are just headless chrome instances .
- ranger_danger 5mo ago
  They have been doing it for years: https://roundproxies.com/blog/bypass-bot-detection/ https://roundproxies.com/blog/bypass-bot-detection/
- account42 4mo ago
  This level of (mis)undertanding perfectly explains that crapware.
basilikum 5mo ago
The real answer is that it makes sites behave different requiring the bots to make slight adjustments. And there are just not enough sites using Anubis for the people and companies running the bots to care to do that. If you do care bypassing Anubis is trivial.
redwall_hp 5mo ago

BetterThanSober 5mo ago

With a tuned cool down period this isn't a problem, especially if you frequent the sites. OpenWRT uses Anubis and usually when I need to peruse their site I'm on a very low-end device. I prefer waiting much more over finding Waldos But in principle I agree that there's no good answer to this, scraping _is_ useful and I bet most of us here had scraped something, it is AI company and their use of human's material for training without consent and return that led us to this (I know botting exists in forum since forum is a thing but it is easily solved by human moderators and keyword filter)