4 ms·
The way they're internally implemented doesn't allow pinning an IP. They buy a rotating proxy service from a vendor, and don't get to choose their source IP.
by inigyou 2mo ago
The way they're internally implemented doesn't allow pinning an IP. They buy a rotating proxy service from a vendor, and don't get to choose their source IP.
- semiquaver 2mo agoWhy are you and other defenders of the Anubis approach so fixated on this one specific limitation of a certain type of scraping architecture? It’s hardly an immutable characteristic. You say “they” as if all scrapers are a monolithic group with the same constraints and goals. Part of the problem is the massive diversity.
- inigyou 2mo agoWhy do you think anyone thinks it's immutable? Do you understand the basic concept of an arms race, or cat and mouse game?
- rstuart4133 2mo agoIt's not hard to test. Go to a page that demands PoW, change your IP and see what happens. I just did it. Spoiler: kernel.org asks for a new PoW. If the source IP was an issue, you could do it other ways: for example, make the cookie rotate on every access, and insist there is a single stream of accesses.