4 ms·
That only works if the scrapers are either in a country where you can do something about it. Also poisoning only works for a while. As soon as they detect the
by lawl 9y ago
That only works if the scrapers are either in a country where you can do something about it.
Also poisoning only works for a while. As soon as they detect the poisoning, they can easily figure out what tripped the scraping detection, and now you need to poison in an even more subtle way because the scrapers know what to look for.
You can't win this game. Especially the "obvious" type of browser checks, where you can tell the js is checking your browser are so easy to circumvent, because you can tell what they're doing.
Really though, I've reversed a few fingerprinting libs in the wild and also looked at some counter measures that are being sold in the blackhat world. Both sides completely and utterly suck.
The fingerprinting stuff is easy to circument if you're willing to reverse a bit of minified JS, the browser automation 'market leader?' is comically bad and it took me 15 minutes to reliably detect it.
That browser/profile/fingerprint automation thingy in the default option sometimes uses a firefox fingerprint while using chrome. Protip: Chrome and Firefox send HTTP headers in different order. Detected, passive, without JS.
Claiming you're on windows while actually running on a Linux VM, your TCP fingerprint gives it away... Detected, passive, without JS.
Really, both sides, step up your game, this is still boring! Or, just stop fighting scrapers, you can't win.
- eric_h 9y ago> Claiming you're on windows while actually running on a Linux VM, your TCP fingerprint gives it away This is interesting, I’d not heard of this approach. What’s the technique for generating such a fingerprint?
- lawl 9y agohttp://lcamtuf.coredump.cx/p0f3/ http://lcamtuf.coredump.cx/p0f3/
- eric_h 9y agoThanks!
- mehrdadn 9y agoI don't know the actual fingerprint here but check out nmap: https://nmap.org/ https://nmap.org/
- Savageman 9y agoThe most trivial check is using the TCP packet TTL (Time To Live). Windows and Linux doesn't have the same default value so it's "easy" to recognize which one sent a specific packet. Just look up for "ttl windows vs linux" to find tons of article about it :)
- kasmus 9y agoWe could use some professional advice from you, lawl. Please contact me at adelmo.new@gmail.com. /thanks