3 ms·
Those bots would be really naive not to use curl-impersonate. I basically use it for any request I make even if I don’t expect to be blocked because why wouldn’
by keanb 1y ago
Those bots would be really naive not to use curl-impersonate. I basically use it for any request I make even if I don’t expect to be blocked because why wouldn’t I.
- f4uCL9dNSnQm 1y agoThere are plenty of naive bots. That is why tar pits work so great at trapping them in. And this TLS based detection looks just like offline/broken site to bots, it will be harder to spot unless you are trying to scrap only that one single site.
- _boffin_ 1y agoI heard about curl-impersonate yesterday when I was hitting a CF page. Did something else to completely bypass it, which has been successful, but should try this.
- VladVladikoff 1y agoA lot of the bots are compromised servers (eg hacked Wordpress sites), with limited control over what the TLS fingerprints look like.