3 ms·
I wish that were true, but I'd estimate that less than half of the bot hits on my website properly identify themselves as bots. Most of the crawlers running on
by mjn 11y ago
I wish that were true, but I'd estimate that less than half of the bot hits on my website properly identify themselves as bots. Most of the crawlers running on DigitalOcean/AWS/etc. seem to use a user-agent string lifted from one of the common browsers.
- kuschku 11y agoCheck for the order of the headers and the TLS protocols it claims to support – those are useful for identification.