3 ms·
Obvious question: how do you filter out bot traffic with server side logs? What percent of visitors are bots anyway?
by gentleman11 6y ago
Obvious question: how do you filter out bot traffic with server side logs? What percent of visitors are bots anyway?
- bleepblorp 6y agoMost legitimate bots identify themselves with specific user agent strings. Script kiddie attack bots are generally fairly obvious as they hammer away at things like /wp-login.php for days on end regardless of what error codes the server returns. Most other bots are pretty evident just by looking at access patterns. Just identify their IPs and drop them from your analytics.
- toast0 6y agoUser-Agent for nice bots, and client ASN for naughty bots will get you pretty far. Fake chrome from residential IPs is hard to detect though.
- compumike 6y agoThere are off-the-shelf open-source libraries for this that are pretty decent and are kept up-to-date by the community. For example, you can just do browser.is_bot? after you install https://github.com/fnando/browser#bots https://github.com/fnando/browser#bots
- ratsbane 6y agoWatching the bot traffic is an interesting exercise in itself. The trick is not to filter it out; it's to identify it (to the greatest degree possible.)