4 ms·
I wondered how they distinguished bots. The only info I found on the study [1], on a quick search, says: "For the purpose of this report we observed 1.45 Billi
by ds9 13y ago
I wondered how they distinguished bots. The only info I found on the study [1], on a quick search, says:
"For the purpose of this report we observed 1.45 Billion bot visits, which occurred over a 90 day period. The data was collected from a group of 20,000 sites on Incapsula’s network"
So, in the first place, not necessarily representative of the web generally. And it does not say how they detect bots. Of course some declare themselves [2]; otherwise I guess one would have to rely on captchas [3], which are known to be of limited reliability and subject to an arms race.
1. http://www.incapsula.com/the-incapsula-blog/item/820-bot-traffic-report-2013 http://www.incapsula.com/the-incapsula-blog/item/820-bot-tra...
2. kristopolous on this page, '(bot|spider|crawl|aggregator)'
3. which Incapsula does, as a condition of access to its summary of its previous similar research.