4 ms·
A very aggressive crawler.
by pg 14y ago
A very aggressive crawler.
- rehack 14y agoHi, Perhaps you should post the stats, after subtracting out the bot visits, from the stats. For our daily stats we do that. Also (internally) report the bot visit counts. For example yesterday we had: Bot name No. of visits User-Agent=Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html), 95000* User-Agent=Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm), 700* * - the last few digits have been rounded off Edit: For display