2 ms·
Heh, check out the statistics for that URL: http://bit.ly/info/robots.txt http://bit.ly/info/robots.txt Hundreds of thousands of "direct" hits, and hardly any
by tlrobinson 17y ago
Heh, check out the statistics for that URL:
http://bit.ly/info/robots.txt http://bit.ly/info/robots.txt
Hundreds of thousands of "direct" hits, and hardly any actual referrers. Looks like bit.ly is counting crawlers as hits. Not too surprising given the lack of validation elsewhere.