3 ms·
I'd like to note that the above is the first time I've ever referenced that URL. No links exist to it on my web site. I just got a visit from the "MJ12Bot" (ht
by bediger4000 13y ago
I'd like to note that the above is the first time I've ever referenced that URL. No links exist to it on my web site.
I just got a visit from the "MJ12Bot" (http://www.majestic12.co.uk/projects/dsearch/mj12bot.php http://www.majestic12.co.uk/projects/dsearch/mj12bot.php) asking for robots.txt, and then the URL above. So, MJ12bot reads Hacker News, or at least follows as the URLs listed in it.