4 ms·
If you refer to the "how" page, you'll see that they use noindex and nofollow, a robots.txt to discourage crawling, and a blank 403 to any crawler ignoring the
by abolishme 13y ago
If you refer to the "how" page, you'll see that they use noindex and nofollow, a robots.txt to discourage crawling, and a blank 403 to any crawler ignoring the robot.txt
http://www.donotlink.com/dnl-how http://www.donotlink.com/dnl-how