3 ms·
Have you heard if this thing called the robots.txt file? To get all those pesky, content-stealing search engines off your lawn, all you need to do is put the fo
by mustpax 17y ago
Have you heard if this thing called the robots.txt file? To get all those pesky, content-stealing search engines off your lawn, all you need to do is put the following two lines in your robots.txt file:
User-agent: *
Disallow: /
This has the added benefit of reducing load on your servers greatly, which means you can get cheaper hosting. It's really a win-win.
For the rest of your comment, I can't really say anything without being pointlessly snide.
- catfish71 17y ago>I can't really say anything without being pointlessly snide. Congratulations, you just summarized the Web 2.0 experience with a sentence fragment.