4 ms·
The robots.txt file's content-type is text/html, not text/plain. It won't work. I'm still able to access the contents of this site from Yahoo Pipes. Don't worry
by LouCypher 17y ago
The robots.txt file's content-type is text/html, not text/plain. It won't work. I'm still able to access the contents of this site from Yahoo Pipes. Don't worry about it.