4 ms·
> Very large sites would love to see a 'noindex' parameter in robots.txt; sadly it's not there. Isn't User-agent: * Disallow: / enough?
by gioele 14y ago
> Very large sites would love to see a 'noindex' parameter in robots.txt; sadly it's not there.
Isn't
User-agent: *
Disallow: /
enough?