3 ms·
I only use robots.txt for pages that already issues 403. Something like this: User-agent: * Disallow: /secret/
by HeadlessChild 10y ago
I only use robots.txt for pages that already issues 403. Something like this:
User-agent: *
Disallow: /secret/