3 ms·
What does "unfairly open to google" mean?
by davewiner 16y ago
What does "unfairly open to google" mean?
- ez77 16y agoSomething like User-agent: * Disallow: / User-agent: Googlebot Allow: / This can be found on many /robots.txt files. I imagine that some of them will try to enforce this access policy. My point is that equal access for all has never existed. (Not that it would be realistic to expect it.)
- davewiner 16y agoSo to be more precise you mean open to Google and closed to everyone else. I agree that sucks.