4 ms·
So you're saying that the Google spider should break with expected and normal behavior (honoring nofollow) in order to crawl pages that someone else has gone to
by Karunamon 15y ago
So you're saying that the Google spider should break with expected and normal behavior (honoring nofollow) in order to crawl pages that someone else has gone to the trouble of making noncrawlable?
Yeah, that won't get them into any PR hot water or anything! </sarcasm>