4 ms·
Urgh! They already ignore robots.txt. This is because we need to write some weird sh!t on all our pages. Now they tell us it doesn't even matter if we do...
by nocturnial 7y ago
Urgh! They already ignore robots.txt.
This is because we need to write some weird sh!t on all our pages. Now they tell us it doesn't even matter if we do...
- dickeytk 7y agonofollow is for external links, robots.txt is for pages on your own domain. They don’t reference the same thing.
- bhartzer 7y agoGoogle will index pages disallowed in robots.txt. If you want a page out of the index, you must allow crawling in robots.txt and use noindex on the page. It used to be that you could use robots.txt to stop indexing but google changed the rules a whole back.
- floatingatoll 7y agoIf you're writing nofollow links to try and influence what pages on your site are or are not prominently displayed in search results, you're doing it wrong. Nofollow links were stated as "My link to this page should not benefit that page's rank", but apparently some people found a loophole: "I can exclude pages from Google's index of my site by nofollow'ing all links to them". Loophole closed.