4 ms·

Google has said that it won't add a URL to its index of URLs to crawl as a result of seeing the URL in a rel=nofollow href [1]. This is over and above not weig

by chalst 15y ago

Google has said that it won't add a URL to its index of URLs to crawl as a result of seeing the URL in a rel=nofollow href [1]. This is over and above not weighting the link if the URL does otherwise make it onto its to crawl list. [1] http://googleblog.blogspot.com/2007/02/robots-exclusion-protocol.html http://googleblog.blogspot.com/2007/02/robots-exclusion-prot...

Loic 15y ago
Sure, but the probability of having an URL shared only through a tweet is zero. Google is insanely efficient at finding new URLs. So, it will find it anyway, but it will not be shown to you as "shared by your friend John" because of the nofollow.
- Terretta 15y ago
  > the probability of having an URL shared only through a tweet is zero Countless URLs are shared only through tweets, for example URLs that are private URLs to content on services such as TwitPic[1]. 1. http://www.dailymail.co.uk/tvshowbiz/article-2084361/AnnaLynne-McCord-topless-Twitter-picture-90210-star-accidentally-posts-nude-photo.html http://www.dailymail.co.uk/tvshowbiz/article-2084361/AnnaLyn... (NSFW)
dolbz 15y ago
The post you're linking is discussing the META tag version of nofollow, not the hyperlink rel attribute which is detailed here: http://googleblog.blogspot.com/2005/01/preventing-comment-spam.html http://googleblog.blogspot.com/2005/01/preventing-comment-sp... This article doesn't say explicitly that it won't follow those links and I would suggest it often will as comments are a rich source of pages to index. The nofollow on a link like this just indicates that the page owner can't vouch for the quality of the linked page (e.g. user submitted link) and therefore doesn't want to pass pagerank to it. Edit: it's also implied in the parent's link that the META tag only prevents the following of links to pages within your site. This sounds reasonable as who are you to tell Google they can't index a third party domain?
- mapgrep 15y ago
  "who are you to tell Google they can't index a third party domain?" No tweets link to third-party domains, since Twitter wraps all links in their own URL shortener (t.co). Even if you use an independent URL shortener, it will be wrapped by t.co. (I suppose you could argue that there is no way for Google to determine this algorithmically, since "twitter.com" != "t.co", so it should go ahead with the crawl, but there's the question of how Twitter would respond to that.)
  - Raphael 15y ago
    They could create a robots.txt for t.co and set it to noindex.
gojomo 15y ago
Note when Google first unveiled their 'rel=nofollow' invention in 2005, they were very careful to say only that it meant 'no weighting', but not necessarily 'no discovery via this reference'. The attempt to formalize the attribute maintained that same distinction. [1] The Google manager blogging about it two years later (five years ago) was probably correct about then-current practice, but I wouldn't count on that as an enduring commitment that Google had never and would never follow such links. [1] http://microformats.org/wiki/rel-nofollow http://microformats.org/wiki/rel-nofollow

by chalst 15y ago

Loic 15y ago

Sure, but the probability of having an URL shared only through a tweet is zero. Google is insanely efficient at finding new URLs. So, it will find it anyway, but it will not be shown to you as "shared by your friend John" because of the nofollow.

Terretta 15y ago
> the probability of having an URL shared only through a tweet is zero Countless URLs are shared only through tweets, for example URLs that are private URLs to content on services such as TwitPic[1]. 1. http://www.dailymail.co.uk/tvshowbiz/article-2084361/AnnaLynne-McCord-topless-Twitter-picture-90210-star-accidentally-posts-nude-photo.html http://www.dailymail.co.uk/tvshowbiz/article-2084361/AnnaLyn... (NSFW)

dolbz 15y ago

The post you're linking is discussing the META tag version of nofollow, not the hyperlink rel attribute which is detailed here: http://googleblog.blogspot.com/2005/01/preventing-comment-spam.html http://googleblog.blogspot.com/2005/01/preventing-comment-sp... This article doesn't say explicitly that it won't follow those links and I would suggest it often will as comments are a rich source of pages to index. The nofollow on a link like this just indicates that the page owner can't vouch for the quality of the linked page (e.g. user submitted link) and therefore doesn't want to pass pagerank to it. Edit: it's also implied in the parent's link that the META tag only prevents the following of links to pages within your site. This sounds reasonable as who are you to tell Google they can't index a third party domain?

mapgrep 15y ago
"who are you to tell Google they can't index a third party domain?" No tweets link to third-party domains, since Twitter wraps all links in their own URL shortener (t.co). Even if you use an independent URL shortener, it will be wrapped by t.co. (I suppose you could argue that there is no way for Google to determine this algorithmically, since "twitter.com" != "t.co", so it should go ahead with the crawl, but there's the question of how Twitter would respond to that.)
- Raphael 15y ago
  They could create a robots.txt for t.co and set it to noindex.

gojomo 15y ago

Note when Google first unveiled their 'rel=nofollow' invention in 2005, they were very careful to say only that it meant 'no weighting', but not necessarily 'no discovery via this reference'. The attempt to formalize the attribute maintained that same distinction. [1] The Google manager blogging about it two years later (five years ago) was probably correct about then-current practice, but I wouldn't count on that as an enduring commitment that Google had never and would never follow such links. [1] http://microformats.org/wiki/rel-nofollow http://microformats.org/wiki/rel-nofollow