4 ms·

The cited Danny Sullivan article has a little of his interview with Eric Schmidt: sullivan> I countered that Google seemed to have all the permission it needed

by chalst 15y ago

The cited Danny Sullivan article has a little of his interview with Eric Schmidt: sullivan> I countered that Google seemed to have all the permission it needed, in that they’re not blocked from crawling pages. schmidt>“That’s your opinion,” Schmidt said, then joked: “If you could arrange a letter from Facebook and Twitter to us, that would be helpful.” sullivan> I pushed back that both have effectively given those letters since their robots.txt files — a method of blocking search engines — weren’t telling Google to go away. Well, why stop there? Why shouldn't Google ignore the robots.txt file in its search for shareable nuggets for its search results, which they are equally not "blocked" from using? The answer for both rel=nofollow and robots.txt is that Google has explicitly promised webmasters that it will not do this. Sullivan knows this: this is bad journalism. I'd be curious to see more of the transcript of the interview.

masklinn 15y ago
> The answer for both rel=nofollow and robots.txt is that Google has explicitly promised webmasters that it will not do this. No, not for a/@rel="nofollow", the only promise Google does is that it will not transfer pagerank credit to the link's destination[0]. This does not prevent them from using that link as an outbound in order to discover new content, nor does it prevent Google from reversing the link for the "Shared On" feature. Google specifically advocates[0] using a/@rel=nofollow for user-generated links in order to prevent rewarding spammers, Twitter's use of the attribute is not just sensible, it's necessary. niyazpk's comment[1] looks far more sensible: "Shared On" is a specific feature for content producers and part of a special agreement and API access. [0] http://googleblog.blogspot.com/2005/01/preventing-comment-spam.html http://googleblog.blogspot.com/2005/01/preventing-comment-sp... [1] http://news.ycombinator.com/item?id=3456233 http://news.ycombinator.com/item?id=3456233
- jasonlotito 15y ago
  > the only promise Google does is that it will not transfer pagerank credit to the link's destination Google is fairly clear about this: http://support.google.com/webmasters/bin/answer.py?hl=en&answer=96569 http://support.google.com/webmasters/bin/answer.py?hl=en&#38... "How does Google handle nofollowed links? In general, we don't follow them. This means that Google does not transfer PageRank or anchor text across these links. Essentially, using nofollow causes us to drop the target links from our overall graph of the web. However, the target pages may still appear in our index if other sites link to them without using nofollow, or if the URLs are submitted to Google in a Sitemap. Also, it's important to note that other search engines may handle nofollow in slightly different ways." So, if your tweet includes a link, they drop that link from the post as far as their graph is concerned. And that graph is what makes up their search. It's not just page rank.
  - masklinn 15y ago
    Interesting, so their implementation has broadly expanded upon the original meaning of the attribute (and the one standardized as a microformat), and in recommending this be broadly used to fight against spam they've paved the way for an interesting time for everybody else?
    - jasonlotito 15y ago
      Broadly expanded? Standardized? Please. I think the nofollow attributes intent was fairly straight forward from the beginning: to designate links that should not be accorded attention in search results. In 2005, this was merely via PageRank. Today, it's recommendations in search results. Why would I want to see search results that shouldn't be given weight in a search result in a search result? Explain that to me.
- darkane 15y ago
  Where in Google's blog post do they claim to still follow the link? They provide no technical details for their process, so if we're speculating, it would make far more sense that rel=nofollow adheres to the same guidelines as a meta/robots.txt nofollow. And this speculation for the "shared on" functionality doesn't exactly add up, either. If they're not following links with rel=nofollow, why should Google be legally or even morally responsible for storing a massive and entirely extraneous relational database, and then checking against it every time they follow a short URL from another site, just to see if it was shared on Twitter at some point?
abalone 15y ago
That doesn't make sense. Twitter's robots.txt allows profile pages and tweets to be indexed, and Google does index them. Google is simply choosing to exclude them from their people & places feature so that they can favor their own Google+ profiles. It's only the outbound links within tweets that Google is forbidden from indexing. That has nothing to do with why Google excludes content on twitter.com itself.
- nknight 15y ago
  > That has nothing to do with why Google excludes content on twitter.com itself. Um, they don't. That's not what this is about. You need to research the actual feature that's causing controversy, because it's not "Twitter doesn't show up in search", not even close.
  - abalone 15y ago
    The People & Pages feature. On Google search. Is excluding twitter.com content. Example: http://parislemon.com/post/15682237911/twitter-keeps-right-on-responding-to-search http://parislemon.com/post/15682237911/twitter-keeps-right-o...
    - nknight 15y ago
      So what? That's not part of the standard search. Since when is Google not allowed to advertise its own services alongside standard search results? What's next, do you whine that Bing has links to MSN and Hotmail? Twitter actually contaminates ordinary search results on its site with paid tweets, and you're mad Google puts some Google+ content off to one side? WTF?
      - abalone 15y ago
        Oh, it's definitely part of standard search now. Schmidt said Twitter content wasn't included because they didn't have "permission" to index it. That's false. They are purposefully excluding it -- a point of fact which I'm glad to see you acknowledge now.

by chalst 15y ago

masklinn 15y ago

> The answer for both rel=nofollow and robots.txt is that Google has explicitly promised webmasters that it will not do this. No, not for a/@rel="nofollow", the only promise Google does is that it will not transfer pagerank credit to the link's destination[0]. This does not prevent them from using that link as an outbound in order to discover new content, nor does it prevent Google from reversing the link for the "Shared On" feature. Google specifically advocates[0] using a/@rel=nofollow for user-generated links in order to prevent rewarding spammers, Twitter's use of the attribute is not just sensible, it's necessary. niyazpk's comment[1] looks far more sensible: "Shared On" is a specific feature for content producers and part of a special agreement and API access. [0] http://googleblog.blogspot.com/2005/01/preventing-comment-spam.html http://googleblog.blogspot.com/2005/01/preventing-comment-sp... [1] http://news.ycombinator.com/item?id=3456233 http://news.ycombinator.com/item?id=3456233

jasonlotito 15y ago
> the only promise Google does is that it will not transfer pagerank credit to the link's destination Google is fairly clear about this: http://support.google.com/webmasters/bin/answer.py?hl=en&answer=96569 http://support.google.com/webmasters/bin/answer.py?hl=en&#38... "How does Google handle nofollowed links? In general, we don't follow them. This means that Google does not transfer PageRank or anchor text across these links. Essentially, using nofollow causes us to drop the target links from our overall graph of the web. However, the target pages may still appear in our index if other sites link to them without using nofollow, or if the URLs are submitted to Google in a Sitemap. Also, it's important to note that other search engines may handle nofollow in slightly different ways." So, if your tweet includes a link, they drop that link from the post as far as their graph is concerned. And that graph is what makes up their search. It's not just page rank.
- masklinn 15y ago
  Interesting, so their implementation has broadly expanded upon the original meaning of the attribute (and the one standardized as a microformat), and in recommending this be broadly used to fight against spam they've paved the way for an interesting time for everybody else?
  - jasonlotito 15y ago
    Broadly expanded? Standardized? Please. I think the nofollow attributes intent was fairly straight forward from the beginning: to designate links that should not be accorded attention in search results. In 2005, this was merely via PageRank. Today, it's recommendations in search results. Why would I want to see search results that shouldn't be given weight in a search result in a search result? Explain that to me.
darkane 15y ago
Where in Google's blog post do they claim to still follow the link? They provide no technical details for their process, so if we're speculating, it would make far more sense that rel=nofollow adheres to the same guidelines as a meta/robots.txt nofollow. And this speculation for the "shared on" functionality doesn't exactly add up, either. If they're not following links with rel=nofollow, why should Google be legally or even morally responsible for storing a massive and entirely extraneous relational database, and then checking against it every time they follow a short URL from another site, just to see if it was shared on Twitter at some point?

abalone 15y ago

That doesn't make sense. Twitter's robots.txt allows profile pages and tweets to be indexed, and Google does index them. Google is simply choosing to exclude them from their people & places feature so that they can favor their own Google+ profiles. It's only the outbound links within tweets that Google is forbidden from indexing. That has nothing to do with why Google excludes content on twitter.com itself.

nknight 15y ago
> That has nothing to do with why Google excludes content on twitter.com itself. Um, they don't. That's not what this is about. You need to research the actual feature that's causing controversy, because it's not "Twitter doesn't show up in search", not even close.
- abalone 15y ago
  The People & Pages feature. On Google search. Is excluding twitter.com content. Example: http://parislemon.com/post/15682237911/twitter-keeps-right-on-responding-to-search http://parislemon.com/post/15682237911/twitter-keeps-right-o...
  - nknight 15y ago
    So what? That's not part of the standard search. Since when is Google not allowed to advertise its own services alongside standard search results? What's next, do you whine that Bing has links to MSN and Hotmail? Twitter actually contaminates ordinary search results on its site with paid tweets, and you're mad Google puts some Google+ content off to one side? WTF?
    - abalone 15y ago
      Oh, it's definitely part of standard search now. Schmidt said Twitter content wasn't included because they didn't have "permission" to index it. That's false. They are purposefully excluding it -- a point of fact which I'm glad to see you acknowledge now.