5 ms·

Sullivan said he thought Google had enough permission to include links from networks like Twitter in its search results, Schmidt said: “That’s your opinion. If

Sullivan said he thought Google had enough permission to include links from networks like Twitter in its search results, Schmidt said: “That’s your opinion. If you could arrange a letter from Facebook and Twitter to us, that would be helpful.” [1] Sounds like the reporter expects Google to be able to scrape and index all tweets in real time. The whole point here is that (1) that's a big deal and (2) Twitter would prefer to charge dearly for that access if they allowed it at all. In negotiating terms, Google+ provides Google with a great BATNA in this situation where previously all they had was "pay whatever Twitter asks to index tweets or go without". Frankly, the "I believe you have permission, make it happen!" argument hinted at in the above quote sounds just as naive as the Congressman who says "We can do it! We have the technology!" about SOPA. [1] http://www.youtube.com/watch?feature=player_detailpage&v=o3FEILaTP3o#t=110s http://www.youtube.com/watch?feature=player_detailpage&v...

rsynnott 15y ago
Twitter does provide that access; see 'firehouse' here: https://dev.twitter.com/docs/streaming-api/methods https://dev.twitter.com/docs/streaming-api/methods I have no idea what you have to do to get access to it, though.
- Locke1689 15y ago
  You have to have a contract with Twitter. Which they did. Until Twitter decided not to continue to renew their contract with Google. Now Twitter is complaining that Google isn't serving their results prominently. I honestly cannot begin to comprehend the level of cognitive dissonance needed to whine about this.
  - mnutt 15y ago
    I doubt that Twitter just told Google "Sorry, we don't want to do business with you." Most likely either Google wanted a better deal and couldn't get one, or Twitter raised their rates. Which is kind of what the Twitter rep is saying when they state that they continue to work with lots of other companies of various sizes.
    - Locke1689 15y ago
      Yes but what you're saying is that Twitter is upset because Google wasn't willing to pay them enough to show their tweets in Google results. So not only does Twitter want Google to index their tweets, but Google was offering them money for the service and Twitter turned them down.
      - etcet 15y ago
        I'm not entirely sure that was the terms of the agreement. If I remember correctly, Google showed those tweets at the top of the SERP. How is much the #1 spot on all Google SERPs worth?
      - mnutt 15y ago
        Agreed, I don't think Twitter has much right to be upset.
Zirro 15y ago
"Sounds like the reporter expects Google to be able to scrape and index all tweets in real time." Do we know how this worked on the technical side "back in the days" when tweets appeared on Google? Did they share a database, or did Google actually scrape in close to real time? (I can't imagine the second alternative being particularly efficient for any of them, though)
- smackfu 15y ago
  Google used to have access to a firehose of live tweets, but then their contract expired. http://mashable.com/2011/07/04/google-realtime-search-suspended/ http://mashable.com/2011/07/04/google-realtime-search-suspen...
  - Zirro 15y ago
    "While we will not have access to this special feed from Twitter, information on Twitter that’s publicly available to our crawlers will still be searchable and discoverable on Google" "A special feed", that's the information I was looking for. Thanks for the link.

rsynnott 15y ago

Twitter does provide that access; see 'firehouse' here: https://dev.twitter.com/docs/streaming-api/methods https://dev.twitter.com/docs/streaming-api/methods I have no idea what you have to do to get access to it, though.

Locke1689 15y ago
You have to have a contract with Twitter. Which they did. Until Twitter decided not to continue to renew their contract with Google. Now Twitter is complaining that Google isn't serving their results prominently. I honestly cannot begin to comprehend the level of cognitive dissonance needed to whine about this.
- mnutt 15y ago
  I doubt that Twitter just told Google "Sorry, we don't want to do business with you." Most likely either Google wanted a better deal and couldn't get one, or Twitter raised their rates. Which is kind of what the Twitter rep is saying when they state that they continue to work with lots of other companies of various sizes.
  - Locke1689 15y ago
    Yes but what you're saying is that Twitter is upset because Google wasn't willing to pay them enough to show their tweets in Google results. So not only does Twitter want Google to index their tweets, but Google was offering them money for the service and Twitter turned them down.
    - etcet 15y ago
      I'm not entirely sure that was the terms of the agreement. If I remember correctly, Google showed those tweets at the top of the SERP. How is much the #1 spot on all Google SERPs worth?
    - mnutt 15y ago
      Agreed, I don't think Twitter has much right to be upset.

Zirro 15y ago

"Sounds like the reporter expects Google to be able to scrape and index all tweets in real time." Do we know how this worked on the technical side "back in the days" when tweets appeared on Google? Did they share a database, or did Google actually scrape in close to real time? (I can't imagine the second alternative being particularly efficient for any of them, though)

smackfu 15y ago
Google used to have access to a firehose of live tweets, but then their contract expired. http://mashable.com/2011/07/04/google-realtime-search-suspended/ http://mashable.com/2011/07/04/google-realtime-search-suspen...
- Zirro 15y ago
  "While we will not have access to this special feed from Twitter, information on Twitter that’s publicly available to our crawlers will still be searchable and discoverable on Google" "A special feed", that's the information I was looking for. Thanks for the link.