5 ms·

Forget Indexing Tweets: Google Is Pulling Them Directly from the API

9 points by hoag 16y ago

drivebyacct2 16y ago
This wasn't common knowledge? Google has been returning results from Twitter and forums within seconds of the content being posted for a long time now. Their "realtime" search is exactly this. If Twitter lets people buy access to the "firehouse" or what not, it stands to reason that Google would be the first in line for such access.
- thesethings 16y ago
  Yeah. This was big, public news. Last year within a week, Bing + Google both paid Twitter millions of dollars for "firehose" access. The reason I remember it pretty well is 1) Lots of people made "...finally making money" jokes, and 2) Both Bing and Google made separate realtime URLs shortly after.
- hoag 16y ago
  I didn't know about it: I must have missed it or glosed over it in my daily (hourly?!) readings of tech and web news. Either way, that's why I posted it now. That said, Google's realtime feed has always been so, well, "realtime," I didn't even notice a change.
pilif 16y ago
> No standard indexing technology could work that quickly. Google has tied their search results directly into the Twitter API What exactly is standard index indexing technology? Just because the search engine built into your blog software can't do it, doesn't mean it's not doable. If you have an opinion and want to share, go ahead, but don't sell your assumptions as fact.
mckoss 16y ago
The author uses the terms API and "indexing technology" imprecisely here. I think what he really means is that they are getting data from Twitter via a push protocol rather than a polling protocol. Note that while Twitter has a site-specific protocol (the firehouse), any site that wants to enable real time searching and alerts can also just implement PubSubHubbub, which can notify any interested subscriber about updates in near real time. As far as I know, Google uses PubSubHubbub to update Google Reader feeds in real time - I'm not sure if this integrates into realtime search results as of now.
deleted 16y ago
[deleted]
joe_the_user 16y ago
Which API? Well obviously they don't know because they're just guessing what Google is doing to begin with. Grr... given that I'm going to using the twitter API, something with details would have been nice...

9 points by hoag 16y ago

This wasn't common knowledge? Google has been returning results from Twitter and forums within seconds of the content being posted for a long time now. Their "realtime" search is exactly this. If Twitter lets people buy access to the "firehouse" or what not, it stands to reason that Google would be the first in line for such access.

thesethings 16y ago
Yeah. This was big, public news. Last year within a week, Bing + Google both paid Twitter millions of dollars for "firehose" access. The reason I remember it pretty well is 1) Lots of people made "...finally making money" jokes, and 2) Both Bing and Google made separate realtime URLs shortly after.
hoag 16y ago
I didn't know about it: I must have missed it or glosed over it in my daily (hourly?!) readings of tech and web news. Either way, that's why I posted it now. That said, Google's realtime feed has always been so, well, "realtime," I didn't even notice a change.

pilif 16y ago

> No standard indexing technology could work that quickly. Google has tied their search results directly into the Twitter API What exactly is standard index indexing technology? Just because the search engine built into your blog software can't do it, doesn't mean it's not doable. If you have an opinion and want to share, go ahead, but don't sell your assumptions as fact.

mckoss 16y ago

The author uses the terms API and "indexing technology" imprecisely here. I think what he really means is that they are getting data from Twitter via a push protocol rather than a polling protocol. Note that while Twitter has a site-specific protocol (the firehouse), any site that wants to enable real time searching and alerts can also just implement PubSubHubbub, which can notify any interested subscriber about updates in near real time. As far as I know, Google uses PubSubHubbub to update Google Reader feeds in real time - I'm not sure if this integrates into realtime search results as of now.

deleted 16y ago

[deleted]

joe_the_user 16y ago

Which API? Well obviously they don't know because they're just guessing what Google is doing to begin with. Grr... given that I'm going to using the twitter API, something with details would have been nice...