3 ms·

This could work, especially on high-ranked news sites that are crawled every few minutes for new content. I just noticed a comment on HN that I wrote a few min

by frik 12y ago

This could work, especially on high-ranked news sites that are crawled every few minutes for new content. I just noticed a comment on HN that I wrote a few minutes ago, was already in Google search results - was shocked a bit.

bhartzer 12y ago
They're indexing blog posts and content within seconds, not minutes. I've seen my blog posts indexed within seconds and they've been doing that for a few years now.
- Raphael 12y ago
  The best way to achieve this is with PubSubHubbub to avoid the delay of polling your feed or pages. (Not that it would necessarily stop.)
- 0x0 12y ago
  That's probably because a default wordpress install has pingomatic.com listed in the callbacks for on-post ("update services").
marcosdumay 12y ago
I had a not-important site with many pages, for a while Google was crawing it continuously, with 10 threads in paralell.