3 ms·

You can “ping” them about new articles (which I still do), but if other websites ping them within a few seconds/minutes than you basically have a race-condition

by d2wa 8y ago

You can “ping” them about new articles (which I still do), but if other websites ping them within a few seconds/minutes than you basically have a race-condition trying to see how is indexed first. Notably, this is a bigger problem with Bing and Yandex who don’t have the capacity to index the web as quickly as Google does.

hyperpape 8y ago
Is this just not a problem in practice, or is there some technical barrier I'm not thinking of here? I'm imagining a system where you say "hey Google, I'm about to post content X" that you do right before it goes live on your site. No race condition.
- d2wa 8y ago
  Only Yandex has something where you can inform them of upcoming articles/links before you publish. (As mentioned in the article.) https://yandex.com/support/webmaster/authored-texts/owners.html https://yandex.com/support/webmaster/authored-texts/owners.h... If you ping Bing or Google before you publish, they’ll get a 404 and will take that as a sign that there is no content there. They also will wait longer before trying to reindex a page that previously returned a 404.
  - DenisM 8y ago
    Make a visible but unlisted URL, ping search engines, wait 5 minutes, list / link the URL from the home page of your site, publish the URL to RSS. Solves the 404 problem, doesn't it?
    - d2wa 8y ago
      I experimented with this too. However, I found that just delaying the RSS feed produced the desired results.
  - hyperpape 8y ago
    It seems to me that Yandex is doing the right thing. Why does Google not do the same? It seems to me that it's their responsibility as a search engine to give authors tools to identify their work.
testplzignore 8y ago
For WordPress blogs, pings are sent immediately to ping servers. Google could reasonably crawl a new article within one second of the author clicking the publish button. So if you have automated pinging built into your blogging workflow, then there should be no problem. At least until high-frequency traders get involved in this market :)

by d2wa 8y ago

hyperpape 8y ago

Is this just not a problem in practice, or is there some technical barrier I'm not thinking of here? I'm imagining a system where you say "hey Google, I'm about to post content X" that you do right before it goes live on your site. No race condition.

d2wa 8y ago
Only Yandex has something where you can inform them of upcoming articles/links before you publish. (As mentioned in the article.) https://yandex.com/support/webmaster/authored-texts/owners.html https://yandex.com/support/webmaster/authored-texts/owners.h... If you ping Bing or Google before you publish, they’ll get a 404 and will take that as a sign that there is no content there. They also will wait longer before trying to reindex a page that previously returned a 404.
- DenisM 8y ago
  Make a visible but unlisted URL, ping search engines, wait 5 minutes, list / link the URL from the home page of your site, publish the URL to RSS. Solves the 404 problem, doesn't it?
  - d2wa 8y ago
    I experimented with this too. However, I found that just delaying the RSS feed produced the desired results.
- hyperpape 8y ago
  It seems to me that Yandex is doing the right thing. Why does Google not do the same? It seems to me that it's their responsibility as a search engine to give authors tools to identify their work.

testplzignore 8y ago

For WordPress blogs, pings are sent immediately to ping servers. Google could reasonably crawl a new article within one second of the author clicking the publish button. So if you have automated pinging built into your blogging workflow, then there should be no problem. At least until high-frequency traders get involved in this market :)