4 ms·
Is the common crawl index [1] not being used by search engines? Could someone chime in as to its relative anonymity in many such articles. [1] https://commoncr
by phenkdo 6y ago
Is the common crawl index [1] not being used by search engines? Could someone chime in as to its relative anonymity in many such articles.
[1] https://commoncrawl.org/ https://commoncrawl.org/
- mdaniel 6y agoMy experience with Common Crawl is that it's targeting researchers, because the publish cycle is so long (in web freshness terms) and there doesn't appear to be any webhook or Atom feed which would allow CC to indicate only the diffs that a potential consumer would need to sync