4 ms·

> and the data that I’ve experimented with from 2014 seemed high quality That's because it's from the blekko search engine.

• n1xis10t 6mo ago
  Sounds like blekko had a larger impact on the early urls than I thought. Out of curiosity, do you remember how large blekko’s index was at it’s peak?
  • ccgreg 6mo ago
    The largest index we had was 4 billion, which is tiny. Our crawl frontier was much larger.