5 ms·

The key for people who don't get why it was so much work for Google is this: where I just wrote "then go through all of them for which ones had my phrase in lit

The key for people who don't get why it was so much work for Google is this: where I just wrote "then go through all of them for which ones had my phrase in literal order" I meant "then go through all of the cached contents of every single one of the web pages on the results list -- since the joined list itself would have every web site that has those 6 words, which is pretty much every English-language page on the web of more than a few thousand words. So it neeeded to scan through the cached contents of all of them to find which ones had the phrase in order." For example I just looked at the top post on Reddit right now, it's about 20 page-downs of comments, and has the words this many times: from 17 what 22 it 223 is 169 to 195 a - more than 1000 pretty much every English-language web page on the whole Internet will have those words, unless it is very short.

wereHamster 7y ago
> pretty much every web page on the whole Internet will have those words, unless it is very short. … because everyone speaks english?!? England had a large empire back in the days but it wasn't that big, nor did it span the whole world.
- thrwawy3493 7y ago
  updated my comment. My apologies.
- deleted 7y ago
  [deleted]
- treypitt 7y ago
  "The sun never sets on the British Empire" it was up there historically with Rome and Alexander's conquests (although the latter may not have constituted a bona fide empire)
yorwba 7y ago
> So it neeeded to scan through the cached contents of all of them to find which ones had the phrase in order. If you have a positional index [1] you don't have to go through the full cached content, you only have to check the index to see whether there's an occurrence of the words in the query with the correct distance. E.g. for that Reddit post, you'd check the 17 occurrences of "from" and notice quickly that your query string doesn't match at that position. [1] https://nlp.stanford.edu/IR-book/html/htmledition/positional-indexes-1.html https://nlp.stanford.edu/IR-book/html/htmledition/positional...

wereHamster 7y ago

> pretty much every web page on the whole Internet will have those words, unless it is very short. … because everyone speaks english?!? England had a large empire back in the days but it wasn't that big, nor did it span the whole world.

thrwawy3493 7y ago
updated my comment. My apologies.
deleted 7y ago
[deleted]
treypitt 7y ago
"The sun never sets on the British Empire" it was up there historically with Rome and Alexander's conquests (although the latter may not have constituted a bona fide empire)

yorwba 7y ago

> So it neeeded to scan through the cached contents of all of them to find which ones had the phrase in order. If you have a positional index [1] you don't have to go through the full cached content, you only have to check the index to see whether there's an occurrence of the words in the query with the correct distance. E.g. for that Reddit post, you'd check the 17 occurrences of "from" and notice quickly that your query string doesn't match at that position. [1] https://nlp.stanford.edu/IR-book/html/htmledition/positional-indexes-1.html https://nlp.stanford.edu/IR-book/html/htmledition/positional...