4 ms·
That’s an interesting point about search developing in an absence of a standard for sites to self-index. I can’t quite imagine what that standard index would lo
by onionisafruit 5y ago
That’s an interesting point about search developing in an absence of a standard for sites to self-index. I can’t quite imagine what that standard index would look like.
- dredmorbius 5y agoIt would probably look much like the search index that search engines use. These consist of both individual terms and tuples. (Phrase search can be constructed reasonably reliably by chaining ngram tuples.) Rather than transferring the entire website, it would only be necessary to transfer the index itself. Mind that a sufficiently-large ngram-tuple index is itself large, I suspect limits to 3-word ngrams might be necessary, and should be reasonably sufficient. I'm kicking around the prospect of a DNS-like query mechanism, with cache-and-forward capabilities. That could both distribute search and indexing load. Reputation assessment and management remains the hard piece. Early computer science focused on the relatively easy problem of string search. At Web scale, the problem is of idnetifying reliable and useful sources. That itself isn't a static problem with a single solution as the landscape evolves in response to methods.