4 ms·
I've always called them abstract squared solutions. I worked with a programmer that inadvertently re-invented tf-idf in SQL, without being familiar with soluti
by tomnipotent 5y ago
I've always called them abstract squared solutions.
I worked with a programmer that inadvertently re-invented tf-idf in SQL, without being familiar with solutions in full-text indexing. He created tables mapping words to documents, kept counts, and came up with simple algorithm to rank documents based on word frequency weighted by overall word scarcity of the words across all documents.
He also re-discovered stemming during this process, different heuristics for splitting words, and other pre-processing methods that come with experience.
This would have been mid-2004, so Lucene wasn't omni-present and Apache Solr hadn't made it's debut. He was half-way to rediscovering n-grams when Solr was released and he stumbled upon it.