4 ms·
I'm looking at the source code right now, and it seems Whoosh does use Porter stemmer to include word forms for English. Not sure yet if it weighs exact matches
by ntoshev 18y ago
I'm looking at the source code right now, and it seems Whoosh does use Porter stemmer to include word forms for English. Not sure yet if it weighs exact matches heavier.
EDIT: actually it uses query expansion for word forms for English. Took me a while to understand what you mean, probably because English is not my first language and I don't see the relation between "fender" and "automobile" or "music".
The sort of expansions you mean would be trivially accounted for if the search engine used latent semantic indexing.
- delano 18y agoFender is a brand of musical equipment and also another word for the bumper of a car. A straight relation between "fender" and "music" is a trivial expansion but it can become complex very quickly. You can expand from "fender" as a keyword and also as a brand. If you are in the business of selling musical equipment, you may have sold certain result placements to a competing brand like yamaha. In this case, a search for "fender" should produce results that match "music", "guitar", fender products, etc... but yamaha, as a paying advertiser, still expects yamaha products to have a prominent ranking. I would agree that it's counter-intuitive. If I search for fender, I want fender products. However, that is not the only reality and it's the complexity associated to these edge cases that defines what modern search engines can do.