5 ms·
It's 1000x compared to a very naive baseline, exhaustive traversal of the matching subtree. Nobody in practice would do that, storing the maximum score for each
by ot 3y ago
It's 1000x compared to a very naive baseline, exhaustive traversal of the matching subtree. Nobody in practice would do that, storing the maximum score for each subtree so you can prune is a very common technique, definitely not novel here.
Even doing a Levenshtein-bounded beam search along the trie is pretty much common practice, see for example this paper [1] from 2011.
There are more sophisticated ways of doing this in space-efficient ways, for example this paper [2] I co-authored in 2013.
[1] https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/WWW11-OnlineSpellingCorrection.pdf https://www.microsoft.com/en-us/research/wp-content/uploads/...
[2] https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/TopKCompletion.pdf https://www.microsoft.com/en-us/research/wp-content/uploads/...
- agencies 3y agoWhat is state of the art currently?
- anonzzzies 3y agoAnd also; are there implementations to look at? Or libraries/open source dbs/search engines that use these?
- sa-code 3y agoLucene's WFST is absurdly fast
- rutgersnj22 3y agohere is a more recent paper where I am one of the authors: http://www.vldb.org/pvldb/vol9/p828-deng.pdf http://www.vldb.org/pvldb/vol9/p828-deng.pdf
- itake 3y agohttps link: https://www.vldb.org/pvldb/vol9/p828-deng.pdf https://www.vldb.org/pvldb/vol9/p828-deng.pdf
- alexchamberlain 3y agoIs the `PruningRadixTrie` the same as the Completion Trie?