4 ms·
On the other side of the spectrum, I recently did a mini review of available F/OSS options for full-text indexers /search tools. Requirements: natively compiled
by tannhaeuser 10y ago
On the other side of the spectrum, I recently did a mini review of available F/OSS options for full-text indexers /search tools. Requirements: natively compiled, one-shot/single-process per search, no long-running containers so EL, SOLR, and anything else based on mighty Lucene and Java was out of scope.
The reviewed tools were: clucene, Apache Lucy, Indri/Lemur, Xapian, and Zettair. Of those, only Xapian and Zettair could be made to return useful search results within time constraints. Sadly, I didn't get Apache Lucy to build, which along with Xapian is the only option still being maintained. clucene could be built, but delivered erratic results (it found "impeachment" but not "indictment" in the US constitution text), and was too difficult to extend into a full stand-alone app (though it apparently is being used in Informix still).
- markpapadakis 10y agoFWIW we will likely open source our search engine/IR technology sometime next month (hold up is the documentation and cleaning up some parts of the codebase). It's written in C++, and the core features are performance, simplicity(operation and extension of functionality) and a sane, simple API (the codebase is quite small as well, at about 14k lines of code). Its similar in some regards to Lucene, but does away with most of the exotic features and instead is optimised for rich queries support, fast execution and extensibility. It should be on GH sometime in April.
- tannhaeuser 10y agoLooking forward to it. Has it a project name yet?
- markpapadakis 10y agoYes, its called Trinity.
- rbutcher 10y agoHave you seen Bleve? https://github.com/blevesearch/bleve https://github.com/blevesearch/bleve I have only tried it for an hour or two, and can't speak for its search result quality, but it seems promising. On my laptop, with an index of 10,000 documents each containing 200 random words, it takes about 110ms to perform a fuzzy search. Half of that is the actual search, the other half is the process/initialization overhead.
- tannhaeuser 10y agoThanks. Will try.
- frik 10y agoSphinx search is C++, and uses little memory compared to Lucene. It has many advanced features like Lucene. https://en.wikipedia.org/wiki/Sphinx_(search_engine) https://en.wikipedia.org/wiki/Sphinx_(search_engine)
- tannhaeuser 10y agoI didn't consider Sphinx because it's using a RDBMS as store. For the same reason, I didn't check out sqlite-fts etc. Maybe I should reconsider?
- frik 10y agoSphinx does NOT use an RDBMS as store. It just provide a convienient interface so you can query Sphinx like you would do a SQL database, just one of several APIs to interact with Sphinx. The data store is very similar to Lucene, a folder with several files.