4 ms·
Search really has 3 main levels. 1. Core ranked inverted index data store (Lucene) 2. API layer on top of Lucene with easier query, index, data importing, sha
by lukevp 6y ago
Search really has 3 main levels.
1. Core ranked inverted index data store (Lucene)
2. API layer on top of Lucene with easier query, index, data importing, sharding, etc. (Solr/Elastic Search)
3. Fully hosted API / UI for easier GUI for developers and search relevance engineers (Algolia, Lucidworks Fusion).
The new hotness in search is currently Rust-based tooling. Rust is a great application for search as it's very performance-sensitive and data-structure heavy, and once the indexes are built are fairly stable, leading to the 90th+ percentile latencies and throughput to be much better than Lucene-based libraries that are built on top of the JVM.
For the low level core search engine like Lucene (inverted indexes, TF-IDF ranking, etc), there's Tantivy [1]
For the middle tier (ES) there's Sonic [2]
MeiliSearch [3] is a play for the hipster open-source Algolia. It's in Rust, is MIT licensed, supports self-hosting, has an out of the box web interface.
[1] https://github.com/tantivy-search/tantivy https://github.com/tantivy-search/tantivy
[2] https://github.com/valeriansaliou/sonic https://github.com/valeriansaliou/sonic
[3] https://github.com/meilisearch/MeiliSearch https://github.com/meilisearch/MeiliSearch
- tekkk 6y agoCool, very interesting! Thanks for the summary, I had/have a project that currently uses Solr for quick text search yet the integration has not been totally painless. Also it's kind of ridiculous how much RAM even the tiniest instance uses. I myself picked up Solr (instead of ES) as it was recommended as better bet for my use case but I did not research in full what others options there were. Rust seems like a smart choice like it probably is for this type of very fast processing. I guess this is yet another reason to learn it.
- gifflar 6y agoI also found a nice overview in the MeiliSearch documentation [1]. Maybe, you find it interesting. [1] https://docs.meilisearch.com/resources/comparison_to_alternatives.html#about-meilisearch https://docs.meilisearch.com/resources/comparison_to_alterna...