4 ms·
(disclaimer: I'm one of the cofounder of Quickwit.io, we are doing log search) (edited to add vector search category and add manticore) When looking at OSS se
by francoismassot 4y ago
(disclaimer: I'm one of the cofounder of Quickwit.io, we are doing log search)
(edited to add vector search category and add manticore)
When looking at OSS search projects, it's nice to have a rough ideas of the different use cases, here is the list I have in mind:
- user facing search (think algolia):
elasticsearch[1]/opensearch[2], meilisearch[3], typesense[4]
- enterprise search: elasticsearch/opensearch
- log search: elasticsearch/opensearch, loki[5], quickwit[6], zincsearch[7]
- vector search: Milvus[8], Qdrant[9], Vald[10] Weaviate[11]
Vespa.ai[12] is great but I don't really know in which category to put them, user facing and enterprise search seems to work well with them.
It's interesting to note that Elasticsearch and Opensearch are general purpose search engine, Solr as well. They are all powered by Lucene, the popular and performant search engine library.
Another general purpose search engine is Manticore search[13].
I would love to see some benchmarks by category :)
Note: I don't know well ZincSearch and put it in the log search as said on their front page.
[1]: https://github.com/elastic/elasticsearch https://github.com/elastic/elasticsearch
[2]: https://github.com/opensearch-project/OpenSearch https://github.com/opensearch-project/OpenSearch
[3]: https://github.com/meilisearch/meilisearch/ https://github.com/meilisearch/meilisearch/
[4]: https://github.com/typesense/typesense https://github.com/typesense/typesense
[5]: https://github.com/grafana/loki https://github.com/grafana/loki
[6]: https://github.com/quickwit-oss/quickwit https://github.com/quickwit-oss/quickwit
[7]: https://github.com/zinclabs/zinc https://github.com/zinclabs/zinc
[8]: https://github.com/milvus-io/milvus https://github.com/milvus-io/milvus
[9]: https://github.com/qdrant/qdrant https://github.com/qdrant/qdrant
[10]: https://github.com/vdaas/vald https://github.com/vdaas/vald
[11]: https://github.com/semi-technologies/weaviate https://github.com/semi-technologies/weaviate
[12]: https://github.com/vespa-engine/vespa https://github.com/vespa-engine/vespa
[13]: https://github.com/manticoresoftware/manticoresearch https://github.com/manticoresoftware/manticoresearch
- snikolaev 4y ago> It's interesting to note that Elasticsearch and Opensearch are general purpose search engine, Solr as well. They are all powered by Lucene, the popular and performant search engine library. Another search engine which can be considered general, is not based on Lucene and is not less powerful than Elasticsearch/Solr is Manticore Search [1] [1]: https://github.com/manticoresoftware/manticoresearch https://github.com/manticoresoftware/manticoresearch > I would love to see some benchmarks by category :) I'd love too. We started this work on db-benchmarks [2] , hopefully we'll have resources to continue it. Contributions are very welcome. It's 100% opensource [3] [2]: https://db-benchmarks.com/ https://db-benchmarks.com/ [3]: https://github.com/db-benchmarks/db-benchmarks https://github.com/db-benchmarks/db-benchmarks
- drcongo 4y agoI've worked with MeiliSearch a little and found it to be a great experience - the exact opposite of working with Elasticsearch.
- jabo 4y agoI would put Vespa and also Weaviate in a new category called "Vector Search". We're also actively working on adding Vector search to Typesense: https://news.ycombinator.com/item?id=32851894 https://news.ycombinator.com/item?id=32851894
- andre-z 4y agoRight, it's a separate category. There are also Vald, Milvus and Qdrant https://github.com/qdrant/qdrant https://github.com/qdrant/qdrant
- fzliu 4y agoAs an Yahoo internal project, Vespa actually started out very similar to Elastic in that it was meant for fast text search. The team then added ANN search capabilities later on. Vector search infrastructure is overall still fairly new (https://milvus.io/docs/text_search_engine.md https://milvus.io/docs/text_search_engine.md), but up and coming for sure. Disclaimer: I'm a part of the Milvus community.
- rsdbdr203 4y agoNo Splunk[1]? They're the defacto enterprise log platform. [1]: https://www.splunk.com/ https://www.splunk.com/
- francoismassot 4y agoYes Splunk is well known of course, I restricted myself to OSS projects.