3 ms·

Elastic/OpenSearch have classically used BM25, but the latter recently added semantic search capabilities using neural embeddings in 2.4. Not sure about the for

• andre-z 4y ago
  ES and OS are desperately slow because based on the lucene vector search index. A dedicated vector database like Qdrant will be always a better choice https://github.com/qdrant/qdrant https://github.com/qdrant/qdrant
  • reschkek 4y ago
    Do we have any idea why lucene vector search underperforms? As of lucene 9.1 (and elastic 8.4), it runs the same sort of filtered/categorical HNSW that qdrant runs (https://lucene.apache.org/core/9_1_0/core/org/apache/lucene/search/KnnVectorQuery.html https://lucene.apache.org/core/9_1_0/core/org/apache/lucene/...). Qdrant's benchmarking code (https://github.com/qdrant/vector-db-benchmark/blob/9263ba/engine/clients/elasticsearch/search.py https://github.com/qdrant/vector-db-benchmark/blob/9263ba/en...) does use the new filtered ann query with elastic 8.4, so it appears to be a fair benchmark. Why is lucene/elastic so much slower? Is it a rust vs. java thing? Or some memory management issues?