3 ms·

> But I mean using a Vector store isn't just a matter of generate some embeddings, slap it in a storage, and then start doing searches like cosine similarity, h

> But I mean using a Vector store isn't just a matter of generate some embeddings, slap it in a storage, and then start doing searches like cosine similarity, hoping for the best. That is 90% of the work with RAG. RAG requires a surprising amount of QA to ensure "the best" is sufficient for your use case. For clarification, cosine similarity is a metric not a search type, and people like it since a) it allows for more defined heuristics since cosine similarity is limited to [-1, 1] and b) it's computationally efficient (dot product, as with Euclidian distance) if the vectors are unit-normalized beforehand. There are indeed multiple search types but HNSW is the best 99% of the time for both performance and latency, and is implemented in almost every modern major vector store.

politelemon 3y ago
> That is 90% of the work with RAG. RAG requires a surprising amount of QA to ensure "the best" is sufficient for your use case. Thanks that's insightful (and also adds to my current impostor syndrome), and makes a lot of sense... so it sounds like it's easy to write the query, but knowing the metric value to use is the testing part. I haven't heard of HNSW but I'll give it a try. It seems PGVector has added HNSW: https://github.com/pgvector/pgvector#hnsw https://github.com/pgvector/pgvector#hnsw
anon373839 3y ago
Has anyone systematically compared embedding based retrieval against traditional full text search? With all of the focus on vector stores lately, it feels like the whole field of information retrieval has been nearly set aside.
- svcrunch 3y ago
  Take a look at the BEIR benchmark, which has served as one of the main drivers for development of neural IR systems since its introduction in 2020. BM25 presents a challenging cross-domain benchmark, and it wasn't till ~2022 that neural methods overtook it. If memory serves, it was the sparse neural methods like Splade, although recent dense models can also beat it. The caveat is that BEIR is suffering from overfitting at this point.
svcrunch 3y ago
> but HNSW is the best 99% of the time for both performance and latency, and is implemented in almost every modern major vector store. In my experience, HNSW indexes are very expensive to build, relative to indexes like IVF. They also have a larger memory footprint. IVF, on the other hand, is pretty trivial to parallelize across multiple machines, and while I'm aware there are techniques for doing that with HNSW, I don't know the details well enough. Also, if you review papers like "SOAR: Improved Quantization for Approximate Nearest Neighbor Search", they hint at some of the throughput barriers faced by graph-based methods like HNSW.

politelemon 3y ago

> That is 90% of the work with RAG. RAG requires a surprising amount of QA to ensure "the best" is sufficient for your use case. Thanks that's insightful (and also adds to my current impostor syndrome), and makes a lot of sense... so it sounds like it's easy to write the query, but knowing the metric value to use is the testing part. I haven't heard of HNSW but I'll give it a try. It seems PGVector has added HNSW: https://github.com/pgvector/pgvector#hnsw https://github.com/pgvector/pgvector#hnsw

anon373839 3y ago

Has anyone systematically compared embedding based retrieval against traditional full text search? With all of the focus on vector stores lately, it feels like the whole field of information retrieval has been nearly set aside.

svcrunch 3y ago
Take a look at the BEIR benchmark, which has served as one of the main drivers for development of neural IR systems since its introduction in 2020. BM25 presents a challenging cross-domain benchmark, and it wasn't till ~2022 that neural methods overtook it. If memory serves, it was the sparse neural methods like Splade, although recent dense models can also beat it. The caveat is that BEIR is suffering from overfitting at this point.

svcrunch 3y ago

> but HNSW is the best 99% of the time for both performance and latency, and is implemented in almost every modern major vector store. In my experience, HNSW indexes are very expensive to build, relative to indexes like IVF. They also have a larger memory footprint. IVF, on the other hand, is pretty trivial to parallelize across multiple machines, and while I'm aware there are techniques for doing that with HNSW, I don't know the details well enough. Also, if you review papers like "SOAR: Improved Quantization for Approximate Nearest Neighbor Search", they hint at some of the throughput barriers faced by graph-based methods like HNSW.