3 ms·

Nit: the drawback of “not working well in disk based systems” isn’t a drawback unless you’re already using disk based systems. The difference in recall is also

Nit: the drawback of “not working well in disk based systems” isn’t a drawback unless you’re already using disk based systems. The difference in recall is also significant — what you really get with HNSW is a system made to give good cost:approximation quality. These IVFPQ based systems are ones I’ve seen people rip and replace if the use case is high value. I really don’t understand the push to make pg do everything. It wasn’t designed for search, and trying to shove these features into the platform feels like some misguided cost optimization that puts all of your data infrastructure on the same critical path.

VoVAllen 2y ago
Hi, I'm the author of the article. In our actual product, VectorChord, we adopted a new quantization algorithm called RaBitQ. The accuracy has not been compromised by the quantization process. We’ve provided recall-QPS comparison curves against HNSW, which you can find in our blog: https://blog.pgvecto.rs/vectorchord-store-400k-vectors-for-1-in-postgresql https://blog.pgvecto.rs/vectorchord-store-400k-vectors-for-1.... Many users choose PostgreSQL because they want to query their data across multiple dimensions, including leveraging time indexes, inverted indexes, geographic indexes, and more, while also being able to reuse their existing operational experiences. From my perspective, vector search in PostgreSQL does not have any disadvantages compared to specialized vector databases so fat.
- nostrebored 2y ago
  But why are you benchmarking against pgvector HNSW, which is known to struggle with recall and performance at large numbers of vectors? Why is the graph measuring precision and not recall? The feature dump is entirely a subset of Vespa features. This is just an odd benchmark. I can tell you in the wild, for revenue attached use cases, I saw _zero_ companies choose pg for embedding retrieval.

VoVAllen 2y ago

Hi, I'm the author of the article. In our actual product, VectorChord, we adopted a new quantization algorithm called RaBitQ. The accuracy has not been compromised by the quantization process. We’ve provided recall-QPS comparison curves against HNSW, which you can find in our blog: https://blog.pgvecto.rs/vectorchord-store-400k-vectors-for-1-in-postgresql https://blog.pgvecto.rs/vectorchord-store-400k-vectors-for-1.... Many users choose PostgreSQL because they want to query their data across multiple dimensions, including leveraging time indexes, inverted indexes, geographic indexes, and more, while also being able to reuse their existing operational experiences. From my perspective, vector search in PostgreSQL does not have any disadvantages compared to specialized vector databases so fat.

nostrebored 2y ago
But why are you benchmarking against pgvector HNSW, which is known to struggle with recall and performance at large numbers of vectors? Why is the graph measuring precision and not recall? The feature dump is entirely a subset of Vespa features. This is just an odd benchmark. I can tell you in the wild, for revenue attached use cases, I saw _zero_ companies choose pg for embedding retrieval.