4 ms·

Forgive me I'm not super familiar with the vector indexes outside of the basic tsvector for text search. What's the difference between pg_embedding, pg_vector,

Forgive me I'm not super familiar with the vector indexes outside of the basic tsvector for text search. What's the difference between pg_embedding, pg_vector, and tsvector? Are they compariable/interchangable? And how do you know which one to pick? My understanding is pg_vector has poorer performance compared to some dedicated vector databases, does pg_embedding perform better? Sorry if these are silly questions.

lukev 3y ago
Different vector indexing algorithms under the hood, with different tradeoffs. In this case "hnsw" is the name of the algorithm. If you want to know more, just search "hnsw" and "ivfflat" to understand the differences. tsvector is something else; it's not a numeric vector index, it's a different kind of data structure that stores the actual lexemes (keywords) with positions and weights to facilitate full text search (rather than using a text embedding model.)
HammadB 3y ago
pg_embedding is based on a different approximate nearest neighbors algorithm - HNSW, which is generally considered - and studied - to be faster as well as more accurate than pgvectors IVF (ignoring a lot of nuance). However pg_embedding serves the index out of disk, whereas most vector databases opt to serve the index out of memory, or delegate to mmap'ed files. HNSW is a graph-based algorithm, thus access patterns are random and this does not lend itself to disk based access. I'd expect pg_embedding to be slower than memory-resident indices due to this fact. Also in general with a postgres index, my concern would be scalability and resource isolation. It's convenient to colocate these things but ANN indices have very different CPU/Memory/Disk usage patterns than what you may need for just your relational data. For example Chroma -> Serves HNSW out of memory, persists to disk with a WAL. Weviate -> Serves HNSW out of memory, writes HNSW graph search to WAL and uses that for durability. Milvus -> Serves HNSW out of memory, supports partial mmap, also supports another algorithm called DiskANN which is optimized for SSD. It uses cloud storage with a shared-everything architecture for durability (a lot of nuance here.) QDrant -> Has a WAL, supports memory and mmap'ed indices.
- roseway4 3y ago
  As I mentioned elsewhere, there's more to vector database selection than raw performance: Developers leveraging their existing experience with Postgres, existing infrastructure investment (often in managed Postgres on AWS/GCP etc), and a single API into the vector store shared with other parts of their app (their ORM / DB layer). Many teams can also get away with good performance vs the fastest performance, given smaller index sizes and the other tradeoffs I mentioned. That said, I can imagine the pgvector folks precaching the new HNSW index support they're working on, as they do with their IVFFLAT index. * edited for the grammar gremlins
  - HammadB 3y ago
    Sure, I don't disagree that there is more to vector database selection than raw performance. Any technical decision is filled with many considerations. The commenter - thewataccount - asked about performance and I shared my intuition.
  - nikita 3y ago
    pgvector is working HNSW too. We have a blog post about it: https://neon.tech/blog/pgvector-meets-hnsw-index https://neon.tech/blog/pgvector-meets-hnsw-index
- nikita 3y ago
  Well it doesn't serve it from disk. It's persisted to disk and Postgres buffer cache keeps the working set in memory.
  - HammadB 3y ago
    Maybe I am misunderstanding but the postgres buffer cache is LRU and will evict pages right? At times data may go to disk and then during serving it will have to be loaded into memory? So it is quite dependent on the size of your buffer cache as well as contention for that buffer cache. Also the cache access patterns will vary between these implementations and is worth considering.

lukev 3y ago

Different vector indexing algorithms under the hood, with different tradeoffs. In this case "hnsw" is the name of the algorithm. If you want to know more, just search "hnsw" and "ivfflat" to understand the differences. tsvector is something else; it's not a numeric vector index, it's a different kind of data structure that stores the actual lexemes (keywords) with positions and weights to facilitate full text search (rather than using a text embedding model.)

HammadB 3y ago

pg_embedding is based on a different approximate nearest neighbors algorithm - HNSW, which is generally considered - and studied - to be faster as well as more accurate than pgvectors IVF (ignoring a lot of nuance). However pg_embedding serves the index out of disk, whereas most vector databases opt to serve the index out of memory, or delegate to mmap'ed files. HNSW is a graph-based algorithm, thus access patterns are random and this does not lend itself to disk based access. I'd expect pg_embedding to be slower than memory-resident indices due to this fact. Also in general with a postgres index, my concern would be scalability and resource isolation. It's convenient to colocate these things but ANN indices have very different CPU/Memory/Disk usage patterns than what you may need for just your relational data. For example Chroma -> Serves HNSW out of memory, persists to disk with a WAL. Weviate -> Serves HNSW out of memory, writes HNSW graph search to WAL and uses that for durability. Milvus -> Serves HNSW out of memory, supports partial mmap, also supports another algorithm called DiskANN which is optimized for SSD. It uses cloud storage with a shared-everything architecture for durability (a lot of nuance here.) QDrant -> Has a WAL, supports memory and mmap'ed indices.

roseway4 3y ago
As I mentioned elsewhere, there's more to vector database selection than raw performance: Developers leveraging their existing experience with Postgres, existing infrastructure investment (often in managed Postgres on AWS/GCP etc), and a single API into the vector store shared with other parts of their app (their ORM / DB layer). Many teams can also get away with good performance vs the fastest performance, given smaller index sizes and the other tradeoffs I mentioned. That said, I can imagine the pgvector folks precaching the new HNSW index support they're working on, as they do with their IVFFLAT index. * edited for the grammar gremlins
- HammadB 3y ago
  Sure, I don't disagree that there is more to vector database selection than raw performance. Any technical decision is filled with many considerations. The commenter - thewataccount - asked about performance and I shared my intuition.
- nikita 3y ago
  pgvector is working HNSW too. We have a blog post about it: https://neon.tech/blog/pgvector-meets-hnsw-index https://neon.tech/blog/pgvector-meets-hnsw-index
nikita 3y ago
Well it doesn't serve it from disk. It's persisted to disk and Postgres buffer cache keeps the working set in memory.
- HammadB 3y ago
  Maybe I am misunderstanding but the postgres buffer cache is LRU and will evict pages right? At times data may go to disk and then during serving it will have to be loaded into memory? So it is quite dependent on the size of your buffer cache as well as contention for that buffer cache. Also the cache access patterns will vary between these implementations and is worth considering.