8 ms·
VectorChord: Store 400k Vectors for $1 in PostgreSQL
- isogn 2y ago[flagged]
- deleted 2y ago[deleted]
- gaocegege 2y agoHey everyone! We’ve developed a new PostgreSQL extension that supports 400k vectors for just $1. Check it out!
- 7qW24A 2y agoThe “external index build” idea seems pretty interesting. How does it work with updates to the underlying data (e.g., new embeddings being added)? For that matter, I guess, how do incremental updates to pgvector’s HNSW indexes work?
- VoVAllen 2y agoThe IVF indexing can be considered into two phases, computing the centroids (KMeans), and assigning each point to the centroids as the inverted lists. The most time-consuming part is at the KMeans stage, and can be greatly accelerated with GPU. 1M 960dim vec can be clustered in less than 10s. We did the KMeans phase externally, and the assignment phase inside postgres. The KMeans part depends only on the data distribution, not on any specific data. So we can do sampling on the data, and inserting/deleting the data won't affect the KMeans result significantly. For the update, it's just a matter of assigning the new vector to a specific cluster and appending it to the corresponding list. It's very light compared to inserting in hnsw
- nextworddev 2y agoWhat dimension vectors are we talking here
- jesperwe 2y agoFirst paragraph of tfa mentions "768-dimensional vectors"
- jasonkester 2y agoIn five pages of text, we never get to learn what a Vector is (in this context), why we’d want to store one in pgsql, or why it costs so much to store them compared to anything else you’d store there. For an example of how you can communicate with domain experts, while still giving everyone else some form of clue as to what this hell you’re talking about, check out the link to the product that this thing claims to be a successor to: https://pgvecto.rs/ https://pgvecto.rs/ That starts off by telling us what it is and what it does.
- lja 2y agoThat's because this product isn't for you then. My team has been evaluating vector databases for years and everything on the VectorChord page resonated with me. We run one of the world's largest vector databases and we'll likely benchmark vectorchord to see if it lives up to its promises here.
- gaocegege 2y agoHi, we’re here to help if you need assistance (via GitHub issue, Discord, or email). Could you let us know the scale of your vectors—are they 1B or 10B?
- lja 2y ago10B, feel free to email me luke@ the domain in my profile.
- thelittleone 2y agoDo you have a blog per chance? Or any recommended reading on pre processing / data chunking strategies to improve results?
- VoVAllen 2y agoI recently came across a project that looks promising: [WordLlama](https://github.com/dleemiller/WordLlama?tab=readme-ov-file#semantic-text-splitting https://github.com/dleemiller/WordLlama?tab=readme-ov-file#s...). It appears to be well-suited for semantic chunking, though I haven’t had a chance to try it out yet.
- curl-up 2y agoDoes this mean you won't support pgvecto.rs anymore?
- VoVAllen 2y agoI'm the project lead for both projects. We're still in the process of supporting all the function from pgvecto.rs in VectorChord (int8, more than 2000 dim vec, etc.). We'll provide the migration docs for pgvecto.rs users to VectorChord. User will have better experience with VectorChord due to better integration with postgres storage system. We will stop supporting pgvecto.rs early next year when everything on VectorChord is ready.
- deleted 2y ago[deleted]
- marcyb5st 2y agoAwesome work! But aren't the comparisons missing ScaNN [1, 2]? I think it's the overall SOTA [3] at the moment regarding vector indexing. [1] https://github.com/google-research/google-research/tree/master/scann https://github.com/google-research/google-research/tree/mast... [2] Also available on something like AlloyDB on GCP: https://cloud.google.com/alloydb/docs/ai/store-index-query-vectors?resource=scann https://cloud.google.com/alloydb/docs/ai/store-index-query-v... [3] https://ann-benchmarks.com/glove-100-angular_10_angular.html https://ann-benchmarks.com/glove-100-angular_10_angular.html Disclaimer: Working for Google, but nowhere close to Databases.
- VoVAllen 2y agoI'm the project lead for VectorChord. I have tested ScaNN on AlloyDB Omni but have struggled to achieve reasonable recall on the GIST 1M dataset, with results peaking at only around 0.8. The limited documentation makes it challenging to understand the underlying causes of this performance. Additionally, I couldn’t find any performance benchmarks for ScaNN integrated with PostgreSQL, particularly in comparison to pgvector or its standalone. The publicly available metrics focus exclusively on query-only indexing outside of the database. On our side, we’ve implemented the fastscan kernel for bit vector scanning, which is considered as one of ScaNN’s key advantages.
- marcyb5st 2y agoThanks for the explanation! Really appreciate it and it makes perfect sense.
- tarasglek 2y agoI am still waiting for a good pattern for using multivector embeddings like ColBert and ColPali in postgres. I get that its fun to optimize single vector stuff, but multivector is that happy middleground between single vector and reranker that seems to be only validated in specialized exotic search dbs like Vespa
- tarasglek 2y agoLooks like that there is recent work to make a pgvector example for this https://github.com/pgvector/pgvector-python/blob/master/examples/colpali/exact.py https://github.com/pgvector/pgvector-python/blob/master/exam...
- VoVAllen 2y agoThere's no easy way to index ColBert multi-vectors in a scalable way that I know of. Vespa seems to rely heavily on binary quantization, which can cost a lot in recall loss. And for most cases, using ColBert as a reranker is good enough, as the pgvector example you posted.
- tarasglek 2y agoSeems like like doing a proper relational 1:N chunk:multiple-vectors foreign key, binarization and a clever join or multistage CTE would get us pretty close to useful. I am ok with it being less efficient as the dev ux will be amazing. Vespa ops (even in their cloud) are a complete nightmare compared to postgres
- jonathan-adly 2y agoI would like to throw our project in the ring that solves this problem: https://github.com/tjmlabs/ColiVara https://github.com/tjmlabs/ColiVara 1. Uses half-vecs, so you cut down everything by half with no recall loss 2. Uses token pooling with hierarchial clustering at 3, so, you further cut down things by 2/3rd with <1% loss 3. Everything is on Postgres and pgvector, so you can do all the Postgres stuff and decrease corpus size by document metadata filtering 4. We have a 5000+ pages corpus in production with <3 seconds latency. 5. We benchmark against the Vidore leaderboard, and very near SOTA You can read about half-vecs here: https://jkatz05.com/post/postgres/pgvector-scalar-binary-quantization/ https://jkatz05.com/post/postgres/pgvector-scalar-binary-qua... Hierarchical token pooling: https://www.answer.ai/posts/colbert-pooling.html https://www.answer.ai/posts/colbert-pooling.html And how we implemented them here: https://blog.colivara.com/ https://blog.colivara.com/
- rkuzsma 2y agoWould you be willing to speculate on how VectorChord's ingestion and query performance might compare to Elasticsearch/OpenSearch for dense vector and sparse vector search use cases, particularly when dealing with larger full text data sets (>5M records)?
- VoVAllen 2y agoIn the LAION-5M benchmark, we’ve compared our performance against ElasticSearch and OpenSearch. However, comparing ingestion performance is more challenging due to differences in architecture. Both ElasticSearch and OpenSearch, like most vector databases, use the concept of shards. Each shard represents a separate vector index, and queries aggregate results across these shards. Larger shards lead to faster queries but come with higher resource requirements and slower update speeds. It’s also worth noting that ElasticSearch has implemented RaBitQ support for HNSW. So it's difficult to compare without running actual benchmarks. However, ElasticSearch typically requires at least double, if not triple, the memory size of the vector dataset to maintain system stability. In contrast, PostgreSQL can achieve a stable system with far fewer resources—for example, 32GB of memory is sufficient to manage 100 million vectors efficiently. From my perspective, it would be faster in query comparing to ElasticSearch due to the extensive optimizations. And much much faster with the updates (insert and delete) due to using IVF instead of HNSW.
- estebarb 2y agoIt would have been nice a comparison with pgvectorscale, which uses binary quantization and StreamingDiskANN.
- gaocegege 2y agoWe will look into this. We've tried it before, but it took too long to build the index.
- whakim 2y agoCould you talk about how updates are handled? My understanding is that IVF can struggle if you're doing a lot of inserts/updates after index creation, as the data needs to be incrementally re-clustered (or the entire index needs to be rebuilt) in order to ensure the clusters continue to reflect the shape of your data?
- VoVAllen 2y agoWe don’t perform any reclustering. As you said, users would need to rebuild the index if they want to recluster. However, based on our observations, the speed remains acceptable even with significant data growth. We did a simple experiment using nlist=1 on the GIST dataset, the top-10 retrieval results took less than twice the time compared to using nlist=4096. This is because only the quantized vectors (with a 32x compression) need to be inserted into the posting list, and only quantized vector distances need more computations. And the quantized vector computation only accounts for a small amount of time. Most of the time is spent on re-ranking using full-precision vectors. Let's say the breakdown is approximately 20% for quantized vector computations and 80% for full-precision vector computations. So even if the time for quantized vector computations triples, the overall increase in query time would be only about 40%. If the data distribution shifts, the optimal solution would be to rebuild the index. We believe that HNSW also experiences challenges with data distribution to some extent. However, without rebuilding, our observations suggest that users are more likely to experience slightly longer query times rather than a significant loss in recall.
- dmezzetti 2y agoThis looks like an interesting project. Though it's worth noting that the license is AGPL. So if the idea is for this to take over for pgvecto.rs, it's an important data point for those building SaaS products. It will make pgvector the only permissively licensed option, given it has the same license as Postgres.
- gaocegege 2y agoHi, now we adopted a dual license model (AGPLv3 and ELv2).
- _mmarshall 2y agoThe cost to store a static set of 400k 768-dimension vectors is also $1 a month on Datastax's AstraDB. However, for that $1, AstraDB replicates the data 3x instead of storing it on a single machine. Here is a link to the cost calculator. Note that the calculator includes cost of ingestion, but the article only mentions storage costs, not ingestion costs: https://www.datastax.com/pricing/vector-search?cloudProvider=aws&cloudRegion=us-east-2&nonVectorReadSizeBytes=1024&nonVectorWriteSizeBytes=4096&vectorDimensions=768&vectorReadsPerMonth=0&vectorWritesPerMonth=400000 https://www.datastax.com/pricing/vector-search?cloudProvider... Disclaimer: I work on vectorsearch/AstraDB at DataStax.
- VoVAllen 2y agoIt's hard to compare the cost with the Serverless pricing model, as write and read have extra costs. On the pricing page, datastax costs $4000 to write 100M 768-dim vectors. And 10M query will cost $300, which is only 4 QPS. As comparison, VectorChord can achieve 100 QPS on $250 instance.
- xieydd 2y agoThe pricing estimator is fake, the bill is real : ).
- deleted 2y ago[deleted]
- isogn 2y ago[flagged]