6 ms·
From the intro: > we've seen many companies and organizations hop on the vector search bandwagon, from NoSQL database providers such as MongoDB (via Atlas Vect
by jitl 3y ago
From the intro:
> we've seen many companies and organizations hop on the vector search bandwagon, from NoSQL database providers such as MongoDB (via Atlas Vector Search) to traditional relational databases such as Postgres (via pgvector). The general messaging I hear around these vector search plugins is largely the same and goes something like this: developers should stick with us since you can store tables/JSON in addition to vectors, so there is no need to manage multiple pieces of infrastructure!
> This kind of statement always cracks me up, as it's clearly crafted by unsophisticated marketing teams
From the conclusion:
> Once your application starts requiring more complex workloads (such as joins or aggregations), that's when you'll want to contemplate using different data stores.
I don’t know, after reading through the post I’ve come away on the side of the “unsophisticated marketing teams”; if the vector DB doesn’t have transactions, joins, aggregations it sounds like I’d hit its limitations pretty quick and then I’d rather have an all-in-one industrial strength DB like Postgres instead of two systems.
- w1 3y agoPostgres (with pgvector) is an unbelievably goated vector db. Idk why anyone would use anything else.
- Bella-Xiang 3y agoThere are significant differences in performance, accuracy, and other aspects when using a general-purpose database like PostgreSQL. These differences can lead to bottlenecks in performance and data scale. You can refer to this comparison article to learn more https://myscale.com/blog/myscale-vs-postgres-opensearch/ https://myscale.com/blog/myscale-vs-postgres-opensearch/
- brigadier132 3y agoPgVector stores the index in memory right? Do you really want your primary db using all its memory for vectors and all its cpu for doing vector dot products?
- stephen37 3y agoIt's nice to begin with but you usually quickly hit a scale issue from what I tried and talked to different people using it. Also they only offer index stored in Memory as far as I know, also lack the support of different index or more advanced ones like GPU index. https://zilliz.com/comparison/milvus-vs-pgvector https://zilliz.com/comparison/milvus-vs-pgvector
- trosterud 3y agoAnd Datomic is the goat outside of vectors
- manishsharan 3y agoSo you want to run mission critical workloads on Postgres and do a vector similarity search on the same DB?
- jurip 3y ago"More complex workloads such as joins". How far we have fallen.
- esafak 3y agoYou joke, but you would not blame redis for not doing joins. Vector databases are really more like caches that do similarity search. Maybe we should call them vector stores.
- flawsofar 3y agoI can see uses for joining vector queries. Give me all the people who are attracted to people like this one. Boom, embedded.
- esafak 3y ago> Give me all the people who are attracted to people like this one This can be handled with a simple batch query rather than a join, even if you want to query for multiple users, fetching all of their related users. In qdrant, for example: https://qdrant.tech/blog/batch-vector-search-with-qdrant/ https://qdrant.tech/blog/batch-vector-search-with-qdrant/
- Culonavirus 3y agoThis is what gets me more and more with LLMs and their "ability to code". A join is a bedrock of a database. Always has been. Always will be. All LLMs are an absolute shitshow when it comes to any SQL logic involving joins. It's hilariously bad. But what can I expect from an LLM when apparently even people consider a JOIN a "more complex workload"? What a joke.
- sgarland 3y agoEh? SQL is one area where I’ve found LLMs – at least, ChatGPT – to be surprisingly decent. I’ve even had it translate relational algebra back and forth.
- 3y ago
- redwood 3y agoThis "cracks me up" framing is a bit embarrassing to read, maybe they don't realize how they come across