4 ms·
Has anyone seen a convincing argument for why you would want a dedicated vector database in place of a normal database with a good, fast vector index implementa
by simonw 3y ago
Has anyone seen a convincing argument for why you would want a dedicated vector database in place of a normal database with a good, fast vector index implementation?
The existing DB + vector index option seems so obvious to me that I'm worried I'm missing something.
- prabhatjha 3y agoInitially I thought we need a dedicated vector database but as we tried to build even a simple gen-ai applicaitons we realized that we need another "regular" database to build a complete application. Instead with DataStax's Vector Search, we designed a document style API and corresponding clients that give you a Vector Native experience to do CRUD of vector and meta-data as well CRUD of other data models. Here is one client ref for example https://docs.datastax.com/en/astra/astra-db-vector/clients/python.html https://docs.datastax.com/en/astra/astra-db-vector/clients/p...
- brigadier132 3y agowhy do companies use databases like elastic search when they could just use inverted indexes in postgres? Performance, features, and scalability
- simonw 3y agoI use Elasticsearch along with PostgreSQL, but I don't think of Elasticsearch as a database. It's an external index that I denormalize some of my data into.
- tyre 3y agoES is a good parallel to a vector database. run a search against ES => hydrate records from postgres run a search against a vector database => hydrate LLM context from documents Neither is a primary canonical datastore. Postgres is for ES and S3 is for documents.
- riku_iki 3y agoso, its just happened that there is no good implementation of full text search in postgres, but fundamentally nothing prevents from implementing such search, except horizontal scalability if it is needed.
- johnthescott 3y agothe pg extension called "rum" is very performant on millions of pdfs. https://github.com/postgrespro/rum https://github.com/postgrespro/rum
- riku_iki 3y agomy case is tens of billions of short docs, and default search is fine except index creation is single threaded, so it takes multiple days to build index, and github page says rum is even slower..
- aforwardslash 3y agoI love Postgres, but you're comparing apples and oranges. You don't have the same level of query sophistication neither the same indexing quality you get from eg. elasticsearch. As a quick example, the other day I got a bug in PostgreSQL because "searching" does not perform diacritics conversion. Also, stuff like stop words, similarity search and whatnot either required dedicated indexes or are downright "round hole square peg" problems.
- mkarrmann 3y agoI believe they're being rhetorical, and agree that there are good reasons to use elasticsearch over postgres.
- dcastm 3y agoI guess that at a big enough scale it starts making sense. But not sure what “big” really is. Pgvector seems to perform very poorly compared to Qdrant: https://nirantk.com/writing/pgvector-vs-qdrant/ https://nirantk.com/writing/pgvector-vs-qdrant/
- simonw 3y agoI absolutely believe that at the moment - Qdrant is clearly a fantastic piece of technology. But if I was using Qdrant myself I'd treat it like ElasticSearch - I'd denormalize some of my data into it, but I'd still do most of my work in the relational database and treat Qdrant as effectively an external index for my stuff. Maybe I'm getting hung up on the world "database" here as indicating that you use that instead of an RDBMS, when actually everyone selling a vector database expects you to use it as effectively an external indexing mechanism instead.
- zmgsabst 3y agoI personally read it as “vector cache”, because you can use it to, eg, store a phrasing to repeat later.
- swyx 3y agoi wish nirantk would add an addendum noting the pgvector issue was fixed or people like me will have to put up the counter response every single time https://youtu.be/MDxEXKkxf2Q?si=eUhNtghbiLRJ7yHB https://youtu.be/MDxEXKkxf2Q?si=eUhNtghbiLRJ7yHB
- dcastm 3y agoVery interesting. I didn’t know about it. Thanks!
- tveita 3y agoThere are good reasons to split embeddings off from your regular DB load. Embeddings can be quite large (kilobytes per item), and querying is CPU and IO intensive, so they are good candidate for scaling separately. The embeddings are not primary data, they're derived from other content and can be recreated if necessary. The vectors will probably be fed or read in bulk by your ML pipeline at times. I'd say you probably want the primary storage of your embeddings to be file storage like S3 or similar, and then feed them directly into a search engine like Vespa or Elasticsearch.
- icyfox 3y agoThe only production case I've seen of someone switching to a vector database totally in-lieu of a RDBMS is when their raw data lives in parquet / arrow files and they need fast read-only searches in production. Then a periodic indexing daemon with some additional metadata bolted onto the vector definitions worked well and saved them from having to dip their toes into classic database optimization. Every place where I'm deploying vector databases, though, I'm using them as an alternative index just for searches (indexing embedding/id only) and doing the full object value lookup in the RDBMS. It's easier to optimize their vector clustering / sort performance outside the constraints of postgres - which is perhaps why pgvector suffers more under higher load.
- jhj 3y agoThe two main classes of approximate k-NN algorithms om high dimensional data would involve either (1) linear table scans (e.g., IVF / cell-probe indexes; brute-force exact indexing or the coarse quantizer for an IVF index) or (2) graph walking (HNSW, NN-Descent, CAGRA, etc). Some other approaches like LSH (depending on how it's represented as a data structure) can be in between the two. For (1), all of the indexing features of a typical RDBMS are overkill for IVF, where the only indexing you need is by IVF bucket, and then you simply sequentially scan and evaluate distances for everything in those buckets. However, if you want to perform filtering based on other criteria (e.g., filtering based on metadata associated with each vector), then multiple column indexes could be useful with a RDBMS, as you can more efficiently extract just the vectors that meet criteria prior to a sequential scan (e.g., give me all vectors which have a date in the range between X and Y and whose name contains the string "red"). For (2), these algorithms would be inefficient to express in a typical RDBMS. While you can easily encode a graph in a RDBMS, walking the graph sequentially based on criteria is another matter, because each time you'd have to hit the disk (or in-memory cache) to find the next row to look at, then repeat. Latency could be an issue unless it's all in memory, and if it's all in memory, a RDBMS-type representation is overkill versus more simple pointer chasing in memory. I guess it also depends upon latency / throughput needs (caching in memory on front of a database could help, but a pure in-memory vector index would be faster), persistence / fault tolerance / replication needs (in memory databases would need some kind of persistent store), or update needs (adding/removing vectors, though adding/removing vectors from an IVF or graph-based index becomes non-trivial if you add or remove too many vectors). Traditional databbases that have been around for a while might provide better guarantees on reliability. (I'm the author of the GPU half of the Faiss library)
- anonymousDan 3y agoWould a graph database (e.g. Neo4j) be any better?
- toasted-subs 3y agoI cannot think of a situation where neo4j does anything better than traditional relational databases
- dmezzetti 3y agotxtai can use Postgres as a data store and combine that with a vector index like Faiss or Hnswlib. In this case, you get the robustness of Postgres (which has been around a while) and Faiss (which is one of the most mature vector indexes). https://neuml.hashnode.dev/external-database-integration https://neuml.hashnode.dev/external-database-integration
- omneity 3y agotxtai looks cool. Did you use it before in scenarios with all 4 components? I’m curious about which use cases it enables.
- dmezzetti 3y agoIn case it wasn't clear, I'm the main author of txtai. I use txtai in my everyday work. While txtai is open source, I also support a number of consulting projects that are txtai focused. In most cases, I've used SQLite + Faiss for the vector database config paired with an LLM for RAG support. This setup has worked well and it's what I would start with until it didn't scale.
- refulgentis 3y agoNah you nailed it. Beyond the tech even, I don't much understand why there's a cottage industry for it. Either you're touching people's personal documents and should be doing it locally and preferably in-memory, or you're touching people's personal documents and you should own where the data is stored.