3 ms·

As an architect working on LLM applications I have these criteria for a database. - Full SQL support - Has good tooling around migrations (i.e. dbmate) - Goo

As an architect working on LLM applications I have these criteria for a database. - Full SQL support - Has good tooling around migrations (i.e. dbmate) - Good support for running in Kubernetes or in the cloud - Well understood by operations i.e. backups and scaling - Supports vectors and similarity search. - Well supported client libraries So basically Postgres and PgVector.

ofermend 3y ago
Yes totally agree with that (and other comments below). Moving from a toy example to production deployment requires all the things we are used to having in robust/mature products like postgres.
seanhunter 3y ago
Exactly. The whole point about databases is you don't need "a database for AI" you need a database, ideally with an extension to add additional AI functionality (ie postgres and pgvector). Trying to take a special store you invent for AI and retrofit all the desirable things you need to make it work properly in the context of a real application you're just going to end up with a mess. As a thought-experiment for people who don't understand why you need (for example) regular relational columns alongside vector storage, consider how you would implement RAG for a set of documents where not everyone has permission to view every document. In the pgvector case it's easy - I can add one or more label columns and then when I do my search query filter to only include labels that user has permission to view. Then my vector similarity results will definitely not include anything that violates my access control. Trivial with something like pgvector - basically impossible (afaics) with special-purpose vector stores. Or think about ranking. Say you want to do RAG over a space where you want to prioritise the most recent results, not just pure similarity. Or prioritise on a set of other features somehow (source credibility whatever). Easy to do if you have relational columns, no bueno if you just have a vector store. And that's not to mention the obvious things around ACID, availability, recovery, replication, etc.
tinco 3y ago
Can I add one more nice to have? Good support for graph data. I'm not 100% certain on it yet, but there's a lot of ideas surrounding storing knowledge as a graph out there and it makes a lot of intuitive sense. I haven't found a killer use case for it yet as so far I can get by just tagging things and sql querying on the tags is powerful enough. Maybe someone could pitch in. Is knowledge really a graph (for your problem domain), or is that just some bullshit people made up when they still thought AI could be captured mathematically? It feels to me now knowledge is much more like the way vector embeddings work, it's in a cloud where things are related to each other in an analog or statistical way, not a discrete way. But, perhaps for similar reasons, vector embeddings haven't been super useful to me in building RAG agents yet. Knowledge is either relevant or it's not, and at least for me if it's relevant it has the keywords or tags I need, and just a straight up SQL query brings it in.
- akgfab 3y ago
  You can think of a vector database with n vectors as a network whose adjacency matrix is nxn and each edge is represented by whatever similarity metric between nodes you choose to use. So you can have strongly connected edges and weakly connected edges.
roseway4 3y ago
You may want to take a look at Zep, an LLM application platform that wraps Postgres, pgvector, embedding models, and more to offer chat memory persistence and document vector search. The Python and TS SDKs are designed to support drop-in replacements for the bits of LangChain that don’t scale, but nothing stops you accessing Postgres directly. https://github.com/getzep/zep https://github.com/getzep/zep Disclosure: I’m the primary author.

ofermend 3y ago

Yes totally agree with that (and other comments below). Moving from a toy example to production deployment requires all the things we are used to having in robust/mature products like postgres.

seanhunter 3y ago

Exactly. The whole point about databases is you don't need "a database for AI" you need a database, ideally with an extension to add additional AI functionality (ie postgres and pgvector). Trying to take a special store you invent for AI and retrofit all the desirable things you need to make it work properly in the context of a real application you're just going to end up with a mess. As a thought-experiment for people who don't understand why you need (for example) regular relational columns alongside vector storage, consider how you would implement RAG for a set of documents where not everyone has permission to view every document. In the pgvector case it's easy - I can add one or more label columns and then when I do my search query filter to only include labels that user has permission to view. Then my vector similarity results will definitely not include anything that violates my access control. Trivial with something like pgvector - basically impossible (afaics) with special-purpose vector stores. Or think about ranking. Say you want to do RAG over a space where you want to prioritise the most recent results, not just pure similarity. Or prioritise on a set of other features somehow (source credibility whatever). Easy to do if you have relational columns, no bueno if you just have a vector store. And that's not to mention the obvious things around ACID, availability, recovery, replication, etc.

tinco 3y ago

Can I add one more nice to have? Good support for graph data. I'm not 100% certain on it yet, but there's a lot of ideas surrounding storing knowledge as a graph out there and it makes a lot of intuitive sense. I haven't found a killer use case for it yet as so far I can get by just tagging things and sql querying on the tags is powerful enough. Maybe someone could pitch in. Is knowledge really a graph (for your problem domain), or is that just some bullshit people made up when they still thought AI could be captured mathematically? It feels to me now knowledge is much more like the way vector embeddings work, it's in a cloud where things are related to each other in an analog or statistical way, not a discrete way. But, perhaps for similar reasons, vector embeddings haven't been super useful to me in building RAG agents yet. Knowledge is either relevant or it's not, and at least for me if it's relevant it has the keywords or tags I need, and just a straight up SQL query brings it in.

akgfab 3y ago
You can think of a vector database with n vectors as a network whose adjacency matrix is nxn and each edge is represented by whatever similarity metric between nodes you choose to use. So you can have strongly connected edges and weakly connected edges.

roseway4 3y ago

You may want to take a look at Zep, an LLM application platform that wraps Postgres, pgvector, embedding models, and more to offer chat memory persistence and document vector search. The Python and TS SDKs are designed to support drop-in replacements for the bits of LangChain that don’t scale, but nothing stops you accessing Postgres directly. https://github.com/getzep/zep https://github.com/getzep/zep Disclosure: I’m the primary author.