5 ms·
This article is about why you shouldn't enter the vector database field, and it's reasonable. But I want to comment on another thing I often hear: "You don't n
by lexandstuff 3y ago
This article is about why you shouldn't enter the vector database field, and it's reasonable.
But I want to comment on another thing I often hear: "You don't need a vector database - just use Postgres or Numpy, etc". As someone who moved to Pinecone from a Numpy-based solution, I have to disagree.
Using a hosted vector database is straightforward. Get an API key from Pinecone, send them your vectors, and then query it with new vectors. It's fast, supports metadata filtering, and scales horizontally.
On the other hand, setting up pgvector is a hassle - especially since none of the Cloud vendors support it natively, and a Numpy-based solution, while great for a POC, quickly becomes a hassle when trying to append to it and scale it horizontally.
If you need a vector database, use a vector database. You won't regret it.
- rstocker99 3y agoPostgres RDS has version 0.5 of the pgvector extension installed by default. Adding vector support to our app was as easy as enabling the extension and created a table with vector columns. No additional database required and trivial to do mixed queries. Maybe Pinecone and friends have better scalability, but if you need basic vector support, you can do it easily on RDS. See here for details: https://aws.amazon.com/about-aws/whats-new/2023/10/amazon-rds-postgresql-pgvector-hnsw-indexing/ https://aws.amazon.com/about-aws/whats-new/2023/10/amazon-rd...
- lexandstuff 3y agoFair enough. I researched it earlier in the year and it wasn't available. I stand by my point: we often hear that adding a new tool to your stack is introducing unnecessary complexity, when more often not using the right tool is the path of most complexity.