5 ms·

There is a lot done vector search technology right now. I was less fortunate when looking at ways to store the vectors in databases. I already looked at Pinecon

by Kydlaw 5y ago

There is a lot done vector search technology right now. I was less fortunate when looking at ways to store the vectors in databases. I already looked at Pinecone or Weaviate but they are all paid products. Is there some people having feedback on this?

lmeyerov 5y ago
A lot of VCs & founders trying to commercialize other people's SW are wondering this too :) AFAICT: Most of those are basically UIs, data management, & integrations around the same set of vector index libs like FAISS + same set of models like HF, and even the same set of inference server libs (triton, aws/gcp, fastapi, ...). So you'd be evaluating different commercializations of the same core OSS tech. There are useful evaluations to do at that level, but more like licensing, business model, UI, model management, etc. Another commenter below recognized that regular DBs (ES, Postgres, ..) are starting to add vector indexes. As someone doing a lot of architecting for log/event/graph correlation/investigation systems, I've been tracking whether managed neural search db is a feature vs new category, and how big. Ex: those corporate sides definitely feels like an Algolia competitor, but for the 99% case we normally do, maybe just an OSS feature/lib of whatever DB/framework you're already using? Not obvious!
gk1 5y ago
I don’t know when you last looked but as of a few months ago Pinecone has a free tier that fits 1M items with <100ms latency.
kateshao0510 5y ago
Another option is Milvus:https://github.com/milvus-io https://github.com/milvus-io. It's an open-source vector database.
- forgotmyoldacc 5y ago
  Milvus is a layer on top of FAISS.
homarp 5y ago
Have you looked at https://github.com/jina-ai/executor-hnsw-postgres https://github.com/jina-ai/executor-hnsw-postgres ?
ss892714028 5y ago
Milvus Vector database is completely open-source. https://github.com/milvus-io/milvus https://github.com/milvus-io/milvus
andreer 5y ago
There is https://vespa.ai https://vespa.ai which is fully open source and free (but we do run a service if you want one).

by Kydlaw 5y ago

lmeyerov 5y ago

A lot of VCs & founders trying to commercialize other people's SW are wondering this too :) AFAICT: Most of those are basically UIs, data management, & integrations around the same set of vector index libs like FAISS + same set of models like HF, and even the same set of inference server libs (triton, aws/gcp, fastapi, ...). So you'd be evaluating different commercializations of the same core OSS tech. There are useful evaluations to do at that level, but more like licensing, business model, UI, model management, etc. Another commenter below recognized that regular DBs (ES, Postgres, ..) are starting to add vector indexes. As someone doing a lot of architecting for log/event/graph correlation/investigation systems, I've been tracking whether managed neural search db is a feature vs new category, and how big. Ex: those corporate sides definitely feels like an Algolia competitor, but for the 99% case we normally do, maybe just an OSS feature/lib of whatever DB/framework you're already using? Not obvious!

gk1 5y ago

I don’t know when you last looked but as of a few months ago Pinecone has a free tier that fits 1M items with <100ms latency.

kateshao0510 5y ago

Another option is Milvus:https://github.com/milvus-io https://github.com/milvus-io. It's an open-source vector database.

forgotmyoldacc 5y ago
Milvus is a layer on top of FAISS.

homarp 5y ago

Have you looked at https://github.com/jina-ai/executor-hnsw-postgres https://github.com/jina-ai/executor-hnsw-postgres ?

ss892714028 5y ago

Milvus Vector database is completely open-source. https://github.com/milvus-io/milvus https://github.com/milvus-io/milvus

andreer 5y ago

There is https://vespa.ai https://vespa.ai which is fully open source and free (but we do run a service if you want one).