3 ms·

Imagine "day 2" search tasks like the user wanting to filter results for a specific time range You could force the data team to go through hoops like multimoda

Imagine "day 2" search tasks like the user wanting to filter results for a specific time range You could force the data team to go through hoops like multimodal embeddings, multiple indexes, post-filtering... or figure it out Near-term, it makes sense to do one thing well, so many new vector DBs are really mutable vector indexes with lackluster search. But eventually, if you are not a narrowly-scoped kernel (faiss) but the main search platform needing $$$-level marketshare for growing payrolls, you're incentivized to figure search out beyond ANN, and your competitors are advertising that they are doing the same. So you add post-filtering, and then eventually, maybe even push-down/dynamic. Source: We have been happily using on-the-fly GPU knn embeddings for years in pygraphistry, but when we added billion-scale vector indexing to power louie.ai RAG search for real-time news & database querying, we outgrow pure ann on our first user, and I'm thankful we did diligence on our DB selection

sroussey 3y ago
And what did that due diligence reveal?
- lmeyerov 3y ago
  Most prominent for a surprising amount of the vendors we looked at last year (https://gradientflow.com/the-vector-database-index/ https://gradientflow.com/the-vector-database-index/): - We would have paid more for storing billion+ rows in-memory than our gov & enterprise customers were paying for a full product. Since then, ivfpq etc have gotten more popular to do responsive searches for bigger-than-memory datasets. If we needed small in-memory workloads, we could have done faiss+pandas+fastapi and moved on. - Vendors pitched themselves as managed databases yet their users complain about them going down & losing data - Many did not support basic search operators: time filtering, ... - Few/none had real modern security, e.g., row-level multi-tenant ABAC I haven't been tracking on a per-vendor basis as each publicizes making piecemeal progress on these. We found one that did #1-3 without failing us yet, and we're biting the bullet to implement #4. A scalable OSS answer to #4 might be the one that causes us to switch.

sroussey 3y ago

And what did that due diligence reveal?

lmeyerov 3y ago
Most prominent for a surprising amount of the vendors we looked at last year (https://gradientflow.com/the-vector-database-index/ https://gradientflow.com/the-vector-database-index/): - We would have paid more for storing billion+ rows in-memory than our gov & enterprise customers were paying for a full product. Since then, ivfpq etc have gotten more popular to do responsive searches for bigger-than-memory datasets. If we needed small in-memory workloads, we could have done faiss+pandas+fastapi and moved on. - Vendors pitched themselves as managed databases yet their users complain about them going down & losing data - Many did not support basic search operators: time filtering, ... - Few/none had real modern security, e.g., row-level multi-tenant ABAC I haven't been tracking on a per-vendor basis as each publicizes making piecemeal progress on these. We found one that did #1-3 without failing us yet, and we're biting the bullet to implement #4. A scalable OSS answer to #4 might be the one that causes us to switch.