3 ms·
I would suggest that anyone trying a real comparison of vector DB's consider the following - necessary functions / use cases (eg prefiltering, dense search
by huac 3y ago
I would suggest that anyone trying a real comparison of vector DB's consider the following
- necessary functions / use cases (eg prefiltering, dense search)
- embeddings version management
- anticipated embedding size (the article only considers glove-100 on ANN-benchmarks, which is quite different from openai-ada-002 1536 - both in terms of their output distribution and the vector size)
- required precision / recall
- required ingestion speed
- required ingestion throughput / time to ingest periodic updates
- required query speed (percentiles, not average!)
- required query throughput
- required RBAC, data privacy, active-active, etc
...and so much more. ANN-benchmarks is a good start for thinking about this but remember that actual throughput is quite different from whatever you see in the algorithms benchmarking!
- sebastiennight 3y agoWouldn't "ease of putting into production" also factor in? For many use cases, being able to put a proof of concept out of the door in hours vs days vs weeks is the top selection criterion if everything else is "good enough".
- huac 3y agodev experience is very very important, but I think it's so subjective -- if you are really pushing the boundaries, maybe you want a super powerful platform which can use all the bells and whistles, but if you want to hack a weekend project together, maybe you just want some API calls.
- antman 3y agoalso filtering and benchmarks including filtering
- huac 3y agoagree! and notably - there's a difference between pre- and post-filtering -- left as an exercise to the reader :)