3 ms·

Great article. Vector databases are still somewhat niche, but there's been rapid growth in interest (I'm a part of the Milvus community). https://db-engines.com

by fzliu 4y ago

Great article. Vector databases are still somewhat niche, but there's been rapid growth in interest (I'm a part of the Milvus community). https://db-engines.com https://db-engines.com also provides a holistic popularity metric (Milvus and Weaviate are on there), but I'm not exactly sure how it's calculated. One quick note - FAISS is an ANN library rather than a vector database. Vector databases support a multitude of other features that you'd see in a traditional database such as caching, replication, horizontal scalability, etc. I've also found Elastic's ANN search functionality a bit on the slow side, likely due to its original architecture being focused on more general text search.

thirdtrigger 4y ago
Agreed with your observations about the difference between the libraries and databases. Maybe for a follow-up article? It's nice to see some independent research on the topic, tho.
- lmeyerov 4y ago
  (one of the authors) Yes! We carefully distinguish vector search vs vector databases throughout the article. The end sections on architectures touch on that, such as questioning compute-tier solution vs not. Reality vs marketing seem to diverge here. On the compute side, Faiss and some others have scaling extensions (GPU, bigger-than-memory, ...) that both out-perform and out-scale some of the vector database solutions. Likewise, similar to redis, teams will use it in a rather persistent way in practice. Conversely, some vector db teams market traditional db notions like CRUD operations, but it is unclear how much that is their ideas talking, the need to meet VC funding stories, niche use cases, or what managed vector search users want. Likewise, if we did a comprehensive benchmark of vector databases vs say graph databases, I'm not sure which would win in many key cases, including on those vector databases advertise on. It's an interesting time!

by fzliu 4y ago

thirdtrigger 4y ago

Agreed with your observations about the difference between the libraries and databases. Maybe for a follow-up article? It's nice to see some independent research on the topic, tho.

lmeyerov 4y ago
(one of the authors) Yes! We carefully distinguish vector search vs vector databases throughout the article. The end sections on architectures touch on that, such as questioning compute-tier solution vs not. Reality vs marketing seem to diverge here. On the compute side, Faiss and some others have scaling extensions (GPU, bigger-than-memory, ...) that both out-perform and out-scale some of the vector database solutions. Likewise, similar to redis, teams will use it in a rather persistent way in practice. Conversely, some vector db teams market traditional db notions like CRUD operations, but it is unclear how much that is their ideas talking, the need to meet VC funding stories, niche use cases, or what managed vector search users want. Likewise, if we did a comprehensive benchmark of vector databases vs say graph databases, I'm not sure which would win in many key cases, including on those vector databases advertise on. It's an interesting time!