4 ms·

I've actually been looking at Milvus for storing embeddings, why do y'all have options for both Pulsar and Kafka inside Milvus? Seems like unnecessary choices f

• fzliu 4y ago
  The design of Milvus 2.x follows this paper we published a while back: https://arxiv.org/abs/2206.13843 https://arxiv.org/abs/2206.13843. In short, we used Pulsar to implement the write-ahead log, which provides coordination and a single source of truth across all Milvus components. You're right in that it's a bit heavyweight, so we're working to see how we can make pub/sub and other cluster components lighter and more efficient overall.
• politician 4y ago
  Based on some superficial research yesterday, Weaviate looks like an easier option for messing around locally, but Milvus looks better for a production use case.