2 ms·

I think you are vastly overstating the issue. The issue in the Jepsen post (http://aphyr.com/posts/293-call-me-maybe-kafka http://aphyr.com/posts/293-call-me-ma

I think you are vastly overstating the issue. The issue in the Jepsen post (http://aphyr.com/posts/293-call-me-maybe-kafka http://aphyr.com/posts/293-call-me-maybe-kafka) requires a complex failure case to cause data loss. For most people using Kafka, a little data loss very infrequently, while still accepting writes, is acceptable. The alternative is to refuse all writes till the system is back. This in unacceptable for typical use cases. (See http://blog.empathybox.com/post/62279088548/a-few-notes-on-kafka-and-jepsen http://blog.empathybox.com/post/62279088548/a-few-notes-on-k...)

justinsb 12y ago
It's a trade-off: C vs A. If you can't lose data, Kafka's current configuration is unacceptable. There's an open issue, but it isn't fixed yet, and this means Kafka is (currently) unsuitable e.g. for use as a transaction log. Kafka is great if your data is lower value (e.g. web analytics). It's a pity, because it would be great to have an open source component for the Kafka architecture, but where data-loss is unacceptable. Hopefully this will be fixed in Kafka itself; I've also been working on something that is based around Raft.