4 ms·
> just that they all eventually get consumed Unless you have some sort of replication or consensus, this is still just best-effort delivery. For example, the m
by Xorlev 11y ago
> just that they all eventually get consumed
Unless you have some sort of replication or consensus, this is still just best-effort delivery. For example, the message could be acknowledged & fsynced followed by drive death. At scale, this would happen.
However, with asynchronous best-effort replication, broker death would be much less likely to lead to a loss of messages.
In general, it's all about playing the odds. Acknowledged writes can still be lost in the best of systems if the entire DC catches fire.
If you tune Kafka settings, you essentially get this behavior. You have to ensure ZK is up, but it will allow all topics to be published to so long as a single broker is alive.
- im_down_w_otp 11y agoIf I tune my Kafka settings I can't get this behavior. The closest I can get is allowing sloppy leader election, but that will definitely, definitely lose writes that have been ACK'd by a stale leader who comes back and has to become a follower. So I can get Kafka to basically accept all writes regardless of level of cluster degradation, but I can't keep it from dropping some of those writes silently onto the floor.