2 ms·
To all the people wondering why "replace battle tested ZK" and how "consensus is hard": its right there under the motivation header: > enabling support for mor
by MrBuddyCasino 7y ago
To all the people wondering why "replace battle tested ZK" and how "consensus is hard": its right there under the motivation header:
> enabling support for more partitions
I don't know if anyone of you ever ran a high-throughput Kafka cluster with a large number of partitions (as in, thousands of them), but its not pretty. Rebalancing can easily take half an hour after a rollout, and throughput is degraded during that time. We recently had to move to shared topics because it became untenable.
This is a very welcome change!