4 ms·
Isn't using Kubernetes kind of an anti-pattern due to failover and rebalancing logic clashing? If Kubernetes is killing and re-starting nodes and the cluster's
by bubbleRefuge 7y ago
Isn't using Kubernetes kind of an anti-pattern due to failover and rebalancing logic clashing? If Kubernetes is killing and re-starting nodes and the cluster's brokers are detecting dead brokers and rebalancing partitions as a result, it seems counterproductive.
- rhizome 7y agoTo the degree that that conflict exists in their implementation I would think that it's possible to account for all of that.
- addisonj 7y agoThis is one of the main benefits of Pulsar is that because state is split between brokers and bookkeeper and bookkeeper doesn't need re-balanced (due to it's segment based architecture where you choose new bookies with each new segment), we really don't have to worry about re-balancing (in general, not just in case of failover) of storage. It is true that topics map to a single broker, but generally, Pulsar has really good limits on memory so we don't see nodes getting killed by limits and we only really see re-scheduling for real issues. While there certainly is some aspects you need to be aware of, generally, Pulsar is much more "cloud native" and maps quiet well to k8s primitives.
- skube 7y agoUsing kubernetes is always an anti-pattern.
- jupp0r 7y agoIt's a good pattern because it regularly forces you to deal with pods/nodes going away so that the system is designed to handle this well without human intervention. There is no system where nodes don't go away because of hardware errors/updates/decommissions, so you might as well establish it as unexciting routine from the start.