4 ms·

PetSet co-author here: Think of PetSet as a design pattern for running a cluster on Kubernetes. It should offer a convenient pattern that someone can use to D

PetSet co-author here: Think of PetSet as a design pattern for running a cluster on Kubernetes. It should offer a convenient pattern that someone can use to Do-The-Right-Thing. The hard part of clustering software is configuration change management. If an admin adds a new member to the cluster by hand, they're usually correct (the human "understands" that the change is safe because no partition is currently going on). But an admin can also induce split brain. PetSets on Kube try to make configuration changes predictable by leveraging their own strong consistency (backed by the etcd quorum underneath the Kube masters) to tell each member the correct configuration at a point in time. The PetSet goal is to allow relative experts in a particular technology to make a set of safe decisions for their particular clustered software that behaves exactly as they would predict, so that they can make a set of recommendations about how to deploy on the platform. For instance, CoreOS is working to provide a set of recommendations for etcd on pet sets that would be correct for anyone who wants to run a cluster safely, and encode that into tutorials / docs / actual pet definitions. They're also driving requirements. Other folks have been doing that for elastic search, zookeeper, galera, cassandra, etc. PetSets won't be "done" until it's possible to take the real world considerations into account safely when writing one.

jat850 10y ago
That's really valuable insight, thank you. The biggest struggle so far has been with zookeeper and Kafka. Things that are stateful have posed a lot of difficulty - from a mental perspective more than fighting against Kubernetes specifically, just trying to think and adhere more to microservices principles. I'm following PetSets very closely and I think they're going to help a great deal. Is it accurate that PetSets were introduced to compensate for the type of thing I've singled out? That certain models just don't quite fit with the "everything is equal and interchangeable" notion? Or does it just feel that way? I don't want to end up missing the point of PetSets or using them for something they're not truly intended for, or leaning on them only to hit a stumbling block.
- smarterclayton 10y ago
  Yes. PetSets are intended to be step one of "some things need to be stable and replaceable at the same time". So that unit can then be composed into apps (i.e. Zookeeper + Kafka might be a PetSet for ZK and multiple scale groups for Kafka). basically trying to boil the problem space down into "what do cluster aware software need to be safe", like protection from split brain and predictable ordering. There is likely to be more needed (on top or around) because we need to also solve problems for highly available "leaders" and active/passive setups. Think of what something like pacemaker does to ensure you can run two Linux systems in an HA fashion, and then map the necessary parts to Kube (fencing, ip failover, cloning, failure detection). Still a lot of hard problems to solve - PetSets are the first building block.

jat850 10y ago

That's really valuable insight, thank you. The biggest struggle so far has been with zookeeper and Kafka. Things that are stateful have posed a lot of difficulty - from a mental perspective more than fighting against Kubernetes specifically, just trying to think and adhere more to microservices principles. I'm following PetSets very closely and I think they're going to help a great deal. Is it accurate that PetSets were introduced to compensate for the type of thing I've singled out? That certain models just don't quite fit with the "everything is equal and interchangeable" notion? Or does it just feel that way? I don't want to end up missing the point of PetSets or using them for something they're not truly intended for, or leaning on them only to hit a stumbling block.

smarterclayton 10y ago
Yes. PetSets are intended to be step one of "some things need to be stable and replaceable at the same time". So that unit can then be composed into apps (i.e. Zookeeper + Kafka might be a PetSet for ZK and multiple scale groups for Kafka). basically trying to boil the problem space down into "what do cluster aware software need to be safe", like protection from split brain and predictable ordering. There is likely to be more needed (on top or around) because we need to also solve problems for highly available "leaders" and active/passive setups. Think of what something like pacemaker does to ensure you can run two Linux systems in an HA fashion, and then map the necessary parts to Kube (fencing, ip failover, cloning, failure detection). Still a lot of hard problems to solve - PetSets are the first building block.