4 ms·

if you run your stateless services on an orchestration system like k8s, you have two options: a) run your stateful services (such as Postgres) on the same orch

if you run your stateless services on an orchestration system like k8s, you have two options: a) run your stateful services (such as Postgres) on the same orchestration system b) choose a second orchestration system (often a more "traditional" infrastructure provisioning system like Chef/Puppet/Ansible/SaltStack) that will only manage your stateful services, and deal with the care & feeding & mental overhead of having two things that do fairly similar jobs and need to interact with each other (because of course your k8s-hosted stateless services need to be able to look up Postgres in DNS) (option C would be to use a managed service like RDS...but options A and B assume you've already made a decision, for whatever reason, to run the stateful services yourself)

morelisp 5y ago
If you choose a) you don't really get to avoid b), since something still has to bootstrap your Kubernetes nodes themselves. So it's more a question whether you'd rather write a Kubernetes manifest or a Salt/Ansible config; and with the complexity and limitations of a lot of storage operators, and no real "orchestration" benefit once you have pinned deployments to particular nodes anyway, sometimes it's the latter. The main force that's pushing us to move our storage into K8s isn't simplifying our node configuration, but to keep parity between our prod and more dynamic staging environments. For the latter k8s storage operators are great because we don't generally care about data durability or performance.
- cdogl 5y ago
  > If you choose a) you don't really get to avoid b), since something still has to bootstrap your Kubernetes nodes themselves Not quite true if you use a managed K8S offering - and it seems a self-managed cluster is something to be avoided until absolutely necessary. Even with a managed cluster like GKE there's plenty of operational overhead to keep everything up-to-date and working properly.
  - morelisp 5y ago
    If you're running on a managed K8s offering yet not using the managed storage, you're getting ripped off. (I'm well aware this describes a lot of companies / projects.)
goodpoint 5y ago
Aka: k8s is the new hammer and every workload looks like a nail. I'll be downvoted to hell for this.

morelisp 5y ago

If you choose a) you don't really get to avoid b), since something still has to bootstrap your Kubernetes nodes themselves. So it's more a question whether you'd rather write a Kubernetes manifest or a Salt/Ansible config; and with the complexity and limitations of a lot of storage operators, and no real "orchestration" benefit once you have pinned deployments to particular nodes anyway, sometimes it's the latter. The main force that's pushing us to move our storage into K8s isn't simplifying our node configuration, but to keep parity between our prod and more dynamic staging environments. For the latter k8s storage operators are great because we don't generally care about data durability or performance.

cdogl 5y ago
> If you choose a) you don't really get to avoid b), since something still has to bootstrap your Kubernetes nodes themselves Not quite true if you use a managed K8S offering - and it seems a self-managed cluster is something to be avoided until absolutely necessary. Even with a managed cluster like GKE there's plenty of operational overhead to keep everything up-to-date and working properly.
- morelisp 5y ago
  If you're running on a managed K8s offering yet not using the managed storage, you're getting ripped off. (I'm well aware this describes a lot of companies / projects.)

goodpoint 5y ago

Aka: k8s is the new hammer and every workload looks like a nail. I'll be downvoted to hell for this.