4 ms·

Here's a quick question - isn't StatefulSets and PersistentVolumeClaim the exact opposite of how fault tolerant database systems are generally built ? The kub

Here's a quick question - isn't StatefulSets and PersistentVolumeClaim the exact opposite of how fault tolerant database systems are generally built ? The kubernetes StatefulSets project is actually trying to build a usecase for Postgresql-on-s3. I think its the wrong abstraction to go after in most usecases for database systems including things like ElasticSearch. I dont think that even Aurora is trying to solve this using persistent storage. You dont really care about solving underlying persistence... you make the whole system fault tolerant : which includes high availability, failover, etc. IMHO kubernetes still has problems in orchestrating an ElasticSearch/Zookeeper cluster for high availability. The problem of discovery, load balancing, etc are bottlenecks.

alexk 10y ago
> Here's a quick question - isn't StatefulSets and PersistentVolumeClaim the exact opposite of how fault tolerant database systems are generally built ? In some cases it is perfectly valid to rely on underlying storage to get HA. E.g. RDS relies on EBS https://aws.amazon.com/message/65648/ https://aws.amazon.com/message/65648/ > IMHO kubernetes still has problems in orchestrating an ElasticSearch/Zookeeper cluster for high availability I don't think that is the case any more. It is not trivial to build a proper automation on top of K8s, but possible, here is an example of a complex distributed DB deployed on top of Kubernetes: https://github.com/coreos/etcd-operator https://github.com/coreos/etcd-operator There are many challenges, but they are not specific to Kubernetes, rather than just automating the deployment of such complex systems is generally hard.
- markonen 10y ago
  As your link illustrates, Multi-AZ is the only real HA option for RDS PostgreSQL, and it doesn't share storage between nodes. So even though RDS uses EBS, you can't really say it is the underlying storage layer that provides RDS's high availability.
- sandGorgon 10y ago
  Thanks for pointing out the Operator framework. Please correct me if I am wrong, but Operator is not a Kubernetes thing - its a custom functionality built by coreos. It is undoubtedly awesome, but it is very hard to get it working on non coreos distro. I personally believe that orchestration primitives like operators, linkerd, ingress, etc are what the k8s project should focus on. Im actually unsure if StatefulSets are solving production problems for anybody. IMHO they are a fairly Googley concept and hard to apply outside of Google for anybody (without solving the networked filesystem problem first).
  - xnxn 10y ago
    Small correction: Operators aren't a CoreOS-specific thing, and they're easy to install on any cluster. Ultimately, the "Operator framework" is just a pattern: you represent some class of software (e.g. etcd, Elasticsearch, PostgreSQL) as a ThirdPartyResource, and you have some controller that monitors those TPRs, driving other resources toward the desired state.
    - sandGorgon 10y ago
      sounds good - my bad then. I think they are awesome and not very well utilized. It might be positioning - Operators vs Helm for example (even though the blog post tries to differentiate)
      - crb 10y ago
        Think of Helm like a Linux package manager. You tell it you want etcd, and it makes it happen for you. The way it makes it happen might be by using an operator, a Deployment/ReplicaSet, or a StatefulSet. Think of operators as a "replica set that is aware of what it is replicating", and can do work to get things in the correct state.

alexk 10y ago

> Here's a quick question - isn't StatefulSets and PersistentVolumeClaim the exact opposite of how fault tolerant database systems are generally built ? In some cases it is perfectly valid to rely on underlying storage to get HA. E.g. RDS relies on EBS https://aws.amazon.com/message/65648/ https://aws.amazon.com/message/65648/ > IMHO kubernetes still has problems in orchestrating an ElasticSearch/Zookeeper cluster for high availability I don't think that is the case any more. It is not trivial to build a proper automation on top of K8s, but possible, here is an example of a complex distributed DB deployed on top of Kubernetes: https://github.com/coreos/etcd-operator https://github.com/coreos/etcd-operator There are many challenges, but they are not specific to Kubernetes, rather than just automating the deployment of such complex systems is generally hard.

markonen 10y ago
As your link illustrates, Multi-AZ is the only real HA option for RDS PostgreSQL, and it doesn't share storage between nodes. So even though RDS uses EBS, you can't really say it is the underlying storage layer that provides RDS's high availability.
sandGorgon 10y ago
Thanks for pointing out the Operator framework. Please correct me if I am wrong, but Operator is not a Kubernetes thing - its a custom functionality built by coreos. It is undoubtedly awesome, but it is very hard to get it working on non coreos distro. I personally believe that orchestration primitives like operators, linkerd, ingress, etc are what the k8s project should focus on. Im actually unsure if StatefulSets are solving production problems for anybody. IMHO they are a fairly Googley concept and hard to apply outside of Google for anybody (without solving the networked filesystem problem first).
- xnxn 10y ago
  Small correction: Operators aren't a CoreOS-specific thing, and they're easy to install on any cluster. Ultimately, the "Operator framework" is just a pattern: you represent some class of software (e.g. etcd, Elasticsearch, PostgreSQL) as a ThirdPartyResource, and you have some controller that monitors those TPRs, driving other resources toward the desired state.
  - sandGorgon 10y ago
    sounds good - my bad then. I think they are awesome and not very well utilized. It might be positioning - Operators vs Helm for example (even though the blog post tries to differentiate)
    - crb 10y ago
      Think of Helm like a Linux package manager. You tell it you want etcd, and it makes it happen for you. The way it makes it happen might be by using an operator, a Deployment/ReplicaSet, or a StatefulSet. Think of operators as a "replica set that is aware of what it is replicating", and can do work to get things in the correct state.