5 ms·

> Stateful services like databases are not really a challenge anymore I've faced so many issues with the CSI driver which doesn't attach/deattach properly when

> Stateful services like databases are not really a challenge anymore I've faced so many issues with the CSI driver which doesn't attach/deattach properly when you modify anything on the Statetfulset. Additionally, you need to pin your DBs ("statefulset workloads") to certain Nodes based on the AZs since a block storage like an EBS cannot really be attached from Zone 1 to Zone 2 automatically. Based on these limitations, I dont really see any utility/usefulness in running a DB on K8s. I get all of the other niceties that K8s offers, but here it makes the Ops work more complex than desired.

arianvanp 5y ago
This is not true. Once your PVC is assigned to a pod the scheduler takes into account the zone information stored in the PVC/PV to make subsequent scheduling decisions if the pod gets restarted. The scheduler will make sure to only start a pod in a place where the PV can get reattached. There used to be bugs with this years ago but all storage providers now get this right.
FridgeSeal 5y ago
Are you on a cloud provider? I’ve not had issues with EBS/AWS, and affinity configs meant it was really easy to spread nodes throughout AZ’s. On Azure though, volume mounts/unmounts were an absolute dumpster fire.
nullify88 5y ago
> Additionally, you need to pin your DBs ("statefulset workloads") to certain Nodes based on the AZs since a block storage like an EBS cannot really be attached from Zone 1 to Zone 2 automatically. Having Postgres replicate the data across AZs or run Ceph on top of EBS to do it at the storage level are options. The complexity of setup is made simpler by running their respective operators.
manigandham 5y ago
This is where complexity comes in. Let Kubernetes abstract the resources for your workloads. For example, use a storage service (like Rook/openEBS/portworx/etc) that uses node-attached disks to create a data layer. Then you can assign volumes to your individual workloads created and managed by the storage service. This provides reliable and portable storage for your other services without having to tying them to lower-level details.

arianvanp 5y ago

This is not true. Once your PVC is assigned to a pod the scheduler takes into account the zone information stored in the PVC/PV to make subsequent scheduling decisions if the pod gets restarted. The scheduler will make sure to only start a pod in a place where the PV can get reattached. There used to be bugs with this years ago but all storage providers now get this right.

FridgeSeal 5y ago

Are you on a cloud provider? I’ve not had issues with EBS/AWS, and affinity configs meant it was really easy to spread nodes throughout AZ’s. On Azure though, volume mounts/unmounts were an absolute dumpster fire.

nullify88 5y ago

> Additionally, you need to pin your DBs ("statefulset workloads") to certain Nodes based on the AZs since a block storage like an EBS cannot really be attached from Zone 1 to Zone 2 automatically. Having Postgres replicate the data across AZs or run Ceph on top of EBS to do it at the storage level are options. The complexity of setup is made simpler by running their respective operators.

manigandham 5y ago

This is where complexity comes in. Let Kubernetes abstract the resources for your workloads. For example, use a storage service (like Rook/openEBS/portworx/etc) that uses node-attached disks to create a data layer. Then you can assign volumes to your individual workloads created and managed by the storage service. This provides reliable and portable storage for your other services without having to tying them to lower-level details.