3 ms·

I've never been a fan of deploying DBs, either traditional RDBMS or distributed, under containerization/orchestration other than quickly spinning up dev and tes

I've never been a fan of deploying DBs, either traditional RDBMS or distributed, under containerization/orchestration other than quickly spinning up dev and test environments. Certainly not for production. Databases have been built for high availability for many decades. Sure, maybe RDBMS doesn't scale quite as easily as a node API application, but it's still not really rocket science and you usually have some leeway if you get a spike. Adding Swarm or K8 just seems like redundant and unnecessary complexity.

steveBK123 3y ago
Wouldn't most data driven, disk-IO intensive workloads be a bad fit for k8s? Especially when its not purely read-only.. That seems the crux of it.
- snapetom 3y ago
  Absolutely. I haven't gotten in the weeds of K8 volumes, but certainly have done a lot with Docker volumes including adding some PRs to fix some issues I've ran into. There is a lot under the hood with Docker volumes. With how DBs like to optimize their IO operations, I'm frankly a little amazed how it somehow mostly works between the DB -> orchestrator volumes -> file system.
  - dilyevsky 3y ago
    It works totally fine on k8s with iscsi/fcoe/nvme-backed volumes provided remote targets themselves offer sufficient performance for your db workload
  - p_l 3y ago
    K8s mostly drops docker volumes code (and many other parts of docker, TBH). You can pass a random mount point or even a raw device and it will have only the overhead of kernel IO code for namespaces, overhead you hit anyway. The only "docker graph" stuff is in OCI container data itself.
- cwiggs 3y ago
  This seems to be the common advise given, but I don't fully agree. There have been many times in my career where a DB was on a VM with the storage attached via a cloud providers block storage. When asked if we should move it to k8s, people are quick to mention k8s doesn't do well with persistent storage. However all of the big cloud providers offer the ability to easily create a persistent volume in k8s that then just creates a block device, attaches it to the k8s host and makes it available to the pod. So in both situations you have the same IO limits of block storage. The question is does k8s persistent volume api add enough of an IO bottleneck to cause issues, IME that isn't the case. Now if you want direct attached NVMe drives for higher IO than a network attached block storage will give, then it might be easier with a VM vs k8s, but I can't speak to that much.
  - p_l 3y ago
    Given k8s origin as bare-metal oriented system, attaching physical SAN volumes was there pretty early on, and now it's only more capable in that area (including passing devices for running exotic stuff)

steveBK123 3y ago

Wouldn't most data driven, disk-IO intensive workloads be a bad fit for k8s? Especially when its not purely read-only.. That seems the crux of it.

snapetom 3y ago
Absolutely. I haven't gotten in the weeds of K8 volumes, but certainly have done a lot with Docker volumes including adding some PRs to fix some issues I've ran into. There is a lot under the hood with Docker volumes. With how DBs like to optimize their IO operations, I'm frankly a little amazed how it somehow mostly works between the DB -> orchestrator volumes -> file system.
- dilyevsky 3y ago
  It works totally fine on k8s with iscsi/fcoe/nvme-backed volumes provided remote targets themselves offer sufficient performance for your db workload
- p_l 3y ago
  K8s mostly drops docker volumes code (and many other parts of docker, TBH). You can pass a random mount point or even a raw device and it will have only the overhead of kernel IO code for namespaces, overhead you hit anyway. The only "docker graph" stuff is in OCI container data itself.
cwiggs 3y ago
This seems to be the common advise given, but I don't fully agree. There have been many times in my career where a DB was on a VM with the storage attached via a cloud providers block storage. When asked if we should move it to k8s, people are quick to mention k8s doesn't do well with persistent storage. However all of the big cloud providers offer the ability to easily create a persistent volume in k8s that then just creates a block device, attaches it to the k8s host and makes it available to the pod. So in both situations you have the same IO limits of block storage. The question is does k8s persistent volume api add enough of an IO bottleneck to cause issues, IME that isn't the case. Now if you want direct attached NVMe drives for higher IO than a network attached block storage will give, then it might be easier with a VM vs k8s, but I can't speak to that much.
- p_l 3y ago
  Given k8s origin as bare-metal oriented system, attaching physical SAN volumes was there pretty early on, and now it's only more capable in that area (including passing devices for running exotic stuff)