6 ms·

I got really excited about this and then realised it's only for Kubernetes: the one platform I've never believed you should deploy a database to (relational or

by movedx 2y ago

I got really excited about this and then realised it's only for Kubernetes: the one platform I've never believed you should deploy a database to (relational or otherwise.) I guess there are some use cases for such a deployment, but after 20 years of experience across many organisations on three continents, I've never encountered a situation that involves constantly rolling forward the database engine. Bring the engine inline with updates, sure, but weekly? Even monthly? No.

awsanswers 2y ago
Explain why kubernetes isn't a good choice for hosting a relational database.
- sgarland 2y ago
  For small databases (anything with ~< 10,000,000 rows), sure, it's probably fine. Other than that, no. * Unless you are self-hosting K8s and thus have a large amount of control over the underlying storage, the amount of IOPS you're getting will be hazy at best. Tbf this is also true with every single DBaaS, because the latency on network storage is absurd, so IOPS become somewhat meaningless. * Unless you have modified the CPU scheduling options [0] in K8s, you have no control over core pinning or NUMA layout. This is even worse due to the fact that your K8s nodes are probably multi-tenancy. * By its nature, K8s is designed to host stateless apps. It is fantastic at doing this, to be clear. I love K8s. But a system where the nodes can (and should, if you're taking advantage of spot pricing) disappear with a few minutes' warning is not a great host for an RDBMS. * Hot take: it makes provisioning a database even easier, which means people with even less understanding or care of how they operate will be doing so with reckless abandon, which means people like me have even more work to do cleaning up their mess. I am a big fan of gatekeeping things that keep companies afloat. If you want to touch the thing that every service is depending on, learn how it works first – I'd be thrilled to help you. But don't come in and just yolo a copy-paste YAML into prod and then start chucking JSON blobs into it because you can't be bothered to learn proper data modeling, nor reading RDBMS docs. Re: [0], if you don't care about core pinning, then it's unlikely you're going to care about any of these other points, and you probably also don't understand (or care) how blindingly fast an RDBMS on metal with NVMe can be. I am not a Luddite. To reiterate, I have administrated self-hosted and managed K8s professionally. I also run it at home. I just have strong opinions about understanding fundamentals (and not causing myself extra work by allowing people who don't care about them to run infra). [0]: https://kubernetes.io/docs/tasks/administer-cluster/cpu-management-policies/ https://kubernetes.io/docs/tasks/administer-cluster/cpu-mana...
  - achanda358 2y ago
    > By its nature, K8s is designed to host stateless apps. It is fantastic at doing this, to be clear. I love K8s. But a system where the nodes can (and should, if you're taking advantage of spot pricing) disappear with a few minutes' warning is not a great host for an RDBMS. Why is this a problem? A typical deployment will have multiple replicas, with (hopefully) small replication lag. Those should be able to be promoted to be the new primary within a minute.
    - anonzzzies 2y ago
      > typical deployment Ah yes, HN. You know there are billions of sites(wp mostly), LoB apps etc that run on 1 mysql/pg/etc instance right? Replicas are not typical and a tiny minority.
      - movedx 2y ago
        Exactly. Kubernetes and micro services? Sure, for about 0.5% of the industry. Everyone else needs two servers and a load balancer.
        sgarland 2y ago
        Technically four servers, because you’ll want a HA LB as well, but yes. Tech is rife with people who have never set up an old school HA solution proffering advice on how a miasma of cloud services makes theirs better.
      - kgeist 2y ago
        Why would I want k8s for a simple Wordpress site in that case?
        anonzzzies 2y ago
        OP was talking about 'typical database setup' being a replicated db. It's not typical. Nor is the use of k8s for stuff outside HN and massive companies. Not that I mentioned k8s anyway.
eropple 2y ago
I wouldn't have put a database on k8s five years ago, but today the options are a lot better for persistent and resilient storage on k8s and there are pretty significant advantages to being able to normalize your deployment platform. I use CrunchyData's Postgres operator on top of Longhorn volumes for most of my stuff and being able to have what amounts to one-click deployment of multi-host-redundant, automatically backed-up databases is really nice. The Percona Everest operator doesn't look as full-featured for Postgres, which is the only database I typically use, so this isn't for me, but might be a little better if you're a MySQL user (though that's just me guessing).
voidfunc 2y ago
Kubernetes is bad for running databases is a _very_ outdated belief
- sgarland 2y ago
  As a DBRE, I disagree. See post below [0]. [0]: https://news.ycombinator.com/item?id=41414237 https://news.ycombinator.com/item?id=41414237
- otabdeveloper4 2y ago
  Kubernetes is bad for running anything at all, including databases. (K8s is second-system effect and job security for sysadmins. From a technical point of view it causes more problems than it solves.)
  - movedx 2y ago
    I couldn't agree more.
    - jauntywundrkind 2y ago
      And yet you both throw incredibly weak & broad non-technical aspersions. Casting such broad unspecific unnuanced nets, denying value blanketly where clearly some people do find value, is trolling.
      - otabdeveloper4 2y ago
        From a technology point of view k8s doesn't do anything better than what Perl scripts used to do 25 years ago. (Not that Perl scripts are any good. They're crap technology, but unfortunately so is k8s.) K8s doesn't solve a technical problem. It solves two contradictory social problems: a) It gives sysadmins a job creation program, full of expensive and opaque stuff that requires expensive sysadmins. b) It makes sysadmin stuff fungible and replaceable for developers. Solving both problems is probably an important social issue if you're running a Google scale organization. But it's solving a social and organizational problem, not fulfilling a technological need.
dboreham 2y ago
K8s is just a complicated confusing poorly documented thing for running containers. Database server processes running in a container is totally fine. Not really different than running said processes on a bare kernel like it's 2010.
- thanzex 2y ago
  I have to disagree, k8s is extensively documented and the reference docs and APIs are easily accessible. K8s at its core is a customizable, extensible, dynamic API server, with a focus on containerization. It's built with scale and customization in mind, you're not supposed to use it only for running a few containers. I've worked for people that use it to manage VMs with custom controllers. You can change pretty much anything and fit it to your needs. All this with defined, somewhat opinionated sane defaults and conventions
- FridgeSeal 2y ago
  Poorly documented? Surely you jest, the K8s documentation is _excellent_. For basically every core resource type, there’s a user-guide, examples, a tour of it, in-depth docs and then the api docs themselves.
cortesoft 2y ago
How do you patch and update your host machines? You need to roll to a backup host, have it take over as primary, patch and reboot your DB host, and then roll back. I would much rather have an automated process for doing that. Kubernetes provides a framework for allowing that to happen.
- sgarland 2y ago
  With the exception of kernel patches (and even then, tools like ksplice exist), you generally do not have to restart the OS to apply patches. If the DB itself is being patched, then yes, you’ll restart the DB process, but it’s also not difficult to automate failover.
- zsoltkacsandi 2y ago
  I was managing a database cluster that served 30% of the population of my entire country. We rarely needed to restart the hosts or database service (maybe once in every two years), and even if we had the failover process took minutes and it was automated. Without Kubernetes. Nowadays people often think that Kubernetes is the only answer for scalability and automation problems. As the matter of fact we had another product that was running on Kubernetes, they are now migrating off from it, turned out that the old school approach with bare EC2 instances, Ansible, Terraform and Packer is much more reliable, scalable and cost effective than Kubernetes. I have to add that yes, we had to write some custom tools, but it was way less effort than managing a K8S cluster and all the software/operators/controllers that are needed for running your workload on it.
  - Haaammaa 2y ago
    [dead]
  - movedx 2y ago
    > As the matter of fact we had another product that was running on Kubernetes, they are now migrating off from it, turned out that the old school approach with bare EC2 instances, Ansible, Terraform and Packer is much more reliable, scalable and cost effective than Kubernetes. Absolutely it is. I've found this to be the case across a lot of orgs now.
j45 2y ago
Depending on what you're after, the serversideup.net libraries can be pretty handy. They also have a relatively new project out called spin which is a docker swarm orchestrator. Seems to be the real deal. https://serversideup.net/open-source/spin/ https://serversideup.net/open-source/spin/
Haaammaa 2y ago
[dead]

by movedx 2y ago

awsanswers 2y ago

Explain why kubernetes isn't a good choice for hosting a relational database.

sgarland 2y ago
For small databases (anything with ~< 10,000,000 rows), sure, it's probably fine. Other than that, no. * Unless you are self-hosting K8s and thus have a large amount of control over the underlying storage, the amount of IOPS you're getting will be hazy at best. Tbf this is also true with every single DBaaS, because the latency on network storage is absurd, so IOPS become somewhat meaningless. * Unless you have modified the CPU scheduling options [0] in K8s, you have no control over core pinning or NUMA layout. This is even worse due to the fact that your K8s nodes are probably multi-tenancy. * By its nature, K8s is designed to host stateless apps. It is fantastic at doing this, to be clear. I love K8s. But a system where the nodes can (and should, if you're taking advantage of spot pricing) disappear with a few minutes' warning is not a great host for an RDBMS. * Hot take: it makes provisioning a database even easier, which means people with even less understanding or care of how they operate will be doing so with reckless abandon, which means people like me have even more work to do cleaning up their mess. I am a big fan of gatekeeping things that keep companies afloat. If you want to touch the thing that every service is depending on, learn how it works first – I'd be thrilled to help you. But don't come in and just yolo a copy-paste YAML into prod and then start chucking JSON blobs into it because you can't be bothered to learn proper data modeling, nor reading RDBMS docs. Re: [0], if you don't care about core pinning, then it's unlikely you're going to care about any of these other points, and you probably also don't understand (or care) how blindingly fast an RDBMS on metal with NVMe can be. I am not a Luddite. To reiterate, I have administrated self-hosted and managed K8s professionally. I also run it at home. I just have strong opinions about understanding fundamentals (and not causing myself extra work by allowing people who don't care about them to run infra). [0]: https://kubernetes.io/docs/tasks/administer-cluster/cpu-management-policies/ https://kubernetes.io/docs/tasks/administer-cluster/cpu-mana...
- achanda358 2y ago
  > By its nature, K8s is designed to host stateless apps. It is fantastic at doing this, to be clear. I love K8s. But a system where the nodes can (and should, if you're taking advantage of spot pricing) disappear with a few minutes' warning is not a great host for an RDBMS. Why is this a problem? A typical deployment will have multiple replicas, with (hopefully) small replication lag. Those should be able to be promoted to be the new primary within a minute.
  - anonzzzies 2y ago
    > typical deployment Ah yes, HN. You know there are billions of sites(wp mostly), LoB apps etc that run on 1 mysql/pg/etc instance right? Replicas are not typical and a tiny minority.
    - movedx 2y ago
      Exactly. Kubernetes and micro services? Sure, for about 0.5% of the industry. Everyone else needs two servers and a load balancer.
      - sgarland 2y ago
        Technically four servers, because you’ll want a HA LB as well, but yes. Tech is rife with people who have never set up an old school HA solution proffering advice on how a miasma of cloud services makes theirs better.
    - kgeist 2y ago
      Why would I want k8s for a simple Wordpress site in that case?
      - anonzzzies 2y ago
        OP was talking about 'typical database setup' being a replicated db. It's not typical. Nor is the use of k8s for stuff outside HN and massive companies. Not that I mentioned k8s anyway.

eropple 2y ago

I wouldn't have put a database on k8s five years ago, but today the options are a lot better for persistent and resilient storage on k8s and there are pretty significant advantages to being able to normalize your deployment platform. I use CrunchyData's Postgres operator on top of Longhorn volumes for most of my stuff and being able to have what amounts to one-click deployment of multi-host-redundant, automatically backed-up databases is really nice. The Percona Everest operator doesn't look as full-featured for Postgres, which is the only database I typically use, so this isn't for me, but might be a little better if you're a MySQL user (though that's just me guessing).

voidfunc 2y ago

Kubernetes is bad for running databases is a _very_ outdated belief

sgarland 2y ago
As a DBRE, I disagree. See post below [0]. [0]: https://news.ycombinator.com/item?id=41414237 https://news.ycombinator.com/item?id=41414237
otabdeveloper4 2y ago
Kubernetes is bad for running anything at all, including databases. (K8s is second-system effect and job security for sysadmins. From a technical point of view it causes more problems than it solves.)
- movedx 2y ago
  I couldn't agree more.
  - jauntywundrkind 2y ago
    And yet you both throw incredibly weak & broad non-technical aspersions. Casting such broad unspecific unnuanced nets, denying value blanketly where clearly some people do find value, is trolling.
    - otabdeveloper4 2y ago
      From a technology point of view k8s doesn't do anything better than what Perl scripts used to do 25 years ago. (Not that Perl scripts are any good. They're crap technology, but unfortunately so is k8s.) K8s doesn't solve a technical problem. It solves two contradictory social problems: a) It gives sysadmins a job creation program, full of expensive and opaque stuff that requires expensive sysadmins. b) It makes sysadmin stuff fungible and replaceable for developers. Solving both problems is probably an important social issue if you're running a Google scale organization. But it's solving a social and organizational problem, not fulfilling a technological need.

dboreham 2y ago

K8s is just a complicated confusing poorly documented thing for running containers. Database server processes running in a container is totally fine. Not really different than running said processes on a bare kernel like it's 2010.

thanzex 2y ago
I have to disagree, k8s is extensively documented and the reference docs and APIs are easily accessible. K8s at its core is a customizable, extensible, dynamic API server, with a focus on containerization. It's built with scale and customization in mind, you're not supposed to use it only for running a few containers. I've worked for people that use it to manage VMs with custom controllers. You can change pretty much anything and fit it to your needs. All this with defined, somewhat opinionated sane defaults and conventions
FridgeSeal 2y ago
Poorly documented? Surely you jest, the K8s documentation is _excellent_. For basically every core resource type, there’s a user-guide, examples, a tour of it, in-depth docs and then the api docs themselves.

cortesoft 2y ago

How do you patch and update your host machines? You need to roll to a backup host, have it take over as primary, patch and reboot your DB host, and then roll back. I would much rather have an automated process for doing that. Kubernetes provides a framework for allowing that to happen.

sgarland 2y ago
With the exception of kernel patches (and even then, tools like ksplice exist), you generally do not have to restart the OS to apply patches. If the DB itself is being patched, then yes, you’ll restart the DB process, but it’s also not difficult to automate failover.
zsoltkacsandi 2y ago
I was managing a database cluster that served 30% of the population of my entire country. We rarely needed to restart the hosts or database service (maybe once in every two years), and even if we had the failover process took minutes and it was automated. Without Kubernetes. Nowadays people often think that Kubernetes is the only answer for scalability and automation problems. As the matter of fact we had another product that was running on Kubernetes, they are now migrating off from it, turned out that the old school approach with bare EC2 instances, Ansible, Terraform and Packer is much more reliable, scalable and cost effective than Kubernetes. I have to add that yes, we had to write some custom tools, but it was way less effort than managing a K8S cluster and all the software/operators/controllers that are needed for running your workload on it.
- Haaammaa 2y ago
  [dead]
- movedx 2y ago
  > As the matter of fact we had another product that was running on Kubernetes, they are now migrating off from it, turned out that the old school approach with bare EC2 instances, Ansible, Terraform and Packer is much more reliable, scalable and cost effective than Kubernetes. Absolutely it is. I've found this to be the case across a lot of orgs now.

j45 2y ago

Depending on what you're after, the serversideup.net libraries can be pretty handy. They also have a relatively new project out called spin which is a docker swarm orchestrator. Seems to be the real deal. https://serversideup.net/open-source/spin/ https://serversideup.net/open-source/spin/

Haaammaa 2y ago

[dead]