5 ms·

First, please consider whether you actually need any of the Kubernetes stuff. The odds are that you don't. A huge number of people are switching to k8s just bec

First, please consider whether you actually need any of the Kubernetes stuff. The odds are that you don't. A huge number of people are switching to k8s just because it's the cool thing to do, without understanding any of the implications. Most companies have to undergo major software rearchitectures and renovations to make good use of the featureset that Kubernetes promises. Second, please don't run a database in it, omg. See point one. If there were just one application that is not reasonable to run in a container, it would be a database. Third, yes, this seems to be the new way things happen in the software world. I'm worried about what tools we'll need to develop to overcome the cacophony of noise and half-solutions that is incumbent in the GitHub era.

cookiecaper 9y ago
Here, let me pre-emptively render the standard replies, to save everyone time: * "Nuh-uh! We love Kubernetes and run EVERYTHING in it, so there. And we DID NOT do that just so we could be cool by copying Google or just so I could pad my resume, I triple-double-swear". * "But, StatefulSets! It doesn't matter that it's taken over two years of engineering effort to get to a place where fundamental functionality that you've gotten automatically for the last 3 decades exists, like individually-addressable servers or stable disk space. It doesn't matter that they're still beta. Run your database in it TODAY, or you are in serious danger of exhausting your GCool Points!" * "Please take back your mean comments about modern-day software development culture. Node.js was my first language and I for one find left-pad a miraculous innovation, which never would've been possible without a culture of everyone scrambling to copy some crappy module up to npm and their github profile so that they can brag about how many stars they have. I am thoroughly enjoying this same ethos being carried over into sysadmin/devops."
- eropple 9y ago
  Every time you write about the ongoing silliness that is chasing the car that is infrastructure/platform trends, I want to buy you a drink. The overwhelming proportion of teams neither need nor benefit from all of this stuff. We're all not running Google here--at some point, the mindset of an engineer should go from "this looks fun" to "this isn't going to blow up and screw us at the worst possible time", but unfortunately, it looks like the devops/infrastructure world is currently traipsing down the road tread by other trendy ecosystems and fields. So many developers insist that elegance is having nothing to take away--and then they pile all of this stuff that they don't fully understand on top of other stuff they don't understand at all just because "it's devops". It's maddening and strange. (While we're at it: most of you don't need your million microservices, either, and you're probably not fault-tolerant across them anyway.)
  - cookiecaper 9y ago
    Indeed. The scary part is that eventually the consequences of this will propagate up to someone important and powerful enough to do something about it, and then I worry that there will be legal limitations put on the industry. I believe there are already substantial rumblings in this direction given the current state of cybersecurity, so this new age of recklessness only stands to hasten it. I am really hoping that we can avoid such strenuous regulations, lest highly functional autodidacts and anti-authoritarians find themselves shut out of yet another high-paying, at-least-semi-merit-based career. The evisceration of the legal profession should stand as a warning to us. We do not want the actually good and useful people, who generally have low tolerance for the systemic brutality of rote learning, edged out by status seekers who have no real value to offer but can occupy space in a room for the requisite time to earn their certifications. And on the note about microservices, you couldn't be more right. These architectures are such nightmares to debug that special transaction tracing tooling had to be invented to ease the process, and you have to either go through and update all 90 of your services to use these or go through a centralized service broker (now called a "microservice mesh" so that people don't catch on) to append tracing tags to each request. Please get real here, people. There are times when this kind of stuff may be useful (if you're running Google or Twitter, for example). That does NOT apply to your small company of 100 employees. You are only making things harder on yourselves.
    - nineteen999 9y ago
      Thank goodness there a people that get this. I ran screaming from a devops role I'd been in for two years around 6 months ago. This thought process of "complexity == good", "old == bad" - watching the company starting to pull apart an already unstable product and break it down into microservices when they could't even document their existing codebase, how to bootstrap a new instance. Most puppet installations I've seen run by these small teams just degenerate into snowflake factories. Going to all that effort to run a huge packer pipeline so your templates are the same accross all your clouds providers - and then pulling whatever-the-current-random-version-of-a-bunch-of gems from rubygems.org onto hundreds of different machines. Arggggh! The argument I keep hearing is that "if we don't use all the latest tools then how are we going to attract the best talent". Every few years I dip my toes back into the devops space, just to see if it's improving. Nope. It's just getting worse.
eicnix 9y ago
Why shouldn't you run databases on it? Ive been running larger databases(10tb+) on it for some time. If your database can deal with failure of single node and uses network storage it's totally doable to run databases on Kubernetes. Take a look at Patroni on how to run Postgres on IaaS or Kubernetes.
- eropple 9y ago
  You shouldn't run databases on it unless you can express exactly why you should. Me, I already have resource segmentation throughout my infrastructure. They have their own network stacks and predictable, understandable resource utilization. I call those segments "virtual machines." It's rather harder to blog about an RDS instance (or even a self-hosted EC2 one), but you minimize moving parts and the way your bleeding-edge systems can screw you.
  - eicnix 9y ago
    My main reason to run basically everything in Kubernetes is abstraction between resources and applications and standardization of deployments. While you can achieve the same with various other ways like good configuration management and virtual machines I have found that the overall cost of creating/running and monitoring deployments on Kubernetes is smaller than on a traditional VM based solutions.
- cookiecaper 9y ago
  Database servers are generally designed from the ground-up to utilize basically the entire machine and to be left running for a long time, with caching and memory utilization techniques designed around these assumptions. VMs muck with this somewhat but at least the memory region is reserved for that VM's usage. Docker eliminates that, and k8s eliminates the knowledge even of what type of hardware/memory the system is being executed on (without taking several contrived steps to restore this). Database servers are also designed on the assumption that each node will be available on a consistent basis, at a consistent address, and that the data directory will be available to the server as soon as it starts up (except for rare node bootstrap operations). This is the exact opposite of what Kubernetes/Docker seek to provide, and in fact, such things can only be provided within Kubernetes by extensive special configuration that leans heavily on experimental features. There could not be a worse ideological match. Yes, you can try really hard to ram that square peg through that round hole, eventually pushing it through with significant damage to both the peg and the hole, but why would you?
specialp 9y ago
It is good to have healthy skepticism for new things. But I don't think k8s is just the "cool thing to do". Here's some reasons why running an enterprise on it is better than on bare VM metal: 1. Containers are a better application level abstraction than VMs. Why as someone deploying an app should I care about machine details. I want my application to be deployed n replicas with x resources and to be discoverable. 2. Containers make your application self documenting on its dependencies. Yes it is some work to get your stuff containerized but we haven't come across anything yet that was all that difficult. 3. Running a container on its own on bare Docker does not give you discovery, multiple nodes, restart policies or many other things k8s does. 4. Much of dev ops roll your own that is going on is solving problems that nearly everyone has and why not do it in a proven environment that does nearly all of the work for you. I don't think at all that putting apps in containers, shipping them with a resource request/service descriptor, and then exposing them via service discovery is going anywhere at all. Whether it is k8s and Docker or something else, once you are using it, it is clear that this is really the abstraction deployments should be at. I'd say the odds are that you do need k8s stuff for any enterprise/business deployment of services. I'd be surprised if within a few years people are doing it any way else.
- cookiecaper 9y ago
  1. This is called "shared hosting". I sympathize with that desire. k8s is a way to roll your own shared host. Disinterested application developers who resent having to look at a command line do really like k8s because they treat it like a shared host platform, but they fail to appreciate the middle layer of complication that it entails, or the stickier elements of its architecture. A lot of ignorant app developers don't realize that you don't really get most of k8s's promises unless your application is already specifically designed to handle a transient, disappear-at-a-moment's-notice lifecycle, and most existing software is not built around these assumptions. It can't all be shoehorned in at the infrastructure level. Database servers are a fine example of an application architecture that is antithetical to this. 2. This is called "static linking". I also sympathize with static linking, but again this comes with implications that are not well understood by many of its advocates. In 4 years, I expect that some fad will emerge to repopularize dynamic linking under some new name, like "securable containerized orchestrated hypothalmized applicationators". Everyone will say "Dude, I can't imagine nobody using securable containerized orchestrated hypothamlmized applicatorness in 4 years". Rinse and repeat. 3. Yes, Docker sucks. We agree. 4. Sure, but you don't need to roll your own. Much of trendy devops is a reinvention of old things that we've already had and tried over the last several decades. For the most part, these come with the same old problems, and there are reasons they fell out of favor. There are definitely times when the tradeoffs will fall such that Kubernetes makes sense. Those times are much rarer than most people who are running k8s want to admit. That's because they're subconsciously running k8s as a status symbol, and lying to themselves about its necessity and value in their use case. I'm not saying the way we were doing things before k8s came around was exactly right. For example, going back to shared hosting is probably not a bad idea for the vast majority of things, and stuff like logging into your FTP server and copying your files have been gussied up as "cloud object storage" (and costs a lot more money now). So it's really just a general oscillation of trends, I guess.
dang 9y ago
You've got some good points about this but you're bringing it up repeatedly on HN in ways that are getting tedious (i.e. people have complained to us, and if one is complaining, many are bored). You're also being pretty cavalier with the flamebait (e.g. https://news.ycombinator.com/item?id=14453780 https://news.ycombinator.com/item?id=14453780). We're trying to avoid that sort of argument on HN, so could you please not? Yes engineers chase shiny new things and argue about it defensively. That's real, but it's also perennial—all us curmudgeons did the same thing in our day. This is cycle-of-life stuff. Such discussions quickly converge to the same old thing unless we're careful to avoid that, and the burden of avoiding it falls more on the senior members. We detached this subthread from https://news.ycombinator.com/item?id=14453544 https://news.ycombinator.com/item?id=14453544 and marked it off-topic.
- cookiecaper 9y ago
  I appreciate the explanation, and as multiple obviously-irritated k8s contributors have come in to counteract my other comments, I admit I have been curious how long HN would be willing to withstand the pressure from beings so highly-regarded and attached to such prestigious institutions. Surely, it is not wise to irritate powerful persons, especially not with semi-credible lambasts that affect an important centerpiece of their corporate strategy (k8s is part of the strategy to transition people to Google Cloud as opposed to AWS). This is also good supporting evidence for the amount of SV heterodoxy that HN-the-institution is willing to stomach (that topic was broached last time we crossed paths). Are perennial, cycle-of-life topics not expected to regularly appear on discussion platforms? I find it grating to see "k8s this, k8s that" all the time. Should I register a complaint as well, so that these threads can be killed next time they come up, on the theory that "many are bored" by the topic, based on the existence of my complaint? Specifically, which things are not allowed to be mentioned in the future? Was it: point 1, "Please consider whether you need k8s before you use it"; point 2, "Please don't run a database inside Kubernetes"; or point 3, "Yes, the software world is hardly manageable due to the unintended effects exposed by GitHub and friends"? Which of these is too tedious and boring to express on HN? I definitely would understand if only the flamebait comment were off-topic'd, as I know that was close to the edge. I'm more worried by the fact that counterpoints that are apparently "boring" or "tedious" justify censorship.
lobster_johnson 9y ago
Databases run fine in containers; conceptually there's very little difference between that and running outside of one. The burden is on yourself to know how to manage it. Specifically, Kubernetes provides the tools to "pin down" the database to the extent you want/need: Stateful sets, node affinities, QoS classes, nodepools and so on. You want to avoid OOMkilling the database, you want to dedicate the appropriate amount of resources to it, you may want to dedicate a specific type of hardware to it, etc. It's not for beginners, but nor is it anything like the disaster you portray it as.
- cookiecaper 9y ago
  This is true to the extent that containers remain in the realm of "concept" rather than reality. In the theoretical container that only serves the function of isolating each process within its own resource space, there is little reason why databases can't function reasonably well. This element alone is more frequently referred to as "sandboxing". However, in the real world of Docker and Kubernetes, it abstracts away components that actually matter to the function and administration of the database, and which can only be simulated by relying on experimental features like StatefulSets, or flaky features like node affinities that require each node to be registered with the correct labels and, as far as I know, are treated as preferences, not demands (i.e., pods will be scheduled on other nodes if nodes matching the affinity are not available). It is for beginners because non-beginners would realize that they're just taking a very complicated and winding route back to step one. If you want to avoid all OOM kills triggered by exceeding the cgroup spec, if you want to avoid the complexity and potential risk involved in configuring sturdy backing volumes and dealing with the questions around what happens when a node gets rescheduled onto a node that doesn't have that volume, if you want to ensure that the database is running reliably and stably on a specific type of hardware, the right solution is to go buy a server (or at the very least use a dedicated VM, not a container that will thrash it all over the place and expose the process to the risk of things like a dockerd hang), install the database on it, and leave it alone. Not to try to shoehorn it into k8s and try to grab this stuff that k8s exists to hide out of some weak abstract space. The risks associated far outweigh the benefits of Kubernetes for an application like a database (and arguably many other types of applications). As far as I know, databases are supposed to be administered with a great deal of respect for their rock-solid stability. The very suggestion of running one on a two-year-old platform should make any business very scared, even if that platform was conventional. k8s is anything but, and it is still trying to figure out how to provide the basics. Running a production database in k8s is a shockingly distasteful affront to stability just in principle. It sounds like the only reason people are able to give to justify database-in-k8s is "I want to be able to control it with `kubectl` like I can control everything else". My personal impulse would be to say that people who insist on using one administrative utility only should probably not be allowed to operate servers. A more realistic solution may be something like a kubectl proxy mode, where one can register external servers within the cluster namespace and use `kubectl exec <my-db-server> /bin/bash` to initiate an SSH connection.

cookiecaper 9y ago

Here, let me pre-emptively render the standard replies, to save everyone time: * "Nuh-uh! We love Kubernetes and run EVERYTHING in it, so there. And we DID NOT do that just so we could be cool by copying Google or just so I could pad my resume, I triple-double-swear". * "But, StatefulSets! It doesn't matter that it's taken over two years of engineering effort to get to a place where fundamental functionality that you've gotten automatically for the last 3 decades exists, like individually-addressable servers or stable disk space. It doesn't matter that they're still beta. Run your database in it TODAY, or you are in serious danger of exhausting your GCool Points!" * "Please take back your mean comments about modern-day software development culture. Node.js was my first language and I for one find left-pad a miraculous innovation, which never would've been possible without a culture of everyone scrambling to copy some crappy module up to npm and their github profile so that they can brag about how many stars they have. I am thoroughly enjoying this same ethos being carried over into sysadmin/devops."

eropple 9y ago
Every time you write about the ongoing silliness that is chasing the car that is infrastructure/platform trends, I want to buy you a drink. The overwhelming proportion of teams neither need nor benefit from all of this stuff. We're all not running Google here--at some point, the mindset of an engineer should go from "this looks fun" to "this isn't going to blow up and screw us at the worst possible time", but unfortunately, it looks like the devops/infrastructure world is currently traipsing down the road tread by other trendy ecosystems and fields. So many developers insist that elegance is having nothing to take away--and then they pile all of this stuff that they don't fully understand on top of other stuff they don't understand at all just because "it's devops". It's maddening and strange. (While we're at it: most of you don't need your million microservices, either, and you're probably not fault-tolerant across them anyway.)
- cookiecaper 9y ago
  Indeed. The scary part is that eventually the consequences of this will propagate up to someone important and powerful enough to do something about it, and then I worry that there will be legal limitations put on the industry. I believe there are already substantial rumblings in this direction given the current state of cybersecurity, so this new age of recklessness only stands to hasten it. I am really hoping that we can avoid such strenuous regulations, lest highly functional autodidacts and anti-authoritarians find themselves shut out of yet another high-paying, at-least-semi-merit-based career. The evisceration of the legal profession should stand as a warning to us. We do not want the actually good and useful people, who generally have low tolerance for the systemic brutality of rote learning, edged out by status seekers who have no real value to offer but can occupy space in a room for the requisite time to earn their certifications. And on the note about microservices, you couldn't be more right. These architectures are such nightmares to debug that special transaction tracing tooling had to be invented to ease the process, and you have to either go through and update all 90 of your services to use these or go through a centralized service broker (now called a "microservice mesh" so that people don't catch on) to append tracing tags to each request. Please get real here, people. There are times when this kind of stuff may be useful (if you're running Google or Twitter, for example). That does NOT apply to your small company of 100 employees. You are only making things harder on yourselves.
  - nineteen999 9y ago
    Thank goodness there a people that get this. I ran screaming from a devops role I'd been in for two years around 6 months ago. This thought process of "complexity == good", "old == bad" - watching the company starting to pull apart an already unstable product and break it down into microservices when they could't even document their existing codebase, how to bootstrap a new instance. Most puppet installations I've seen run by these small teams just degenerate into snowflake factories. Going to all that effort to run a huge packer pipeline so your templates are the same accross all your clouds providers - and then pulling whatever-the-current-random-version-of-a-bunch-of gems from rubygems.org onto hundreds of different machines. Arggggh! The argument I keep hearing is that "if we don't use all the latest tools then how are we going to attract the best talent". Every few years I dip my toes back into the devops space, just to see if it's improving. Nope. It's just getting worse.

eicnix 9y ago

Why shouldn't you run databases on it? Ive been running larger databases(10tb+) on it for some time. If your database can deal with failure of single node and uses network storage it's totally doable to run databases on Kubernetes. Take a look at Patroni on how to run Postgres on IaaS or Kubernetes.

eropple 9y ago
You shouldn't run databases on it unless you can express exactly why you should. Me, I already have resource segmentation throughout my infrastructure. They have their own network stacks and predictable, understandable resource utilization. I call those segments "virtual machines." It's rather harder to blog about an RDS instance (or even a self-hosted EC2 one), but you minimize moving parts and the way your bleeding-edge systems can screw you.
- eicnix 9y ago
  My main reason to run basically everything in Kubernetes is abstraction between resources and applications and standardization of deployments. While you can achieve the same with various other ways like good configuration management and virtual machines I have found that the overall cost of creating/running and monitoring deployments on Kubernetes is smaller than on a traditional VM based solutions.
cookiecaper 9y ago
Database servers are generally designed from the ground-up to utilize basically the entire machine and to be left running for a long time, with caching and memory utilization techniques designed around these assumptions. VMs muck with this somewhat but at least the memory region is reserved for that VM's usage. Docker eliminates that, and k8s eliminates the knowledge even of what type of hardware/memory the system is being executed on (without taking several contrived steps to restore this). Database servers are also designed on the assumption that each node will be available on a consistent basis, at a consistent address, and that the data directory will be available to the server as soon as it starts up (except for rare node bootstrap operations). This is the exact opposite of what Kubernetes/Docker seek to provide, and in fact, such things can only be provided within Kubernetes by extensive special configuration that leans heavily on experimental features. There could not be a worse ideological match. Yes, you can try really hard to ram that square peg through that round hole, eventually pushing it through with significant damage to both the peg and the hole, but why would you?

specialp 9y ago

It is good to have healthy skepticism for new things. But I don't think k8s is just the "cool thing to do". Here's some reasons why running an enterprise on it is better than on bare VM metal: 1. Containers are a better application level abstraction than VMs. Why as someone deploying an app should I care about machine details. I want my application to be deployed n replicas with x resources and to be discoverable. 2. Containers make your application self documenting on its dependencies. Yes it is some work to get your stuff containerized but we haven't come across anything yet that was all that difficult. 3. Running a container on its own on bare Docker does not give you discovery, multiple nodes, restart policies or many other things k8s does. 4. Much of dev ops roll your own that is going on is solving problems that nearly everyone has and why not do it in a proven environment that does nearly all of the work for you. I don't think at all that putting apps in containers, shipping them with a resource request/service descriptor, and then exposing them via service discovery is going anywhere at all. Whether it is k8s and Docker or something else, once you are using it, it is clear that this is really the abstraction deployments should be at. I'd say the odds are that you do need k8s stuff for any enterprise/business deployment of services. I'd be surprised if within a few years people are doing it any way else.

cookiecaper 9y ago
1. This is called "shared hosting". I sympathize with that desire. k8s is a way to roll your own shared host. Disinterested application developers who resent having to look at a command line do really like k8s because they treat it like a shared host platform, but they fail to appreciate the middle layer of complication that it entails, or the stickier elements of its architecture. A lot of ignorant app developers don't realize that you don't really get most of k8s's promises unless your application is already specifically designed to handle a transient, disappear-at-a-moment's-notice lifecycle, and most existing software is not built around these assumptions. It can't all be shoehorned in at the infrastructure level. Database servers are a fine example of an application architecture that is antithetical to this. 2. This is called "static linking". I also sympathize with static linking, but again this comes with implications that are not well understood by many of its advocates. In 4 years, I expect that some fad will emerge to repopularize dynamic linking under some new name, like "securable containerized orchestrated hypothalmized applicationators". Everyone will say "Dude, I can't imagine nobody using securable containerized orchestrated hypothamlmized applicatorness in 4 years". Rinse and repeat. 3. Yes, Docker sucks. We agree. 4. Sure, but you don't need to roll your own. Much of trendy devops is a reinvention of old things that we've already had and tried over the last several decades. For the most part, these come with the same old problems, and there are reasons they fell out of favor. There are definitely times when the tradeoffs will fall such that Kubernetes makes sense. Those times are much rarer than most people who are running k8s want to admit. That's because they're subconsciously running k8s as a status symbol, and lying to themselves about its necessity and value in their use case. I'm not saying the way we were doing things before k8s came around was exactly right. For example, going back to shared hosting is probably not a bad idea for the vast majority of things, and stuff like logging into your FTP server and copying your files have been gussied up as "cloud object storage" (and costs a lot more money now). So it's really just a general oscillation of trends, I guess.

dang 9y ago

You've got some good points about this but you're bringing it up repeatedly on HN in ways that are getting tedious (i.e. people have complained to us, and if one is complaining, many are bored). You're also being pretty cavalier with the flamebait (e.g. https://news.ycombinator.com/item?id=14453780 https://news.ycombinator.com/item?id=14453780). We're trying to avoid that sort of argument on HN, so could you please not? Yes engineers chase shiny new things and argue about it defensively. That's real, but it's also perennial—all us curmudgeons did the same thing in our day. This is cycle-of-life stuff. Such discussions quickly converge to the same old thing unless we're careful to avoid that, and the burden of avoiding it falls more on the senior members. We detached this subthread from https://news.ycombinator.com/item?id=14453544 https://news.ycombinator.com/item?id=14453544 and marked it off-topic.

cookiecaper 9y ago
I appreciate the explanation, and as multiple obviously-irritated k8s contributors have come in to counteract my other comments, I admit I have been curious how long HN would be willing to withstand the pressure from beings so highly-regarded and attached to such prestigious institutions. Surely, it is not wise to irritate powerful persons, especially not with semi-credible lambasts that affect an important centerpiece of their corporate strategy (k8s is part of the strategy to transition people to Google Cloud as opposed to AWS). This is also good supporting evidence for the amount of SV heterodoxy that HN-the-institution is willing to stomach (that topic was broached last time we crossed paths). Are perennial, cycle-of-life topics not expected to regularly appear on discussion platforms? I find it grating to see "k8s this, k8s that" all the time. Should I register a complaint as well, so that these threads can be killed next time they come up, on the theory that "many are bored" by the topic, based on the existence of my complaint? Specifically, which things are not allowed to be mentioned in the future? Was it: point 1, "Please consider whether you need k8s before you use it"; point 2, "Please don't run a database inside Kubernetes"; or point 3, "Yes, the software world is hardly manageable due to the unintended effects exposed by GitHub and friends"? Which of these is too tedious and boring to express on HN? I definitely would understand if only the flamebait comment were off-topic'd, as I know that was close to the edge. I'm more worried by the fact that counterpoints that are apparently "boring" or "tedious" justify censorship.

lobster_johnson 9y ago

Databases run fine in containers; conceptually there's very little difference between that and running outside of one. The burden is on yourself to know how to manage it. Specifically, Kubernetes provides the tools to "pin down" the database to the extent you want/need: Stateful sets, node affinities, QoS classes, nodepools and so on. You want to avoid OOMkilling the database, you want to dedicate the appropriate amount of resources to it, you may want to dedicate a specific type of hardware to it, etc. It's not for beginners, but nor is it anything like the disaster you portray it as.

cookiecaper 9y ago
This is true to the extent that containers remain in the realm of "concept" rather than reality. In the theoretical container that only serves the function of isolating each process within its own resource space, there is little reason why databases can't function reasonably well. This element alone is more frequently referred to as "sandboxing". However, in the real world of Docker and Kubernetes, it abstracts away components that actually matter to the function and administration of the database, and which can only be simulated by relying on experimental features like StatefulSets, or flaky features like node affinities that require each node to be registered with the correct labels and, as far as I know, are treated as preferences, not demands (i.e., pods will be scheduled on other nodes if nodes matching the affinity are not available). It is for beginners because non-beginners would realize that they're just taking a very complicated and winding route back to step one. If you want to avoid all OOM kills triggered by exceeding the cgroup spec, if you want to avoid the complexity and potential risk involved in configuring sturdy backing volumes and dealing with the questions around what happens when a node gets rescheduled onto a node that doesn't have that volume, if you want to ensure that the database is running reliably and stably on a specific type of hardware, the right solution is to go buy a server (or at the very least use a dedicated VM, not a container that will thrash it all over the place and expose the process to the risk of things like a dockerd hang), install the database on it, and leave it alone. Not to try to shoehorn it into k8s and try to grab this stuff that k8s exists to hide out of some weak abstract space. The risks associated far outweigh the benefits of Kubernetes for an application like a database (and arguably many other types of applications). As far as I know, databases are supposed to be administered with a great deal of respect for their rock-solid stability. The very suggestion of running one on a two-year-old platform should make any business very scared, even if that platform was conventional. k8s is anything but, and it is still trying to figure out how to provide the basics. Running a production database in k8s is a shockingly distasteful affront to stability just in principle. It sounds like the only reason people are able to give to justify database-in-k8s is "I want to be able to control it with `kubectl` like I can control everything else". My personal impulse would be to say that people who insist on using one administrative utility only should probably not be allowed to operate servers. A more realistic solution may be something like a kubectl proxy mode, where one can register external servers within the cluster namespace and use `kubectl exec <my-db-server> /bin/bash` to initiate an SSH connection.