4 ms·

I remember when this article came up a long time ago. We had just set up an in-house Kubernetes cluster running on AWS. We were attempting to run Cassandra on

by dastx 6y ago

I remember when this article came up a long time ago. We had just set up an in-house Kubernetes cluster running on AWS. We were attempting to run Cassandra on top of it and let's just say the majority of our time for a few months was spent fine tuning the setup (iirc this was early days StatefulSets). In the end we gave up and went to EC2 for Cassandra. Persistent volumes have come a long way, and container technology in general has rapidly evolved. I have yet to try any databases on K8s in a production environment since that attempt, but I don't believe that it's as bad as it was back then. Many people are scared of it, one of the biggest reasons usually given is that containers are meant to be ephemeral. While containers allow for applications to be ephemeral, and quite frankly makes it a lot easier, I don't believe that they're meant to be ephemeral. At the end of the day, containers are just somewhat isolated processes. Processes don't need to be ephemeral, and often aren't. With the right setup, databases can be run from containers, and I believe the rapid evolution of container technology has allowed for this to be possible today.

qeternity 6y ago
Containers are no more ephemeral than any other process. People who say this I think are just saying that because of default docker storage settings.
- lathiat 6y ago
  I mean originally all EC2 instances were ephemeral by default. Shutdown and all your Bitcoin goes bye-bye.
- rualca 6y ago
  >>Containers are no more ephemeral than any other process. You're missing the point. It's not that containers cannot be ephemeral. The whole point is that containers are meant, and expected, to be ephemeral by design. Pets vs cattle and all.
  - spiffytech 6y ago
    Containers, and cattle-vs-pets, is about making dirty system state disposable and making apps cookie-cutter and repeatably deployable. But that's not the same as saying containers are only suitable for ephemeral workloads/data. With persistent storage it's not just possible, but I think makes a lot of sense to run something like a database inside a container. My default is to run Postgres etc. in Docker instead of 'apt install postgres', because it's so much easier to get the right Postgres version with Docker than tracking down whatever unofficial apt repo has the Postgres version I'm looking for. The key is I do this in Docker, with a basic folder bind mount on the exact same kind of machine I would use if I installed Postgres via 'apt'. I don't do this in Kubernetes or use cloud storage volume adapters. Kubernetes brings a lot of complexity and risk into ops, and that's not what you want for a database. Docker is mostly a layered filesystem format plus a cgroups isolation wrapper around native Linux processes. So as long as you're not mixing in some other tool that e.g., adds on network traffic routing or something, it's basically just Linux with convenience features. Switching from 'apt' to Docker with maybe a basic docker-compose.yml is an underrated infrastructure pattern. Operating Postgres from Docker feels about as simple as apt-installed Postgres, with no downsides I've seen besides needing Docker at all. It's also great for deploying in-house apps; moving workloads from vanilla OS processes with systemd and Ansible or whatever over to plain ol' Docker (or maybe Dokku) simplifies things every time I do it. It's fancy tools that build on top of Docker that make things complicated or risky, but someone rejecting vanilla Docker because they don't want to use K8s feels like throwing the baby out with the bathwater.
    - rualca 6y ago
      > But that's not the same as saying containers are only suitable for ephemeral workloads/data. No one said that. > With persistent storage it's not just possible, There's some confusion on your remarks. A container can be ephemeral even if it has access to persistent data. I mean, think about it: isn't a container still ephemeral even if it has a database connection? Ephemeral is about internal state, and how recreating a container after deleting it will get it in the exact same state.
mywittyname 6y ago
What's the functional benefit of using containers over EC2 images for a database? I can't see why anyone would reasonably consider trying such a thing. Containers make a lot of sense when you're the one developing the software, and you have a lot of micro-services to support, none of which really scale to EC2 levels of utilization. But EC2s seem better suited to running pre-installed software that can, and will, fully utilize an entire VM. Such as a database. My intuition on this subject is, docker is the latest, trendy hammer and so everything becomes a nail / container. Maybe people aren't familiar with the tooling around EC2 image creation and deployment.
- closeparen 6y ago
  It is attractive to have everything in your production environment to be of the same type and managed through the same handles.
- rad_gruchalski 6y ago
  > What's the functional benefit of using containers over EC2 images for a database? I can't see why anyone would reasonably consider trying such a thing. Version upgrade speed.
  - mywittyname 6y ago
    How so? With a VM image, the process can be as simple as, yum update, snapshot, then update the configuration to point to the new AMI ID. And if you're using a tool like packer to create AMIs, the process might be to kick off the CI/CD pipeline again.
    - rad_gruchalski 6y ago
      Versus, in the simplest scenario, docker run -e ... -v ... image:<new-version> which can be executed via Ansible or similar. Another method is to simply bump a version in marathon app definition / kubernetes config and reapply.

by dastx 6y ago

qeternity 6y ago

Containers are no more ephemeral than any other process. People who say this I think are just saying that because of default docker storage settings.

lathiat 6y ago
I mean originally all EC2 instances were ephemeral by default. Shutdown and all your Bitcoin goes bye-bye.
rualca 6y ago
>>Containers are no more ephemeral than any other process. You're missing the point. It's not that containers cannot be ephemeral. The whole point is that containers are meant, and expected, to be ephemeral by design. Pets vs cattle and all.
- spiffytech 6y ago
  Containers, and cattle-vs-pets, is about making dirty system state disposable and making apps cookie-cutter and repeatably deployable. But that's not the same as saying containers are only suitable for ephemeral workloads/data. With persistent storage it's not just possible, but I think makes a lot of sense to run something like a database inside a container. My default is to run Postgres etc. in Docker instead of 'apt install postgres', because it's so much easier to get the right Postgres version with Docker than tracking down whatever unofficial apt repo has the Postgres version I'm looking for. The key is I do this in Docker, with a basic folder bind mount on the exact same kind of machine I would use if I installed Postgres via 'apt'. I don't do this in Kubernetes or use cloud storage volume adapters. Kubernetes brings a lot of complexity and risk into ops, and that's not what you want for a database. Docker is mostly a layered filesystem format plus a cgroups isolation wrapper around native Linux processes. So as long as you're not mixing in some other tool that e.g., adds on network traffic routing or something, it's basically just Linux with convenience features. Switching from 'apt' to Docker with maybe a basic docker-compose.yml is an underrated infrastructure pattern. Operating Postgres from Docker feels about as simple as apt-installed Postgres, with no downsides I've seen besides needing Docker at all. It's also great for deploying in-house apps; moving workloads from vanilla OS processes with systemd and Ansible or whatever over to plain ol' Docker (or maybe Dokku) simplifies things every time I do it. It's fancy tools that build on top of Docker that make things complicated or risky, but someone rejecting vanilla Docker because they don't want to use K8s feels like throwing the baby out with the bathwater.
  - rualca 6y ago
    > But that's not the same as saying containers are only suitable for ephemeral workloads/data. No one said that. > With persistent storage it's not just possible, There's some confusion on your remarks. A container can be ephemeral even if it has access to persistent data. I mean, think about it: isn't a container still ephemeral even if it has a database connection? Ephemeral is about internal state, and how recreating a container after deleting it will get it in the exact same state.

mywittyname 6y ago

What's the functional benefit of using containers over EC2 images for a database? I can't see why anyone would reasonably consider trying such a thing. Containers make a lot of sense when you're the one developing the software, and you have a lot of micro-services to support, none of which really scale to EC2 levels of utilization. But EC2s seem better suited to running pre-installed software that can, and will, fully utilize an entire VM. Such as a database. My intuition on this subject is, docker is the latest, trendy hammer and so everything becomes a nail / container. Maybe people aren't familiar with the tooling around EC2 image creation and deployment.

closeparen 6y ago
It is attractive to have everything in your production environment to be of the same type and managed through the same handles.
rad_gruchalski 6y ago
> What's the functional benefit of using containers over EC2 images for a database? I can't see why anyone would reasonably consider trying such a thing. Version upgrade speed.
- mywittyname 6y ago
  How so? With a VM image, the process can be as simple as, yum update, snapshot, then update the configuration to point to the new AMI ID. And if you're using a tool like packer to create AMIs, the process might be to kick off the CI/CD pipeline again.
  - rad_gruchalski 6y ago
    Versus, in the simplest scenario, docker run -e ... -v ... image:<new-version> which can be executed via Ansible or similar. Another method is to simply bump a version in marathon app definition / kubernetes config and reapply.