4 ms·

> A container is basically never a bad idea Eh. If I'm running nginx on Debian I have to worry about Debian and Debian's version of nginx. If I'm running an n

> A container is basically never a bad idea Eh. If I'm running nginx on Debian I have to worry about Debian and Debian's version of nginx. If I'm running an nginx container on Debian I have to worry about Debian, Debian's container manager, the runtime nginx is containerized with, and the version of nginx in the container. The release cadence of Debian is well-known and can be planned around. The release cadence of an nginx container is not well-known, and is another thing the intern can configure wrong (I still shudder every time I see shops use "FROM foo:latest" in production, which is shockingly common). Not to mention the addition of an entire new class of supply chain vulnerabilities to worry about/mitigate (yes, I can build my own container, but I can also just build my own .deb file).

jupp0r 3y ago
You can use Debian stable as a base image and run its nginx everywhere where there is Linux. The host OS doesn't matter much if everything is containerized.
- bandrami 3y ago
  Except the host does matter, that's the thing, because you're still ultimately running on it. I remember this really bit me once in the RHEL 6 to 7 shift when the syscall semantics changed. Our brilliant plan of keeping our applications in RHEL 6 containers and running them on 7 while we did the upgrade, but everything realtime failed (these were audio processing engines) because of that ABI change. Not to mention the fact that your stack is only as secure as its least secure element, and now you've got two (or more) platforms to keep an eye on for that.
  - jupp0r 3y ago
    Wait, did you happen to work at Citrix a few years ago? I ran into the exact same problem, also working on audio processing engines :)
    - bandrami 3y ago
      No, a flight simulator manufacturer, but we ended up not being remotely the only team it bit (for that matter it may have been you guys' mailing list trail we followed to find the kernel tweak to work around it since Red Hat's answer was "yeah don't do that")
  - rbanffy 3y ago
    The downside of containers is that you have whatever kernel is on the host. I avoid running Docker containers on bare metal for that reason. You can still have RH6 VMs running containers hosted on a RH7 server. All this conversation reminded me of Solaris containers having Linux compatibility layers that pretended you were running on Linux even though you had a Solaris kernel under it. Wonder how much of that lives now in Illumos and other descendants of OpenSolaris.
- rbanffy 3y ago
  The host matters a lot in container/VM escape attacks. I actively avoid running workloads on bare metal and have been doing so since the early 2000’s. It’s always easier to move a VM image to a new generic VM host than to reinstall everything from scratch. What Docker does is to make delivering the app easier (it’s less involved than a full VM) and also automate the build process with a simple tool (Docker build).
  - justinclift 3y ago
    > It’s always easier to move a VM image to a new generic VM host than to reinstall everything from scratch. Ansible?
    - doubled112 3y ago
      This is my backup solution on machines that don’t hold data. Rebuild from playbooks, and/or Docker compose.
    - rbanffy 3y ago
      You don’t move the data with Ansible. You can move a VM to do load shifting.
tiew9Vii 3y ago
That's the thing. All containers are, are a standardized way to ship (distribute) your artifact (application) with it's dependencies inside it. It's a tar archive with a manifest file. I find it easier to think of them as RPM/deb files. You get an artifact repository for free and benefits of the onion file system to reduce size. People package entire OS distributions in their container, I consider that an anti pattern. Unless you are shipping static HTML your container shouldn't have NGINX or any other stuff in their, especially if you are running NGINX in front of your container anyway. If your app is written in Go/Rust/C, anything that produces a static executable, the only thing in the container should be the static executable... i.e. 'FROM scratch', 'cp my-app /bin/my-app`, 'ENTRYPOINT /bin/my-app'. If you use Java, Node, Python, anything that needs a runtime you can use NIX to build a container with only the dependencies needed to run those, nothing else. If you don't like NIX there's other ways to do it although I'm not familiar with them personally. All the security issues with containers comes from the junk rather than just your application and only the dependencies needed for that app. If a system library is not in your container you don't need to worry about patching it. Containers I find are always a good idea, it's a standard way to package/distribute. How you run those containers, that's another matter with Kubernetes becoming the de-facto because it's Kuberenetes and everyone else is using it. You don't need Kubernetes to run a container, especially if it is just one or a handful of containers.
- bandrami 3y ago
  Here's the dependency list for Bookworm's version of Python 3: libc6, libbz2, libssl3, libatomic1, libdb5, libffi8, liblzma5, libncursesw6, libnsl2, libreadline8, libsqlite3, libtinfo6, libtirpc3, libuuid1, binutils (why is this here? ar, I think?), libctf0, libgcc-s1, libgprofng0, libjansson4, libstdc++6, libzstd1, zlib1g. It does at least finally stop pulling in 2to3 with Bookworm, but that's nearly 2 dozen non-trivial libraries which now either are duplicated (in which case what was the point?) or exist in 2 versions (which is bad) on my server. And that's if I can put down the iron sysadmin fist and make devs pull only from a blessed image on our local registry (I can deal with the occasional shouts of "graybeard!" and "fascist!" but it makes meetings more acrimonious than they need to be). Now, I 100% agree that an otherwise-empty container is all you need for a Go app, but then what is the container actually doing that I can't just do with cgroups in systemd or runit? The whole point of containers was "Devs! Free yourself from the tyranny of a sysadmin limiting your library choices!" but, well, that's exactly the problem they introduce.
collyw 3y ago
We just had a problem with caused by docker, and it was exactly the problem that docker was supposed to solve. I find Docker to be a net positive, but it is far from perfect.
macNchz 3y ago
> If I'm running nginx on Debian I have to worry about Debian and Debian's version of nginx. This is kind of a funny example to me because it’s one of the reasons I liked containers so much when I first encountered them: Debian’s packages are notoriously old, and anytime I wanted to run a newer version of something I’d wind up with some patchwork of configuration management code and shell scripts. In contrast, a Dockerfile provides a simple and clear way to run whichever version I want without messing with the base system, and even multiple versions of a given package with conflicting dependencies on the same machine without any fuss.
- bandrami 3y ago
  And that's why Docker is a great thing to be sitting on a developer's laptop and making the next version of something. The insanity was when people looked at that and decided it should be the normative way to deploy in production.

jupp0r 3y ago

You can use Debian stable as a base image and run its nginx everywhere where there is Linux. The host OS doesn't matter much if everything is containerized.

bandrami 3y ago
Except the host does matter, that's the thing, because you're still ultimately running on it. I remember this really bit me once in the RHEL 6 to 7 shift when the syscall semantics changed. Our brilliant plan of keeping our applications in RHEL 6 containers and running them on 7 while we did the upgrade, but everything realtime failed (these were audio processing engines) because of that ABI change. Not to mention the fact that your stack is only as secure as its least secure element, and now you've got two (or more) platforms to keep an eye on for that.
- jupp0r 3y ago
  Wait, did you happen to work at Citrix a few years ago? I ran into the exact same problem, also working on audio processing engines :)
  - bandrami 3y ago
    No, a flight simulator manufacturer, but we ended up not being remotely the only team it bit (for that matter it may have been you guys' mailing list trail we followed to find the kernel tweak to work around it since Red Hat's answer was "yeah don't do that")
- rbanffy 3y ago
  The downside of containers is that you have whatever kernel is on the host. I avoid running Docker containers on bare metal for that reason. You can still have RH6 VMs running containers hosted on a RH7 server. All this conversation reminded me of Solaris containers having Linux compatibility layers that pretended you were running on Linux even though you had a Solaris kernel under it. Wonder how much of that lives now in Illumos and other descendants of OpenSolaris.
rbanffy 3y ago
The host matters a lot in container/VM escape attacks. I actively avoid running workloads on bare metal and have been doing so since the early 2000’s. It’s always easier to move a VM image to a new generic VM host than to reinstall everything from scratch. What Docker does is to make delivering the app easier (it’s less involved than a full VM) and also automate the build process with a simple tool (Docker build).
- justinclift 3y ago
  > It’s always easier to move a VM image to a new generic VM host than to reinstall everything from scratch. Ansible?
  - doubled112 3y ago
    This is my backup solution on machines that don’t hold data. Rebuild from playbooks, and/or Docker compose.
  - rbanffy 3y ago
    You don’t move the data with Ansible. You can move a VM to do load shifting.

tiew9Vii 3y ago

That's the thing. All containers are, are a standardized way to ship (distribute) your artifact (application) with it's dependencies inside it. It's a tar archive with a manifest file. I find it easier to think of them as RPM/deb files. You get an artifact repository for free and benefits of the onion file system to reduce size. People package entire OS distributions in their container, I consider that an anti pattern. Unless you are shipping static HTML your container shouldn't have NGINX or any other stuff in their, especially if you are running NGINX in front of your container anyway. If your app is written in Go/Rust/C, anything that produces a static executable, the only thing in the container should be the static executable... i.e. 'FROM scratch', 'cp my-app /bin/my-app`, 'ENTRYPOINT /bin/my-app'. If you use Java, Node, Python, anything that needs a runtime you can use NIX to build a container with only the dependencies needed to run those, nothing else. If you don't like NIX there's other ways to do it although I'm not familiar with them personally. All the security issues with containers comes from the junk rather than just your application and only the dependencies needed for that app. If a system library is not in your container you don't need to worry about patching it. Containers I find are always a good idea, it's a standard way to package/distribute. How you run those containers, that's another matter with Kubernetes becoming the de-facto because it's Kuberenetes and everyone else is using it. You don't need Kubernetes to run a container, especially if it is just one or a handful of containers.

bandrami 3y ago
Here's the dependency list for Bookworm's version of Python 3: libc6, libbz2, libssl3, libatomic1, libdb5, libffi8, liblzma5, libncursesw6, libnsl2, libreadline8, libsqlite3, libtinfo6, libtirpc3, libuuid1, binutils (why is this here? ar, I think?), libctf0, libgcc-s1, libgprofng0, libjansson4, libstdc++6, libzstd1, zlib1g. It does at least finally stop pulling in 2to3 with Bookworm, but that's nearly 2 dozen non-trivial libraries which now either are duplicated (in which case what was the point?) or exist in 2 versions (which is bad) on my server. And that's if I can put down the iron sysadmin fist and make devs pull only from a blessed image on our local registry (I can deal with the occasional shouts of "graybeard!" and "fascist!" but it makes meetings more acrimonious than they need to be). Now, I 100% agree that an otherwise-empty container is all you need for a Go app, but then what is the container actually doing that I can't just do with cgroups in systemd or runit? The whole point of containers was "Devs! Free yourself from the tyranny of a sysadmin limiting your library choices!" but, well, that's exactly the problem they introduce.

collyw 3y ago

We just had a problem with caused by docker, and it was exactly the problem that docker was supposed to solve. I find Docker to be a net positive, but it is far from perfect.

macNchz 3y ago

> If I'm running nginx on Debian I have to worry about Debian and Debian's version of nginx. This is kind of a funny example to me because it’s one of the reasons I liked containers so much when I first encountered them: Debian’s packages are notoriously old, and anytime I wanted to run a newer version of something I’d wind up with some patchwork of configuration management code and shell scripts. In contrast, a Dockerfile provides a simple and clear way to run whichever version I want without messing with the base system, and even multiple versions of a given package with conflicting dependencies on the same machine without any fuss.

bandrami 3y ago
And that's why Docker is a great thing to be sitting on a developer's laptop and making the next version of something. The insanity was when people looked at that and decided it should be the normative way to deploy in production.