6 ms·

But it seems that CoreOS is more suitable as being the underlying OS on which containers are run, rather than being the basis of a container image?

jsmthrowaway 11y ago
Container images do not need operating systems, and I will forever be angry with Docker for tainting the concept in popular usage. Research a static binary as a container to see what I mean. Nobody who built containers before Docker came along shipped an operating system in the image. Now we have gigabytes of Ubuntu alongside nginx thanks to Docker. Cool.
- frik 11y ago
  This. With other sandbox container solutions, you just start it. Docker's is mixing a sandbox, a deployment solution and more together. This might be useful for large deployments, but for smaller one it's too complex.
  - merb 11y ago
    for sandboxes you don't need containers and for resource management, neither. The only thing which docker gives you, is that it enforces immutable disks, which cgroups can't do, can't yet do or whatever. however when even with docker you need to ensure that your program won't mutate the filesystem so anyways nothing special.
    - icebraining 11y ago
      Enforcing an immutable filesystem is actually pretty easy to do, just don't give write permissions to the user under which the program is running :) What Docker integrates is a layered filesystem, in which the application can mutate the filesystem, but only it will see those changes (and they'll go away when the container is destroyed).
      - merb 11y ago
        oh yeah sorry. however on some linux filesystems you could also use `chattr +x` however most things won't play well with that ;) especially not programs which create logs / pids / whatever.
      - justincormack 11y ago
        You can mount the base filesystem read only in the container as of 1.10, with tmpfs mounts for where you need ephemeral mutation.
- shykes 11y ago
  Disclaimer: I work at Docker. > Nobody who built containers before Docker came along shipped an operating system in the image. Now we have gigabytes of Ubuntu alongside nginx thanks to Docker. Cool. First: I believe shipping the entire distro in each container was in fact the most common use of containers before Docker, since openvz/vserver/lxc were focused on "mini-servers" (to this day you can still rent openvz-based vps). So I think your statement is incorrect. I also think your (eternal?) anger might be misplaced. Yes, today most people who use Docker build containers with unnecessary distro bits in them. But before Docker, 99% of them didn't build containers at all, and struggled with non-repeatable deployments and dependency hell. Even better, now all these people use a standardized Dockerfile which can be changed and improved over time - for example to move away from a heavyweight distro like fedora or ubuntu, to something like alpine or even more minimalistic. Incidentay, we have hired Natanael Copa, the awesome creator of Alpine Linux and are in the process of switching the Docker official image library from ubuntu to Alpine. You can help us with pull requests to https://github.com/docker-library https://github.com/docker-library if you want :) Even one such pull request could go a long way to promoting smaller containers, since those images have been downloaded over half a billion times.
  - kylequest 11y ago
    > Yes, today most people who use Docker build containers with unnecessary distro bits in them. >... to move away from a heavyweight distro like fedora or ubuntu, to something like alpine or even more minimalistic. The distro package management is one of the biggest reasons why people still stick with heavyweight distros like Ubuntu. This is why DockerSlim [1] was created (at one of the Docker hack days, by the way :-)) It lets you use heavyweight distros as-is eliminating everything you don't need. You get the best of both worlds where you don't have to change your process and you still get small images (7, 10, 20 and even 30 times smaller than the original). [1] - http://dockersl.im http://dockersl.im P.S. It is safe for use in production :)
    - jtblin 11y ago
      There's a packer manager in alpine. Maybe not as complete as ubuntu but pretty decent imho.
- tachion 11y ago
  > Nobody who built containers before Docker came along shipped an operating system in the image. Now we have gigabytes of Ubuntu alongside nginx thanks to Docker. Cool. This couldnt be more false - other systems had containers capabilities for years now (FreeBSD Jails, Solaris Zones, IBM LPAR's, Linux OpenVZ and so on) and since forever it was done in a way of running another full OS inside a container. Stripping that down to single processes was possible and done by some people but rarely, because it was relatively hard (since it often required hand-crafting the 'packages', hunting down single libs and their requirements using ldd and the like). I myself was running both full OS jail containers on FreeBSD and stripped down single process/service jails more than 10 years ago.
  - jsmthrowaway 11y ago
    As I told Solomon, this confusion is due to muddying of the term. I'm connected to the Google school of thought on this (where basically all the billions of containers in prod are `FROM scratch`, because they did the work you call hard to ship a process as a container), whereas jails are the other school of thought. I would contend that they are different enough to deserve different terms, and this thread of correcting what I meant, and explaining jails to me even though I would think it pretty clear I understand them, is evidence enough. We called those jails when FreeBSD was carrying that torch. I never once heard them called containers until now. That is my point. Docker is not for the jail case. If it were, it would look like OpenVZ and come with completely different architecture and management tooling. It's for deploying artifacts to prod, and my entire point is that starting from `FROM ubuntu` for even a Go binary has muddled this picture irreversibly. Now I have to have this argument, which makes no sense, because we now take it as fact that containers and jails are the same thing, and anyone who says otherwise deserves to have very basic things explained to them carefully and rather condescendingly. This reply applies to everyone who is calling me wrong. I spent last summer employed by CoreOS, have worked on Google Borg, and spent the first half of my career on the production artifact toolchain. I'm not just farting into the breeze here with my two personal servers and opinions.
- nzoschke 11y ago
  Not true. Managing OS images is the killer app for containers. Containers power the Heroku platform and it supports millions of apps by containerizing OS images. When both the bamboo stack and cedar stack were available, we would: Run a base AMI and update it frequently for kernel, security and performance updates. Download a bamboo (Debian) OS layer Dowbload a cedar (Ubuntu 12.04) OS layer. Layer app slugs on top of which ever stack the customer wanted. This was amazing for long term support of apps and backwards compatibility. There are infinitely more rails apps out there that deeply depend on the OS layer than there are static binary apps. Docker deserves and gets all its credit for making this technique available and easy for everyone.
lfittl 11y ago
To answer this more explicitly than the other comment: Yes, CoreOS is intended as the host system for containers. Inside the Docker (or rkt) containers you can run whichever distribution or static binary you like. The combination of CoreOS+Kubernetes+Docker+Alpine Linux (inside the containers) works pretty well in my experience.

jsmthrowaway 11y ago

Container images do not need operating systems, and I will forever be angry with Docker for tainting the concept in popular usage. Research a static binary as a container to see what I mean. Nobody who built containers before Docker came along shipped an operating system in the image. Now we have gigabytes of Ubuntu alongside nginx thanks to Docker. Cool.

frik 11y ago
This. With other sandbox container solutions, you just start it. Docker's is mixing a sandbox, a deployment solution and more together. This might be useful for large deployments, but for smaller one it's too complex.
- merb 11y ago
  for sandboxes you don't need containers and for resource management, neither. The only thing which docker gives you, is that it enforces immutable disks, which cgroups can't do, can't yet do or whatever. however when even with docker you need to ensure that your program won't mutate the filesystem so anyways nothing special.
  - icebraining 11y ago
    Enforcing an immutable filesystem is actually pretty easy to do, just don't give write permissions to the user under which the program is running :) What Docker integrates is a layered filesystem, in which the application can mutate the filesystem, but only it will see those changes (and they'll go away when the container is destroyed).
    - merb 11y ago
      oh yeah sorry. however on some linux filesystems you could also use `chattr +x` however most things won't play well with that ;) especially not programs which create logs / pids / whatever.
    - justincormack 11y ago
      You can mount the base filesystem read only in the container as of 1.10, with tmpfs mounts for where you need ephemeral mutation.
shykes 11y ago
Disclaimer: I work at Docker. > Nobody who built containers before Docker came along shipped an operating system in the image. Now we have gigabytes of Ubuntu alongside nginx thanks to Docker. Cool. First: I believe shipping the entire distro in each container was in fact the most common use of containers before Docker, since openvz/vserver/lxc were focused on "mini-servers" (to this day you can still rent openvz-based vps). So I think your statement is incorrect. I also think your (eternal?) anger might be misplaced. Yes, today most people who use Docker build containers with unnecessary distro bits in them. But before Docker, 99% of them didn't build containers at all, and struggled with non-repeatable deployments and dependency hell. Even better, now all these people use a standardized Dockerfile which can be changed and improved over time - for example to move away from a heavyweight distro like fedora or ubuntu, to something like alpine or even more minimalistic. Incidentay, we have hired Natanael Copa, the awesome creator of Alpine Linux and are in the process of switching the Docker official image library from ubuntu to Alpine. You can help us with pull requests to https://github.com/docker-library https://github.com/docker-library if you want :) Even one such pull request could go a long way to promoting smaller containers, since those images have been downloaded over half a billion times.
- kylequest 11y ago
  > Yes, today most people who use Docker build containers with unnecessary distro bits in them. >... to move away from a heavyweight distro like fedora or ubuntu, to something like alpine or even more minimalistic. The distro package management is one of the biggest reasons why people still stick with heavyweight distros like Ubuntu. This is why DockerSlim [1] was created (at one of the Docker hack days, by the way :-)) It lets you use heavyweight distros as-is eliminating everything you don't need. You get the best of both worlds where you don't have to change your process and you still get small images (7, 10, 20 and even 30 times smaller than the original). [1] - http://dockersl.im http://dockersl.im P.S. It is safe for use in production :)
  - jtblin 11y ago
    There's a packer manager in alpine. Maybe not as complete as ubuntu but pretty decent imho.
tachion 11y ago
> Nobody who built containers before Docker came along shipped an operating system in the image. Now we have gigabytes of Ubuntu alongside nginx thanks to Docker. Cool. This couldnt be more false - other systems had containers capabilities for years now (FreeBSD Jails, Solaris Zones, IBM LPAR's, Linux OpenVZ and so on) and since forever it was done in a way of running another full OS inside a container. Stripping that down to single processes was possible and done by some people but rarely, because it was relatively hard (since it often required hand-crafting the 'packages', hunting down single libs and their requirements using ldd and the like). I myself was running both full OS jail containers on FreeBSD and stripped down single process/service jails more than 10 years ago.
- jsmthrowaway 11y ago
  As I told Solomon, this confusion is due to muddying of the term. I'm connected to the Google school of thought on this (where basically all the billions of containers in prod are `FROM scratch`, because they did the work you call hard to ship a process as a container), whereas jails are the other school of thought. I would contend that they are different enough to deserve different terms, and this thread of correcting what I meant, and explaining jails to me even though I would think it pretty clear I understand them, is evidence enough. We called those jails when FreeBSD was carrying that torch. I never once heard them called containers until now. That is my point. Docker is not for the jail case. If it were, it would look like OpenVZ and come with completely different architecture and management tooling. It's for deploying artifacts to prod, and my entire point is that starting from `FROM ubuntu` for even a Go binary has muddled this picture irreversibly. Now I have to have this argument, which makes no sense, because we now take it as fact that containers and jails are the same thing, and anyone who says otherwise deserves to have very basic things explained to them carefully and rather condescendingly. This reply applies to everyone who is calling me wrong. I spent last summer employed by CoreOS, have worked on Google Borg, and spent the first half of my career on the production artifact toolchain. I'm not just farting into the breeze here with my two personal servers and opinions.
nzoschke 11y ago
Not true. Managing OS images is the killer app for containers. Containers power the Heroku platform and it supports millions of apps by containerizing OS images. When both the bamboo stack and cedar stack were available, we would: Run a base AMI and update it frequently for kernel, security and performance updates. Download a bamboo (Debian) OS layer Dowbload a cedar (Ubuntu 12.04) OS layer. Layer app slugs on top of which ever stack the customer wanted. This was amazing for long term support of apps and backwards compatibility. There are infinitely more rails apps out there that deeply depend on the OS layer than there are static binary apps. Docker deserves and gets all its credit for making this technique available and easy for everyone.

lfittl 11y ago

To answer this more explicitly than the other comment: Yes, CoreOS is intended as the host system for containers. Inside the Docker (or rkt) containers you can run whichever distribution or static binary you like. The combination of CoreOS+Kubernetes+Docker+Alpine Linux (inside the containers) works pretty well in my experience.