8 ms·

Size is such a tiny concern. I'm surprised people make such a big deal about it. When all of your images use the same base, it's only a one-time cost anyway. A

Size is such a tiny concern. I'm surprised people make such a big deal about it. When all of your images use the same base, it's only a one-time cost anyway. And there are FAR more important concerns: - Are the packages in your base system well maintained and updated with security fixes? - Does your base system have longevity? Will it still be maintained a few years from now? - Does it handle all of the special corner cases that Docker causes? That's why I use https://github.com/phusion/baseimage-docker https://github.com/phusion/baseimage-docker

_Codemonkeyism 11y ago
"Will it still be maintained a few years from now?" What would you suggest? Debian? Ubuntu LTS?
- kstenerud 11y ago
  Both. Debian is the gold standard of long term support, and Ubuntu is a stable company that builds upon this. And that's why I approve of phusion/baseimage being based off it.
  - _Codemonkeyism 11y ago
    Thanks!
- e40 11y ago
  Or CentOS 7 (Docker is not supported any more on 6.x).
  - floatboth 11y ago
    CentOS is the gold standard of ANCIENT packages. I think they still ship Ruby 1.8.
    - loudmax 11y ago
      RHEL (and by extension CentOS) 7 provides Ruby 2.0. And a 3.10 kernel even. If you're running docker with CentOS, this is what you're likely to use. RHEL/CentOS 6 provides Ruby 1.8.7 and a 2.6.32 kernel. It can be made to run with docker, but it's unsupported and it won't be easy. RHEL/CentOS 5 provides Ruby 1.8.5. The 2.6.18 kernel it comes with won't even run go binaries such as docker, much less lxc. Yes this is ancient. It was released in 2007 and it will be supported until 2017.
_yy 11y ago
As a sysadmin, I fully agree. I shudder at the thought of using Alpine Linux (or Arch Linux, or...) in production. The value that stable, long term support distros provide shouldn't be underestimated.
- krisdol 11y ago
  We use Alpine Linux for our applications and I like it, and I too shudder at it being used for the entire production system. As a sysadmin, you can still administer the LTS distro that hosts the docker containers and whatever other pieces of the stack you interact with. Alpine Linux containers, like any other container, should host an instance of an application (maybe not even that, depending on how complex the application is) -- not the entire production server, not SSH keys, not iptables, firewall rules, etc.
- anonbanker 11y ago
  Using Gentoo stable in production right now. I'm in charge of how long a package is supported now. All execs get a brand new gentoo machine built with binaries compiled by myself. You wouldn't believe how fast you can get a gentoo machine up and running compared to other distros. Build for a minimum common architechture (all intel binaries are based on Sandy Bridge, all ARM based on Rockchip RK3088), and installing for new computer is little more than untarring a bunch of binaries to /. My record is 5 minutes for a full KDE Plasma 5.5 software stack.
  - _yy 11y ago
    I explicitly did not mention Gentoo - I know a bunch of people who run it in production. But, for anyone considering to do this - if you're running Gentoo, you're essentially building your own distro, which has massive advantages but is also a huge effort. You're now in charge of security updates, maintenance and Q&A. What if you're leaving the company? There are many Debian or Redhat admins, but good luck finding a Gentoo expert.
    - anonbanker 11y ago
      I train my replacements, much like every Sith should.
koffiezet 11y ago
Sorry - but the phusion images are unnecessarily bloated. The existence of the them has been defended by 'fixing' many so-called problems that are actually no problem at all - or at least shouldn't be a problem if you know what the hell you're doing. No well, written software won't spawn zombie-processes - sorry. Reaping dead child processes is something pretty basic if you're using "fork". And then - a logger daemon. Guess mounting /dev/log into a container is too complex if you care about this? Logrotate - sure, useful - but if you care about logs and aren't sending them to your logger daemon or /dev/null, you probably want to store them externally - in a volume or mounted host directory - and have a separate container taking care of that. The ssh server... Containers are no vm's, if you have to log in on a container running in production - you're doing something wrong - unless that container's only job is running SSH (which can be useful for example for Jenkins build slaves). Cron - again - same thing: run in a separate container and give access to the exact things your cronjob needs. That is for me the essential thing about containers: separate everything. But sure, you could treat containers as a special VM only for one service - nobody is going to stop you. I however prefer isolating every single process and explicitly telling it how to communicate with other processes. It's sane from many perspectives: security, maintainability, flexibility and speed.
- detaro 11y ago
  Agree in general, but you can overdo it with splitting services up. E.g. would you really run a extra container just for a cronjob that runs once a night to e-mail some data from a database? Esp. if you run on a platform where you essentially pay per container that seems like a waste.
  - koffiezet 11y ago
    Most of the things I described assume you have full control over your host's OS. For stuff like you mention - you should maybe reconsider not using containers if you're on a pay-per-container platform? They are just a tool, and certainly don't fit every single use-case. Also - paying per container seems like a silly thing to do - since containers can be very short-lived. Resource-based billing would be a better fit - although that could be tricky to measure I guess.
logician76 11y ago
Container size influences memory usage, which is an important concern: http://www.stackengine.com/container-memory-usage-or-size-matters/ http://www.stackengine.com/container-memory-usage-or-size-ma...
- kstenerud 11y ago
  I'd be careful about drawing conclusions from those tests. We know that the number of bits in a container does not directly influence how much RAM it consumes. Therefore, there must be something the images are doing that consumes memory which is not happening in the "smaller" images. The key would be to find the culpable process or daemon(s).
  - ploxiln 11y ago
    It could well be due to things like shared libraries. A larger distro will have more options enabled, causing more shared libraries to be linked into the same running processes, and thus more shared libraries to be fully loaded into memory. A smaller distro might even statically compile most things - Alpine does. If you dynamically link shared libraries, the whole library is loaded into memory to serve the process. If you statically link, only the actually used part of the library is included in the binary. Statically linked binaries can't share the library in memory between each other like dynamically linked binaries can, but if all your processes are running in separate containers, they won't share those libraries anyway (unless they're all in a VM and the rarely used "samepage merging" is enabled for the VM). Finally ... simplicity has knock-on effects. Making things simpler and smaller (not easier), and reducing the number of moving parts in the implementation, makes cleaning up more stuff easier.
    - Rapzid 11y ago
      That's an awful lot of conjecture. I'd wager that most of what you would actually be running in a container would not have its memory usage significantly affected by the presence or absence of optional shared libs.. I'm with the parent on this; such claims warrant research.
ploxiln 11y ago
Size of programs, in terms of disk, memory, cpu time, and network usage, is bloated by multiple orders of magnitude by all the confused people who think the only thing that matters is "developer productivity". Maybe 20% is worth sacrificing, maybe 50%, but 100x? 1000x? It all adds up. One really easy and relevant example, sizes of docker images for running memcached: vagrant@dockerdev:~$ sudo docker images | grep memcached memcached latest 0868b36194d3 2 weeks ago 132.2 MB sylvainlasnier/memcached latest 97a88c3744ef 13 months ago 297.4 MB ploxiln/memcached 2015-07-08 aa4a87ee2c05 5 months ago 7.453 MB (that last one is my own, the other two are the two most popular on docker hub). As another example, a co-worker recently was working with some (out-of-tree) gstreamer plugins, and the most convenient way to do so was with a docker image in which all the major gstreamer dependencies, the latest version of gstreamer, and the out-of-tree plugins were built from source. The offered image was over 10GB and 30 layers, took quite a while to download, and a surprising number of seconds to run. With just a few tweaks it was reduced to 1.1GB and a handful of layers which runs in less than a second. It was just a total lack of care for efficiency that made it 10x less efficient in every way, enough to actually reduce developer productivity. Size matters.
- voltagex_ 11y ago
  Are you able to share any of the tweaks that were used?
  - ploxiln 11y ago
    One really easy one: write a shell script to do most of the image building (run by the Dockerfile), instead of adding a bunch of RUN directives in the Dockerfile, especially if you clean up intermediate files with a "make clean" or something. Each directive in the Dockerfile adds a layer, which adds container setup overhead, and also "locks in" all filesystem space usage at that point.
    - deleted 11y ago
      [deleted]

_Codemonkeyism 11y ago

"Will it still be maintained a few years from now?" What would you suggest? Debian? Ubuntu LTS?

kstenerud 11y ago
Both. Debian is the gold standard of long term support, and Ubuntu is a stable company that builds upon this. And that's why I approve of phusion/baseimage being based off it.
- _Codemonkeyism 11y ago
  Thanks!
e40 11y ago
Or CentOS 7 (Docker is not supported any more on 6.x).
- floatboth 11y ago
  CentOS is the gold standard of ANCIENT packages. I think they still ship Ruby 1.8.
  - loudmax 11y ago
    RHEL (and by extension CentOS) 7 provides Ruby 2.0. And a 3.10 kernel even. If you're running docker with CentOS, this is what you're likely to use. RHEL/CentOS 6 provides Ruby 1.8.7 and a 2.6.32 kernel. It can be made to run with docker, but it's unsupported and it won't be easy. RHEL/CentOS 5 provides Ruby 1.8.5. The 2.6.18 kernel it comes with won't even run go binaries such as docker, much less lxc. Yes this is ancient. It was released in 2007 and it will be supported until 2017.

_yy 11y ago

As a sysadmin, I fully agree. I shudder at the thought of using Alpine Linux (or Arch Linux, or...) in production. The value that stable, long term support distros provide shouldn't be underestimated.

krisdol 11y ago
We use Alpine Linux for our applications and I like it, and I too shudder at it being used for the entire production system. As a sysadmin, you can still administer the LTS distro that hosts the docker containers and whatever other pieces of the stack you interact with. Alpine Linux containers, like any other container, should host an instance of an application (maybe not even that, depending on how complex the application is) -- not the entire production server, not SSH keys, not iptables, firewall rules, etc.
anonbanker 11y ago
Using Gentoo stable in production right now. I'm in charge of how long a package is supported now. All execs get a brand new gentoo machine built with binaries compiled by myself. You wouldn't believe how fast you can get a gentoo machine up and running compared to other distros. Build for a minimum common architechture (all intel binaries are based on Sandy Bridge, all ARM based on Rockchip RK3088), and installing for new computer is little more than untarring a bunch of binaries to /. My record is 5 minutes for a full KDE Plasma 5.5 software stack.
- _yy 11y ago
  I explicitly did not mention Gentoo - I know a bunch of people who run it in production. But, for anyone considering to do this - if you're running Gentoo, you're essentially building your own distro, which has massive advantages but is also a huge effort. You're now in charge of security updates, maintenance and Q&A. What if you're leaving the company? There are many Debian or Redhat admins, but good luck finding a Gentoo expert.
  - anonbanker 11y ago
    I train my replacements, much like every Sith should.

koffiezet 11y ago

Sorry - but the phusion images are unnecessarily bloated. The existence of the them has been defended by 'fixing' many so-called problems that are actually no problem at all - or at least shouldn't be a problem if you know what the hell you're doing. No well, written software won't spawn zombie-processes - sorry. Reaping dead child processes is something pretty basic if you're using "fork". And then - a logger daemon. Guess mounting /dev/log into a container is too complex if you care about this? Logrotate - sure, useful - but if you care about logs and aren't sending them to your logger daemon or /dev/null, you probably want to store them externally - in a volume or mounted host directory - and have a separate container taking care of that. The ssh server... Containers are no vm's, if you have to log in on a container running in production - you're doing something wrong - unless that container's only job is running SSH (which can be useful for example for Jenkins build slaves). Cron - again - same thing: run in a separate container and give access to the exact things your cronjob needs. That is for me the essential thing about containers: separate everything. But sure, you could treat containers as a special VM only for one service - nobody is going to stop you. I however prefer isolating every single process and explicitly telling it how to communicate with other processes. It's sane from many perspectives: security, maintainability, flexibility and speed.

detaro 11y ago
Agree in general, but you can overdo it with splitting services up. E.g. would you really run a extra container just for a cronjob that runs once a night to e-mail some data from a database? Esp. if you run on a platform where you essentially pay per container that seems like a waste.
- koffiezet 11y ago
  Most of the things I described assume you have full control over your host's OS. For stuff like you mention - you should maybe reconsider not using containers if you're on a pay-per-container platform? They are just a tool, and certainly don't fit every single use-case. Also - paying per container seems like a silly thing to do - since containers can be very short-lived. Resource-based billing would be a better fit - although that could be tricky to measure I guess.

logician76 11y ago

Container size influences memory usage, which is an important concern: http://www.stackengine.com/container-memory-usage-or-size-matters/ http://www.stackengine.com/container-memory-usage-or-size-ma...

kstenerud 11y ago
I'd be careful about drawing conclusions from those tests. We know that the number of bits in a container does not directly influence how much RAM it consumes. Therefore, there must be something the images are doing that consumes memory which is not happening in the "smaller" images. The key would be to find the culpable process or daemon(s).
- ploxiln 11y ago
  It could well be due to things like shared libraries. A larger distro will have more options enabled, causing more shared libraries to be linked into the same running processes, and thus more shared libraries to be fully loaded into memory. A smaller distro might even statically compile most things - Alpine does. If you dynamically link shared libraries, the whole library is loaded into memory to serve the process. If you statically link, only the actually used part of the library is included in the binary. Statically linked binaries can't share the library in memory between each other like dynamically linked binaries can, but if all your processes are running in separate containers, they won't share those libraries anyway (unless they're all in a VM and the rarely used "samepage merging" is enabled for the VM). Finally ... simplicity has knock-on effects. Making things simpler and smaller (not easier), and reducing the number of moving parts in the implementation, makes cleaning up more stuff easier.
  - Rapzid 11y ago
    That's an awful lot of conjecture. I'd wager that most of what you would actually be running in a container would not have its memory usage significantly affected by the presence or absence of optional shared libs.. I'm with the parent on this; such claims warrant research.

ploxiln 11y ago

Size of programs, in terms of disk, memory, cpu time, and network usage, is bloated by multiple orders of magnitude by all the confused people who think the only thing that matters is "developer productivity". Maybe 20% is worth sacrificing, maybe 50%, but 100x? 1000x? It all adds up. One really easy and relevant example, sizes of docker images for running memcached: vagrant@dockerdev:~$ sudo docker images | grep memcached memcached latest 0868b36194d3 2 weeks ago 132.2 MB sylvainlasnier/memcached latest 97a88c3744ef 13 months ago 297.4 MB ploxiln/memcached 2015-07-08 aa4a87ee2c05 5 months ago 7.453 MB (that last one is my own, the other two are the two most popular on docker hub). As another example, a co-worker recently was working with some (out-of-tree) gstreamer plugins, and the most convenient way to do so was with a docker image in which all the major gstreamer dependencies, the latest version of gstreamer, and the out-of-tree plugins were built from source. The offered image was over 10GB and 30 layers, took quite a while to download, and a surprising number of seconds to run. With just a few tweaks it was reduced to 1.1GB and a handful of layers which runs in less than a second. It was just a total lack of care for efficiency that made it 10x less efficient in every way, enough to actually reduce developer productivity. Size matters.

voltagex_ 11y ago
Are you able to share any of the tweaks that were used?
- ploxiln 11y ago
  One really easy one: write a shell script to do most of the image building (run by the Dockerfile), instead of adding a bunch of RUN directives in the Dockerfile, especially if you clean up intermediate files with a "make clean" or something. Each directive in the Dockerfile adds a layer, which adds container setup overhead, and also "locks in" all filesystem space usage at that point.
  - deleted 11y ago
    [deleted]