5 ms·

LXC, Docker, and the future of software delivery (LinuxCon)

13 points by julien421 13y ago

peterwwillis 13y ago
I still don't buy the idea of the Linux container as a universal way to do anything. It depends on your kernel [opposite of what the slides claim], it depends on your apps, it depends on your dependencies, it depends on your architecture, etc. The one phrase that's correct is "it's chroot on steroids". That is in fact exactly what it is. The exception is, it's even less portable than just a chroot environment. Docker adds extra features on top of the chroot, but that's basically its core functionality. So the first thing you have to ask yourself is: does my software need to be run in a chroot environment or a VM isolated from all other applications? If no, you very well may not need this at all for your software deployment. If anything, Docker images create a bigger burden on your deployment as you have these large images to distribute, modify, manage. Of course they built in some fancy network transmission magic to make it only copy changed parts of an image, but this is still wildly less efficient than traditional means, and you still have to fuck around with the image to make it incorporate your changes before you push it. If the big selling point is "commoditization", keep in mind that basically everyone rolls their own environment and customizes their deployment. It's the natural order of having your own architecture that fits your application. The one thing that's never going to happen, is you taking Docker images from the internet and never modifying them. This universal container system goes out the window the first minute you have to start modifying everything to fit edge cases, which is always going to happen.
- shykes 13y ago
  > This universal container system goes out the window the first minute you have to start modifying everything to fit edge cases, which is always going to happen. This is why the Dockerfile is so important. I would never use an opaque binary container as part of my production app. But I definitely would use said container if I could point to its exact source repository, modify it to my liking, and rebuild it in a repeatable way. That is possible with the docker build system :)
  - peterwwillis 13y ago
    The Dockerfile, as it currently exists, is a wrapper for shell commands, that does less than the shell does. There's no reason to use it over a plain old Bash script or Makefile.
    - shykes 13y ago
      That's not accurate. The syntax is irrelevant - if you compare it to a shell script you are missing the purpose of the dockerfile. The purpose is to make an otherwise inert source checkout runnable. Point me to a shell script that does the same thing. You will no doubt point me to some script you wrote as a sysadmin. I wrote similar scripts as a sysadmin. They are unusable to anyone outside your immediate team of sysadmins - you can't use my scripts, and I can't use yours. That's because they're ambiguous. They assume that other things are already present on the system, and don't specify how to make sure that's true. There's a README next to them explaining what they require, how to bootstrap a build and so on. Dockerfiles don't require a readme. They are not ambiguous. That's why they're useful. They also specify where to drop source files in a container, which allows you to get rid of your custom scp + flip-a-symlink ghetto deployment script.
      - peterwwillis 13y ago
        What you describe is the exact purpose behind Automake and Autoconf, which, while horrifying to use, are basically just shell scripts that make an app work on any platform. Please don't pretend that a Dockerfile is some new paradigm shift in how to deploy an app on a remote host. And really, who gives a shit about "custom" vs "standardized" argument passing? And what's with the ghetto references? You do realize under the hood your Dockerfile and my 'ghetto script' are calling the same syscalls, making the same network connections, doing the same i/o, right? Who gives a shit if my 'script' is 'ghetto' if it's ten times more user accessible, portable and backwards compatible than your proprietary format? A developer isn't going to spend more than 5 minutes looking at what the assholes in ops have given them to do their job; they learn it, they use it, they move on with writing and testing their code. Docker doesn't improve anything or provide anything you couldn't already do before. It just creates a new niche. Your whole argument for the use of Docker seems to be "we are superior, because our operations are fancier, and we have created a gold standard." Face it: it doesn't matter how the sausage gets made. Docker just prescribes one way for how the meat is ground, vs all the others that work just as well.
- shykes 13y ago
  > Docker images create a bigger burden on your deployment as you have these large images to distribute, modify, manage Compared to what, though? Of course if your environment is homogenous enough that you can express your application as, say a Jar or a gem - then you are part of the lucky few and may indeed not need docker, because you share enough context with your target infrastructure that a lot of the bits are already implicitly deployed. (In other words: someone else had to move a big-ass system image around so that you don't have to). But the typical application stack is not like that. It is heterogeneous and custom, and the only practical way to ship it reliably is to ship the entire system with it, because you've run your tests on a particular libc, postgres and ruby, built by a particular gcc, etc. In that case, your options are limited: 1) ship a VM or 2) ship system packages. And if your current options are indeed to either ship a VM or system packages - then Docker suddenly doesn't seem that heavyweight after all :)
  - peterwwillis 13y ago
    Compared to, for example, using separate systems to manage the development, change management, approval, deployment, bug tracking, revision control, and other aspects of making a change to production. The Docker image may include several of those subsystems, or cross multiple of them, and so it now needs to be flexible enough to change one or several of those parts before it can be pushed to production to fix a bug. And what production environment will it be applied to? And is it possible that indeed you will have several images that are almost the same, except for key parts that can't be easily handled by yet-another overlay? (How many overlays will you have? Will you eventually stop adding overlays and redo the image to include the fix? What else will that affect?) Complex systems require complex interaction, and Docker images do not allow for that; they are monolithic, all-or-nothing changes which can only be "modified" by either remaking the entire image (expensive), or adding another layer of overhead on top, which I don't think anyone has ever investigated to find bottlenecks or overhead problems. To put it in simpler terms: Cfengine delivering a single change on a single file to a dynamically-assigned set of nodes is a lot faster, lower overhead, and direct than deploying a Docker change.
    - contingencies 13y ago
contingencies 13y ago
Slide #40: Typical Workflow is, very much like some other aspects of docker (use of specific filesystems, use of entire filesystems within containers, etc.), a false general case that is in fact unsuited to many people's requirements. Slide #44: Docker roadmap towards 1.0 seems to dodge the question of significant differences in function with regards the apparent plan to adopt a variety of storage backends with different capabilities, use of different virtualization environments as targets, etc. I support docker as a project but I still really think you guys need to stop and ponder your architecture and goals before charging along too far. For projects to survive long term and be useful sometimes separating concerns is necessary, and I would suggest that's perhaps not being done well at present with some one-size-fits-all assumptions that are pretty anti unix philosophy (do one thing and do it well). What is the one thing? Is that really a general need? In all cases? What does a user lose with this abstraction? Rather than increasing scope, what would happen if you tried lopping those bits off entirely?

13 points by julien421 13y ago

I still don't buy the idea of the Linux container as a universal way to do anything. It depends on your kernel [opposite of what the slides claim], it depends on your apps, it depends on your dependencies, it depends on your architecture, etc. The one phrase that's correct is "it's chroot on steroids". That is in fact exactly what it is. The exception is, it's even less portable than just a chroot environment. Docker adds extra features on top of the chroot, but that's basically its core functionality. So the first thing you have to ask yourself is: does my software need to be run in a chroot environment or a VM isolated from all other applications? If no, you very well may not need this at all for your software deployment. If anything, Docker images create a bigger burden on your deployment as you have these large images to distribute, modify, manage. Of course they built in some fancy network transmission magic to make it only copy changed parts of an image, but this is still wildly less efficient than traditional means, and you still have to fuck around with the image to make it incorporate your changes before you push it. If the big selling point is "commoditization", keep in mind that basically everyone rolls their own environment and customizes their deployment. It's the natural order of having your own architecture that fits your application. The one thing that's never going to happen, is you taking Docker images from the internet and never modifying them. This universal container system goes out the window the first minute you have to start modifying everything to fit edge cases, which is always going to happen.

shykes 13y ago
> This universal container system goes out the window the first minute you have to start modifying everything to fit edge cases, which is always going to happen. This is why the Dockerfile is so important. I would never use an opaque binary container as part of my production app. But I definitely would use said container if I could point to its exact source repository, modify it to my liking, and rebuild it in a repeatable way. That is possible with the docker build system :)
- peterwwillis 13y ago
  The Dockerfile, as it currently exists, is a wrapper for shell commands, that does less than the shell does. There's no reason to use it over a plain old Bash script or Makefile.
  - shykes 13y ago
    That's not accurate. The syntax is irrelevant - if you compare it to a shell script you are missing the purpose of the dockerfile. The purpose is to make an otherwise inert source checkout runnable. Point me to a shell script that does the same thing. You will no doubt point me to some script you wrote as a sysadmin. I wrote similar scripts as a sysadmin. They are unusable to anyone outside your immediate team of sysadmins - you can't use my scripts, and I can't use yours. That's because they're ambiguous. They assume that other things are already present on the system, and don't specify how to make sure that's true. There's a README next to them explaining what they require, how to bootstrap a build and so on. Dockerfiles don't require a readme. They are not ambiguous. That's why they're useful. They also specify where to drop source files in a container, which allows you to get rid of your custom scp + flip-a-symlink ghetto deployment script.
    - peterwwillis 13y ago
      What you describe is the exact purpose behind Automake and Autoconf, which, while horrifying to use, are basically just shell scripts that make an app work on any platform. Please don't pretend that a Dockerfile is some new paradigm shift in how to deploy an app on a remote host. And really, who gives a shit about "custom" vs "standardized" argument passing? And what's with the ghetto references? You do realize under the hood your Dockerfile and my 'ghetto script' are calling the same syscalls, making the same network connections, doing the same i/o, right? Who gives a shit if my 'script' is 'ghetto' if it's ten times more user accessible, portable and backwards compatible than your proprietary format? A developer isn't going to spend more than 5 minutes looking at what the assholes in ops have given them to do their job; they learn it, they use it, they move on with writing and testing their code. Docker doesn't improve anything or provide anything you couldn't already do before. It just creates a new niche. Your whole argument for the use of Docker seems to be "we are superior, because our operations are fancier, and we have created a gold standard." Face it: it doesn't matter how the sausage gets made. Docker just prescribes one way for how the meat is ground, vs all the others that work just as well.
shykes 13y ago
> Docker images create a bigger burden on your deployment as you have these large images to distribute, modify, manage Compared to what, though? Of course if your environment is homogenous enough that you can express your application as, say a Jar or a gem - then you are part of the lucky few and may indeed not need docker, because you share enough context with your target infrastructure that a lot of the bits are already implicitly deployed. (In other words: someone else had to move a big-ass system image around so that you don't have to). But the typical application stack is not like that. It is heterogeneous and custom, and the only practical way to ship it reliably is to ship the entire system with it, because you've run your tests on a particular libc, postgres and ruby, built by a particular gcc, etc. In that case, your options are limited: 1) ship a VM or 2) ship system packages. And if your current options are indeed to either ship a VM or system packages - then Docker suddenly doesn't seem that heavyweight after all :)
- peterwwillis 13y ago
  Compared to, for example, using separate systems to manage the development, change management, approval, deployment, bug tracking, revision control, and other aspects of making a change to production. The Docker image may include several of those subsystems, or cross multiple of them, and so it now needs to be flexible enough to change one or several of those parts before it can be pushed to production to fix a bug. And what production environment will it be applied to? And is it possible that indeed you will have several images that are almost the same, except for key parts that can't be easily handled by yet-another overlay? (How many overlays will you have? Will you eventually stop adding overlays and redo the image to include the fix? What else will that affect?) Complex systems require complex interaction, and Docker images do not allow for that; they are monolithic, all-or-nothing changes which can only be "modified" by either remaking the entire image (expensive), or adding another layer of overhead on top, which I don't think anyone has ever investigated to find bottlenecks or overhead problems. To put it in simpler terms: Cfengine delivering a single change on a single file to a dynamically-assigned set of nodes is a lot faster, lower overhead, and direct than deploying a Docker change.
  - contingencies 13y ago

contingencies 13y ago

Slide #40: Typical Workflow is, very much like some other aspects of docker (use of specific filesystems, use of entire filesystems within containers, etc.), a false general case that is in fact unsuited to many people's requirements. Slide #44: Docker roadmap towards 1.0 seems to dodge the question of significant differences in function with regards the apparent plan to adopt a variety of storage backends with different capabilities, use of different virtualization environments as targets, etc. I support docker as a project but I still really think you guys need to stop and ponder your architecture and goals before charging along too far. For projects to survive long term and be useful sometimes separating concerns is necessary, and I would suggest that's perhaps not being done well at present with some one-size-fits-all assumptions that are pretty anti unix philosophy (do one thing and do it well). What is the one thing? Is that really a general need? In all cases? What does a user lose with this abstraction? Rather than increasing scope, what would happen if you tried lopping those bits off entirely?