7 ms·

Why Always Docker?

234 points by Fizzadar 11y ago

bitcointicker 11y ago
Personally the best use case for docker I have at the moment is for the testing of chef cookbooks with the Chef Test Kitchen Docker driver - https://github.com/portertech/kitchen-docker https://github.com/portertech/kitchen-docker I can write my cookbooks and almost instantly test them inside a container. You can even test on multiple platforms at the same time (Debian,Rhel etc) in parallel. You can perform integration testing using serverspec once the container has converged to the required state - http://serverspec.org/ http://serverspec.org/
- iheartmemcache 11y ago
  I've struggled to find a place to use Docker for ages simply because between LXC/VMs/BSD jails, Docker just seemed like some pretty paint on top of what I was already doing. Serverspec is _real_ interesting. Do you have more resources about how it's used, incrementally adjusted, etc?
  - bitcointicker 11y ago
    This is a nice article which describes how it is used in detail and the general workflow: http://dustinrcollins.com/chef-integration-testing-with-serverspec http://dustinrcollins.com/chef-integration-testing-with-serv... If you use it in a pipeline and it reports no errors, you can then upload your cookbook changes to chef server and bump the version. Obviously it relies on cookbook writers being disciplined, to make sure they add tests for new features they have added, but even a simple test to make sure a service is up and running and listening on a port can be enough to make sure a commit has not broken a cookbook. Opscode recently announced chef compliance which appears to build on serverspec, it is more geared towards auditing and security. https://www.chef.io/solutions/audit-compliance/ https://www.chef.io/solutions/audit-compliance/
- bazfoo 11y ago
  I found myself doing the same thing for Ansible. The problem I ran into was where I wanted to test service restarting in a systemd based environment. Older releases using sysvinit work perfectly fine.
  - eropple 11y ago
    This is a major problem with the now-in-vogue use of Docker for testing this sort of thing, yes. They aren't replacements for a virtual machine, and testing against something that doesn't even resemble the deployment environment is wacky to me.
  - arianvanp 11y ago
    This is why you should check out systemd-nspawn. It was designed especially for this use case. Also. If you're on upstart, give lxc a shot. We currently test our ansible scripts by deploying to lxc by giving each container a static IP in a bridged network to simulate our production environment. Just swap ansible inventory files. Works like a charm.
  - glenjamin 11y ago
    This gist works with test-kitchen to run systemd in a centos 7-based container. It should be simple enough to adjust to run ansible. https://gist.github.com/glenjamin/2d04e9c2a163c7848173 https://gist.github.com/glenjamin/2d04e9c2a163c7848173
- geerlingguy 11y ago
  I'm doing something similar using Docker to run tests on Ansible roles on Ubuntu 12/14 and CentOS 6/7 in parallel within Travis CI: https://github.com/geerlingguy/ansible-role-apache/blob/master/.travis.yml https://github.com/geerlingguy/ansible-role-apache/blob/mast... I've been playing around with the technique a while, but a different contributor ultimately did the final work for me, and he wrote up a great blog post on the process: http://bertvv.github.io/notes-to-self/2015/12/13/testing-ansible-roles-with-travis-ci-part-2-multi-platform-tests/ http://bertvv.github.io/notes-to-self/2015/12/13/testing-ans... While I'm yet to find a compelling reason for me to use Docker for my production environments (though the is one SaaS service I run which may benefit as Docker's warts have become much more manageable), it's excellent in a CI environment!
pekk 11y ago
Despite its problems, it is an approximation to a packaging standard which provides enough isolation to manage dependencies successfully. Does anyone not remember what it was like to fight shared library versioning conflicts? Do you want to be handling the GitHub issues attached to people screwing up that kind of thing in 2017 because their distribution or OS X package manager randomly changed?
- XorNot 11y ago
  Docker lets you setup a runtime environment by doing whatever is necessary to make it work, with the huge benefit that you don't wind up breaking the rest of the same system for running anything else.
  - Annatar 11y ago
    > Docker lets you setup a runtime environment by doing whatever is necessary to make it work Does that not qualify as "hack-it-'till-it-works", the worst kind of computer activity, especially when one is stuck trying to reverse engineer what the guy before did? Can you think of what implications "doing whatever is necessary to make it work" has for lifecycle management?
- davexunit 11y ago
  >Despite its problems, it is an approximation to a packaging standard which provides enough isolation to manage dependencies successfully. Docker is ultimately a non-solution that papers over the problems of traditional system package managers, language-specific package managers, and the myriad of software (mostly Java) that no one actually knows how to build from source. Containers do not compose. There are many runtime environments to consider, and Docker can't handle anything but containers. You need to use some other software to manage the host system at the very least. Furthermore, the container images have no useful provenance for users to inspect. It's a security nightmare. Functional package management is the real solution here. Software like GNU Guix and Nix solve real problems. They remove global state (/usr), enable reproducible builds, allow unprivileged package management, support transactional upgrades and roll backs, deduplicate software system-wide for all users, handle full-system configuration in a declarative way, eliminate the need to trust any particular provider of binaries, and more. >Does anyone not remember what it was like to fight shared library versioning conflicts? Using Docker to solve this problem is like using a sledgehammer to drive a nail.
zwischenzug 11y ago
I have some sympathy with a 'Why Docker' rant, but recently I've had experiences which has modified my view. The separation from code and data has made reasoning about my DB upgrades (postgres, mostly) much easier. The 'Docker is great for dev, not prod' view is also one I used to favour, but it's inevitably true that what begins in dev does not stay in dev. Finally, the Dockerfile limitations led me to create my own CM tool (ShutIt) so that I could configure my stateless and complex environments into code that could easily be understood and changed by the casual dev.
sz4kerto 11y ago
"These kind of systems have their own configs, be it elasticsearch.yml or my.cnf. The Dockerfile format is completely fucking useless at this kind of thing." A solution I like: use Docker and mount the config as a volume or add it to the image in an additional build step. (I.e. have a my-app:2.3.4-base and then when moving to prod, create a new image my-app:2.3.4-prod). The reason why 'Docker in production' is inevitable (as I can see it) is because it makes trivial to iterate on your whole setup, not just your application code. If you work with gcc version X and Java version y, then you change and test with new versions, then you want to version control these changes, and update them in production easily, within your normal development flow. (By inevitable, I mean that it's going to happen. Images are the new packages.)
- ownagefool 11y ago
  I don't really get it to be honest. Sure, the Dockerfile format is simple but if you need to do anything complicated, you just call another script that does it. I don't really see how it harms you? Also, I don't really understand why he wouldn't run his private registry in kubernetes if he has such a stack. I'd pretty much run everything in it.
- j_mcnally 11y ago
  I agree. The people bashing "no pun intended" docker, probably aren't using it properly.
- mkhpalm 11y ago
  "(By inevitable, I mean that it's going to happen. Images are the new packages.)" Without packages docker itself becomes mostly useless to the vast majority of people. Packages will always come first no matter how badly people try to avoid them.
- ddw 11y ago
  That isn't unique to Docker though, Ansible can do the same thing. Add Packer and Terraform and you're production-ready too.
godzillabrennus 11y ago
Docker seems to be everywhere these days. Mostly I see it in dev environments and not production though. I'm also waiting for Cal Leeming to post his annual update on Docker. Last years was memorable: http://iops.io/blog/docker-hype/ http://iops.io/blog/docker-hype/
- sleepycal 11y ago
  It's actually coming in about a week or two. I wanted to do it on 17th (exactly 1 year after) but needed more time to work on it. No spoilers, I don't want to the ruin the surprise :)
willcodeforfoo 11y ago
It's a good question, especially in the age of small static binaries with no external depdencies anyway. Even if the isolation isn't of much value, Docker is still useful as transport and storage. Getting back to the shipping container metaphor, it's easier to move things around if they are all the same. And Docker containers are a pretty good way to do that with code.
- sz4kerto 11y ago
  I don't know if this is the age of small binaries. Maybe in some industries. The artifacts we're deploying are -- partly because of various constraint, partly because of the weight of legacy - are between 25-50 MB. Oh, and they are run inside of a Java app server, that's also 100-200 MB. Ah, and that runs on a Java VM. The integration tests require a running Firefox, Chrome, V8, JVM, databases, whatever. (No, I can't replace these with a couple of command-line Unix tools just yet.)
  - lobster_johnson 11y ago
    I suspect the parent is mostly referring to Go.
    - auvrw 11y ago
      i do wonder what aspects of Go make it good for the container use-case. both Docker and the other container system mentioned at the top of the article are written in Go.
      - jacques_chester 11y ago
        > i do wonder what aspects of Go make it good for the container use-case. Same as JARs or C/C++ binaries. You can ship the compiled product to the target runtime and expect it to launch and run as-is. Languages with an interpreted nature require containers to also ship an additional runtime, plus a dependencies mechanism.
meirelles 11y ago
For my use case chef makes much more sense. I like docker, actually is a very important tool to my development environment and testing, but with many moving parts in production, some of them needing persist data, would be a hell split and manage so many app containers. I can't see how Docker would help me save time. To production LXC/KVM/nothing + chef is usually better to me.
- bitcointicker 11y ago
  You can use chef and docker together, if you really want to. Containers do provide some benefits as others have mentioned in this thread ( Packaging, avoiding conflicts, maybe even as a chroot on steroids for isolation purposes). You could have a server managed by chef which installs docker, pulls down a number of containers and then launches them, hooking them together if required. If random ports are used, chef can capture these and then hook into a load balancer to register the containers. You can even have chef build containers from a Dockerfile, to make sure they have the latest updates, tag the image and then launch them. So many options it often makes your head spin :-)
  - meirelles 11y ago
    Yes. I agree with you. Have many other good uses for Docker. But I found LXC easier, as it's possible assign a public IP and let the chef mange the iptables/service discover exactly like a VM/baremetal. Docker drops almost all caps, which is great for security, but isn't possible a container manage his own isolated iptables.
Annatar 11y ago
Why use Docker, when payload can be packaged into an OS package, and run inside of a SmartOS zone, which is a fully functional UNIX system, yet completely isolated and running at the speed of bare metal? Makes no sense to use Docker for anything if I can do configuration managment and payload deployment with OS packages inside of SmartOS zone. https://youtu.be/0T2XFSALOaU?t=1245 https://youtu.be/0T2XFSALOaU?t=1245
- j_mcnally 11y ago
  Wait.... are you saying docker is slower than bare metal? Have you used docker?
  - Annatar 11y ago
    I'm saying that a lot of people end up running Docker in a VM... why? I'm also saying that dumping a bunch of files from a developer's laptop into a Docker image is going to be a nightmare in terms of lifecycle management (how about a subsystem rollback or upgrade inside of that image?) And finally, I'm saying I see no point to Docker, if I can just make OS packages and run them inside of zones. With zones, I have a fully functional UNIX server in complete isolation and security; with Docker, I have a re-invented init which isn't really init, and if I want SSH and all the other things one normally expects of a system, I have to engineer them myself. Why would I use Docker if I can use zones in SmartOS? What does Docker buy me?
    - azernik 11y ago
      a) from a quick look at SmartOS, it looks like yet another implementation of containerization, with an option to run a full KVM if you want. And it has to run as a full OpenSolaris-based system image, instead of just being a binary installable on a Linux system (much more familiar to most developers) b) "dumping a bunch of files from a developer's laptop into a Docker image"... I'm sorry, what? I have no idea what workflow you're referring to here. WRT your specific gripes about subsystem rollback - the usual Docker best practice is to have each container run only a single subsystem, and to have images be generated by checked-in Dockerfiles based only on checked-in resources. If you need to upgrade or downgrade, you spin up a new container running a different image, fail over to it, and kill the old one. Once a container starts running it is immutable. Any of the features of a running container can be inferred just from looking at the Dockerfile(s) that built it and the connections it has to storage volumes, other containers, and the external network.
melted 11y ago
To understand scenarios under which Docker/Kubernetes/LXC are useful, you need to understand the environment in which Linux containers originated. They were born in Google data centers, where engineers may routinely need to reliably deploy tens of thousands of preemptible nodes and tie them together into a service by providing health checks, monitoring, endpoint enumeration, resource limits, isolation, and so on. Docker/Kubernetes let you do exactly that. At Google, though, Borg is run by SREs, and engineers don't have to worry about managing it, so it is quite economical to spin up even single task jobs there. Google also makes deployment much easier by using static linking, and structuring their build system outputs in such a way that they can be either easily deployed by copying over or packaging into an official, versioned deployment package (+providing command line flags through Borg config files). When tasks/jobs go away (get preempted or killed -- servers in this environment usually do not support orderly shutdown), whatever they wrote to local disk gets cleaned out, including binaries and data files. Persistent data is written to persistent, distributed storage backends, where it belongs. As you can see, most parts of this picture map nearly exactly to how Kubernetes/Docker are supposed to be used. Used in this way to manage large deployments, containers provide an unbeatable value proposition.
- jacques_chester 11y ago
  Put another way: Google has built several generations of internal PaaSes. This unblocks continuous deployment at the final step, so latency from idea to production falls from years/months/weeks to hours. Or minutes. Docker's had an interesting life: they built a PaaS, discarded the PaaS, now they're building a PaaS. Because that's what most developers actually need for their daily lives. It turns out that tinkering with V8s is a lot of fun for a lot of people, but most drivers just want to know how to turn on the car and have the same basic interface work for any workload: wheel, accelerator, brake. Disclaimer: I work for Pivotal, which is the majority donor of engineering effort to Cloud Foundry, a PaaS inspired in part by Google's experiences.
  - melted 11y ago
    I wouldn't quite go as far as to call Borg a PaaS. It's at a somewhat lower level, closer to IaaS. You get a known quantity from it in the form of a stable, tuned, stripped down underlying Linux image, plus a relatively small set of services, but you can deploy pretty much whatever the hell you want without a lot of constraints a "true" PaaS would force you to accept.
MichaelBurge 11y ago
If it's a single Go binary, I imagine you can just compile it using the Makefile. I googled and found this: https://github.com/docker/distribution https://github.com/docker/distribution It has a dockerfile that just calls make. Everyone uses the usual unix tools to build software - Docker makes some sense for deployment, but it's not really suitable for development(what if I need to add profiling? Dwarf debugging information? Tweak the optimization settings? Disassemble one of the object files? Attach gdb to a process? Strace the process to understand it?). So there'll always be the basic build instructions, and the Dockerfile will probably wrap them: Adding Docker is way more abstraction than I'm willing to deal with when debugging a tricky problem.
j_mcnally 11y ago
I for one am excited to get to the point where it doesnt make sense to dockerize everything? Docker is like violence, if its not working you aren't using enough.
euroclydon 11y ago
If the only instructions or working distribution for a piece of software is a Docker image and you're not into Docker, than that is probably not an OSS project you should use. I learned this the hard way with Bosun. I should have just avoided it totally, and saved a bunch of time.
KirinDave 11y ago
> "The Dockerfile format is completely fucking useless at this kind of thing." Right... which is why we have Docker Compose. The point of the image is to provide the code and the harness for launching it WITHOUT those assumptions. > "But wait - how do we configure these services for multiple environments (test/prod clusters)? They don't read our ENVvars, nor do they know of our internal service discovery tools." This is why docker containers are composed out of other containers. You use an elasticsearch container as a basis and extend it out with your tools to make your unique flavor of deployable search unit. This is not a new technique to anyone, as even the es docker image itself is built off another base image. I get the impression the writer of this has yet to really internalize what docker containers are. > "Tools like pyinfra and Ansible are much more suitable for this kind of work (and don't install useless crap to generate a config file)." Are they though? This is said without really any justification. To me, I'd 100% rather do it via Docker. Next to actually locking everything into one big solid lump via Nix, Docker actually gives you reproducible and reusable chunks of code with nearly infinite and modular configurability, without any care about installations stepping over one another or even library conflicts. Sure, things like an unprunable stale image cache filling up small disks is annoying. But the alternative is a continuous and inscrutable agglomeration of code and configuration files onto a box, eventually leading to total disaster. But kinda typical of someone who wants to run Go. If you're building Go you've already accepted that you'll never ship the same executable twice.
- jacques_chester 11y ago
  > This is why docker containers are composed out of other containers. You use an elasticsearch container as a basis and extend it out My limited experience is that this recreated all the worst properties of single-inheritance subclassing. In particular, a lot of subclassing for construction.
  - KirinDave 11y ago
    Docker containers shouldnt have substantial subclassing. In fact, for production work you should remake it from scratch for security. The benefit is the triviality and the orthogonality. Docker makes system components that can't interact and that can cleanly mesh with each other via simple contracts. As a means of retrofitting older software models into a new style of system assembly, it's excellent.
olalonde 11y ago
> These kind of systems have their own configs, be it elasticsearch.yml or my.cnf. The Dockerfile format is completely fucking useless at this kind of thing. confd is meant to solve this problem [0]. We use it at work to keep our bitcoind server configuration in sync with etcd [1]. Deis (the PaaS) also relies heavily on it, to generate nginx configuration files for example [2]. [0] https://github.com/kelseyhightower/confd https://github.com/kelseyhightower/confd [1] https://github.com/olalonde/coreos-bitcoind https://github.com/olalonde/coreos-bitcoind [2] https://github.com/deis/deis/tree/master/router/rootfs/etc/confd https://github.com/deis/deis/tree/master/router/rootfs/etc/c...
- SevereOverfl0w 11y ago
  I've always felt like config should be "COPY"'d into an image. Etcd/Confd looks really neat, in principle, but I feel like it's asking for trouble in terms of "immutable containers." I'm not overly familiar with the system though, so I may be misunderstanding.
willejs 11y ago
This raises some interesting points, and I agree in part. I think docker makes sense in some cases, single, compiled binaries that adhere to 12 factor app standards, I'm all for putting in a docker container. I would then run them on a PaaS, if the reasoning is sound. I am working on a project doing this currently. However, shoe horning something like a php-fpm & nginx stack in there, or anything that doesn't fit the aforementioned spec, seems and mostly is complicated. Doing something like this has caveats, becomes confusing and when you look into it, crazy. I am fed up of the hype and people thinking that docker is the silver bullet that solves all problems, and should be used for everything. Ultimately, I feel like a lot of people don't understand how docker works under the hood, and what it takes to deploy and operate applications in docker containers in production. The result of this is mostly scary. I feel like I don't have to go into details about security, entry point scripts, gosu, multi processes, logging, sketchy build processes, mounting config volumes, persistent storage, layer caches, networking, links, SDN and more. These are some of the things you have to work with with, around or avoid with docker, they are the issues people are not aware of, or don't yet understand.
- dragonsh 11y ago
  My feeling is the real things get lost in hype. I still feel given the current tool support for continuous integration docker containers are useful. But going from development to production is entirely different story. Moreover what docker calls single process (actually what they mean is single daemon, which might fork multiple processes). You can use LXD in those scenario, where you needed extra security and run unprivileged containers as lightweigh VM. So containers in itself has evolved to support different scenarios. not sure when docker will better join forces with LXD and LXC teams whose work is phenomenal and was foundation of docker.
- jacques_chester 11y ago
  > I am working on a project doing this currently. You would find OpenShift or Cloud Foundry interesting references. > However, shoe horning something like a php-fpm & nginx stack in there, or anything that doesn't fit the aforementioned spec, seems and mostly is complicated. Different stacks have different ecosystems with different patterns of historical evolution. PHP grew from a shared hosting world and pretty much everything written in PHP assumes some version of that. I used to work on Cloud Foundry buildpacks. "Shoehorning" is the entire point of the buildpacks abstraction. Ruby, Python, PHP, Java, Golang and NodeJS developers all have the same interface to a Heroku or Cloud Foundry installation.
dragonsh 11y ago
Real things get lost in hype cycle, as it is said right tool for right job. If you are looking for lightweight container VM use LXD. This let you use saltstack, ansible, chef or puppet etc. CM for system management. Same configuration can run on bare metal, VM based vagrant on desktop or cloud services like AWS, Azure, Google or many others. If you are looking for application containers running single daemmon use docker (I am not using the term process since many daemons fork multiple processes and docker still call it single process). Docker by default doesn't yet support unprivileged containers which poses security risks on multi-tenant system so can only be used with added overhead of virtual machine in AWS, Google, Azure etc. But its still good for continuous integration and development given hype resulted in integration of many tools around it.
zeta0134 11y ago
I use docker to build a particularly complex (at least to me) NDS project. I do this because I regularly develop on Windows or Linux, so do my friends, and none of us have simple working arrangements. The toolkits for NDS are a bit of a pain to set up, and I want the source for my project to be usable by anyone in the community, regardless of what platform they use, or what changes happen to the development tools over time. Thus, docker. It lets me figure out compile-time libraries and dependencies once, ever, on one platform. (Debian base.) Then magically everyone else on the team can just hit up the build script, which calls into the docker image (building it first if needed) and viola, project built. It's not nearly as efficient for compiling and making frequent changes, but in our case, the lack of complex setup and differences between build environments is worth the extra overhead. I think there's something to be said for Docker as a development tool in general; it's nice to be able to play around with development libraries without (a) cluttering up my main machine's list of installed packages, or (b) spinning up a virtual machine and sapping my workstation's RAM.
markbnj 11y ago
>> ... and I need none of Dockers scaling properties, so I'll run it direct on hardware. What is not "direct on hardware" about Docker containers? There's a bit of a misunderstanding here, and I wouldn't nitpick on it if I didn't think it betrayed something about the author's point. In some way or other he sees Docker as additional overhead, like a VM. While there obviously is _some_ overhead this isn't an accurate picture. As for the overall point of doing everything in a container, according to various sources that is exactly what Google does now, for example. The reason is that containers capture dependencies, and they make for much more fluid and manageable systems. As with most changes of this magnitude there are waves of adulation and revulsion, but overall I think this is the new world. On the elasticsearch point: you can use environment variables inside the elasticsearch.yml file, and you can set environment variables inside a container when you execute it so there is a complete pathway to pipe configuration information into the container. There are really only two things that cause an issue: discovery and disk volumes. Discovery is a problem because es uses udp multicast by default, but there are plugins that substitute other mechanisms for listing cluster members. On kubernetes/GKE we use a fabric8 plugin for this. Disk volumes are an issue just because most container platforms don't yet deal well with them. We had to roll our own solution for dynamically attaching replication controllers to GCE persistent disks, but there are some better solutions in the release pipe.
dschiptsov 11y ago
The same reason as with Why Always Java - availability bias, self-serving bias, attribution error and related mass hysteria.
NamPNQ 11y ago
> how do we configure these services for multiple environments (test/prod clusters)? Docker have config ENV varibale and VOLUME, just research about it
castell 11y ago
Run an Linux application in a container like FreeBSD jail or Sandboxie, that's what I want. I don't need the management overhead Give me Docker "light" or a good tutorial for LXC(?).

234 points by Fizzadar 11y ago

Personally the best use case for docker I have at the moment is for the testing of chef cookbooks with the Chef Test Kitchen Docker driver - https://github.com/portertech/kitchen-docker https://github.com/portertech/kitchen-docker I can write my cookbooks and almost instantly test them inside a container. You can even test on multiple platforms at the same time (Debian,Rhel etc) in parallel. You can perform integration testing using serverspec once the container has converged to the required state - http://serverspec.org/ http://serverspec.org/

iheartmemcache 11y ago
I've struggled to find a place to use Docker for ages simply because between LXC/VMs/BSD jails, Docker just seemed like some pretty paint on top of what I was already doing. Serverspec is _real_ interesting. Do you have more resources about how it's used, incrementally adjusted, etc?
- bitcointicker 11y ago
  This is a nice article which describes how it is used in detail and the general workflow: http://dustinrcollins.com/chef-integration-testing-with-serverspec http://dustinrcollins.com/chef-integration-testing-with-serv... If you use it in a pipeline and it reports no errors, you can then upload your cookbook changes to chef server and bump the version. Obviously it relies on cookbook writers being disciplined, to make sure they add tests for new features they have added, but even a simple test to make sure a service is up and running and listening on a port can be enough to make sure a commit has not broken a cookbook. Opscode recently announced chef compliance which appears to build on serverspec, it is more geared towards auditing and security. https://www.chef.io/solutions/audit-compliance/ https://www.chef.io/solutions/audit-compliance/
bazfoo 11y ago
I found myself doing the same thing for Ansible. The problem I ran into was where I wanted to test service restarting in a systemd based environment. Older releases using sysvinit work perfectly fine.
- eropple 11y ago
  This is a major problem with the now-in-vogue use of Docker for testing this sort of thing, yes. They aren't replacements for a virtual machine, and testing against something that doesn't even resemble the deployment environment is wacky to me.
- arianvanp 11y ago
  This is why you should check out systemd-nspawn. It was designed especially for this use case. Also. If you're on upstart, give lxc a shot. We currently test our ansible scripts by deploying to lxc by giving each container a static IP in a bridged network to simulate our production environment. Just swap ansible inventory files. Works like a charm.
- glenjamin 11y ago
  This gist works with test-kitchen to run systemd in a centos 7-based container. It should be simple enough to adjust to run ansible. https://gist.github.com/glenjamin/2d04e9c2a163c7848173 https://gist.github.com/glenjamin/2d04e9c2a163c7848173
geerlingguy 11y ago
I'm doing something similar using Docker to run tests on Ansible roles on Ubuntu 12/14 and CentOS 6/7 in parallel within Travis CI: https://github.com/geerlingguy/ansible-role-apache/blob/master/.travis.yml https://github.com/geerlingguy/ansible-role-apache/blob/mast... I've been playing around with the technique a while, but a different contributor ultimately did the final work for me, and he wrote up a great blog post on the process: http://bertvv.github.io/notes-to-self/2015/12/13/testing-ansible-roles-with-travis-ci-part-2-multi-platform-tests/ http://bertvv.github.io/notes-to-self/2015/12/13/testing-ans... While I'm yet to find a compelling reason for me to use Docker for my production environments (though the is one SaaS service I run which may benefit as Docker's warts have become much more manageable), it's excellent in a CI environment!

pekk 11y ago

Despite its problems, it is an approximation to a packaging standard which provides enough isolation to manage dependencies successfully. Does anyone not remember what it was like to fight shared library versioning conflicts? Do you want to be handling the GitHub issues attached to people screwing up that kind of thing in 2017 because their distribution or OS X package manager randomly changed?

XorNot 11y ago
Docker lets you setup a runtime environment by doing whatever is necessary to make it work, with the huge benefit that you don't wind up breaking the rest of the same system for running anything else.
- Annatar 11y ago
  > Docker lets you setup a runtime environment by doing whatever is necessary to make it work Does that not qualify as "hack-it-'till-it-works", the worst kind of computer activity, especially when one is stuck trying to reverse engineer what the guy before did? Can you think of what implications "doing whatever is necessary to make it work" has for lifecycle management?
davexunit 11y ago
>Despite its problems, it is an approximation to a packaging standard which provides enough isolation to manage dependencies successfully. Docker is ultimately a non-solution that papers over the problems of traditional system package managers, language-specific package managers, and the myriad of software (mostly Java) that no one actually knows how to build from source. Containers do not compose. There are many runtime environments to consider, and Docker can't handle anything but containers. You need to use some other software to manage the host system at the very least. Furthermore, the container images have no useful provenance for users to inspect. It's a security nightmare. Functional package management is the real solution here. Software like GNU Guix and Nix solve real problems. They remove global state (/usr), enable reproducible builds, allow unprivileged package management, support transactional upgrades and roll backs, deduplicate software system-wide for all users, handle full-system configuration in a declarative way, eliminate the need to trust any particular provider of binaries, and more. >Does anyone not remember what it was like to fight shared library versioning conflicts? Using Docker to solve this problem is like using a sledgehammer to drive a nail.

zwischenzug 11y ago

I have some sympathy with a 'Why Docker' rant, but recently I've had experiences which has modified my view. The separation from code and data has made reasoning about my DB upgrades (postgres, mostly) much easier. The 'Docker is great for dev, not prod' view is also one I used to favour, but it's inevitably true that what begins in dev does not stay in dev. Finally, the Dockerfile limitations led me to create my own CM tool (ShutIt) so that I could configure my stateless and complex environments into code that could easily be understood and changed by the casual dev.

sz4kerto 11y ago

"These kind of systems have their own configs, be it elasticsearch.yml or my.cnf. The Dockerfile format is completely fucking useless at this kind of thing." A solution I like: use Docker and mount the config as a volume or add it to the image in an additional build step. (I.e. have a my-app:2.3.4-base and then when moving to prod, create a new image my-app:2.3.4-prod). The reason why 'Docker in production' is inevitable (as I can see it) is because it makes trivial to iterate on your whole setup, not just your application code. If you work with gcc version X and Java version y, then you change and test with new versions, then you want to version control these changes, and update them in production easily, within your normal development flow. (By inevitable, I mean that it's going to happen. Images are the new packages.)

ownagefool 11y ago
I don't really get it to be honest. Sure, the Dockerfile format is simple but if you need to do anything complicated, you just call another script that does it. I don't really see how it harms you? Also, I don't really understand why he wouldn't run his private registry in kubernetes if he has such a stack. I'd pretty much run everything in it.
j_mcnally 11y ago
I agree. The people bashing "no pun intended" docker, probably aren't using it properly.
mkhpalm 11y ago
"(By inevitable, I mean that it's going to happen. Images are the new packages.)" Without packages docker itself becomes mostly useless to the vast majority of people. Packages will always come first no matter how badly people try to avoid them.
ddw 11y ago
That isn't unique to Docker though, Ansible can do the same thing. Add Packer and Terraform and you're production-ready too.

godzillabrennus 11y ago

Docker seems to be everywhere these days. Mostly I see it in dev environments and not production though. I'm also waiting for Cal Leeming to post his annual update on Docker. Last years was memorable: http://iops.io/blog/docker-hype/ http://iops.io/blog/docker-hype/

sleepycal 11y ago
It's actually coming in about a week or two. I wanted to do it on 17th (exactly 1 year after) but needed more time to work on it. No spoilers, I don't want to the ruin the surprise :)

willcodeforfoo 11y ago

It's a good question, especially in the age of small static binaries with no external depdencies anyway. Even if the isolation isn't of much value, Docker is still useful as transport and storage. Getting back to the shipping container metaphor, it's easier to move things around if they are all the same. And Docker containers are a pretty good way to do that with code.

sz4kerto 11y ago
I don't know if this is the age of small binaries. Maybe in some industries. The artifacts we're deploying are -- partly because of various constraint, partly because of the weight of legacy - are between 25-50 MB. Oh, and they are run inside of a Java app server, that's also 100-200 MB. Ah, and that runs on a Java VM. The integration tests require a running Firefox, Chrome, V8, JVM, databases, whatever. (No, I can't replace these with a couple of command-line Unix tools just yet.)
- lobster_johnson 11y ago
  I suspect the parent is mostly referring to Go.
  - auvrw 11y ago
    i do wonder what aspects of Go make it good for the container use-case. both Docker and the other container system mentioned at the top of the article are written in Go.
    - jacques_chester 11y ago
      > i do wonder what aspects of Go make it good for the container use-case. Same as JARs or C/C++ binaries. You can ship the compiled product to the target runtime and expect it to launch and run as-is. Languages with an interpreted nature require containers to also ship an additional runtime, plus a dependencies mechanism.

meirelles 11y ago

For my use case chef makes much more sense. I like docker, actually is a very important tool to my development environment and testing, but with many moving parts in production, some of them needing persist data, would be a hell split and manage so many app containers. I can't see how Docker would help me save time. To production LXC/KVM/nothing + chef is usually better to me.

bitcointicker 11y ago
You can use chef and docker together, if you really want to. Containers do provide some benefits as others have mentioned in this thread ( Packaging, avoiding conflicts, maybe even as a chroot on steroids for isolation purposes). You could have a server managed by chef which installs docker, pulls down a number of containers and then launches them, hooking them together if required. If random ports are used, chef can capture these and then hook into a load balancer to register the containers. You can even have chef build containers from a Dockerfile, to make sure they have the latest updates, tag the image and then launch them. So many options it often makes your head spin :-)
- meirelles 11y ago
  Yes. I agree with you. Have many other good uses for Docker. But I found LXC easier, as it's possible assign a public IP and let the chef mange the iptables/service discover exactly like a VM/baremetal. Docker drops almost all caps, which is great for security, but isn't possible a container manage his own isolated iptables.

Annatar 11y ago

Why use Docker, when payload can be packaged into an OS package, and run inside of a SmartOS zone, which is a fully functional UNIX system, yet completely isolated and running at the speed of bare metal? Makes no sense to use Docker for anything if I can do configuration managment and payload deployment with OS packages inside of SmartOS zone. https://youtu.be/0T2XFSALOaU?t=1245 https://youtu.be/0T2XFSALOaU?t=1245

j_mcnally 11y ago
Wait.... are you saying docker is slower than bare metal? Have you used docker?
- Annatar 11y ago
  I'm saying that a lot of people end up running Docker in a VM... why? I'm also saying that dumping a bunch of files from a developer's laptop into a Docker image is going to be a nightmare in terms of lifecycle management (how about a subsystem rollback or upgrade inside of that image?) And finally, I'm saying I see no point to Docker, if I can just make OS packages and run them inside of zones. With zones, I have a fully functional UNIX server in complete isolation and security; with Docker, I have a re-invented init which isn't really init, and if I want SSH and all the other things one normally expects of a system, I have to engineer them myself. Why would I use Docker if I can use zones in SmartOS? What does Docker buy me?
  - azernik 11y ago
    a) from a quick look at SmartOS, it looks like yet another implementation of containerization, with an option to run a full KVM if you want. And it has to run as a full OpenSolaris-based system image, instead of just being a binary installable on a Linux system (much more familiar to most developers) b) "dumping a bunch of files from a developer's laptop into a Docker image"... I'm sorry, what? I have no idea what workflow you're referring to here. WRT your specific gripes about subsystem rollback - the usual Docker best practice is to have each container run only a single subsystem, and to have images be generated by checked-in Dockerfiles based only on checked-in resources. If you need to upgrade or downgrade, you spin up a new container running a different image, fail over to it, and kill the old one. Once a container starts running it is immutable. Any of the features of a running container can be inferred just from looking at the Dockerfile(s) that built it and the connections it has to storage volumes, other containers, and the external network.

melted 11y ago

To understand scenarios under which Docker/Kubernetes/LXC are useful, you need to understand the environment in which Linux containers originated. They were born in Google data centers, where engineers may routinely need to reliably deploy tens of thousands of preemptible nodes and tie them together into a service by providing health checks, monitoring, endpoint enumeration, resource limits, isolation, and so on. Docker/Kubernetes let you do exactly that. At Google, though, Borg is run by SREs, and engineers don't have to worry about managing it, so it is quite economical to spin up even single task jobs there. Google also makes deployment much easier by using static linking, and structuring their build system outputs in such a way that they can be either easily deployed by copying over or packaging into an official, versioned deployment package (+providing command line flags through Borg config files). When tasks/jobs go away (get preempted or killed -- servers in this environment usually do not support orderly shutdown), whatever they wrote to local disk gets cleaned out, including binaries and data files. Persistent data is written to persistent, distributed storage backends, where it belongs. As you can see, most parts of this picture map nearly exactly to how Kubernetes/Docker are supposed to be used. Used in this way to manage large deployments, containers provide an unbeatable value proposition.

jacques_chester 11y ago
Put another way: Google has built several generations of internal PaaSes. This unblocks continuous deployment at the final step, so latency from idea to production falls from years/months/weeks to hours. Or minutes. Docker's had an interesting life: they built a PaaS, discarded the PaaS, now they're building a PaaS. Because that's what most developers actually need for their daily lives. It turns out that tinkering with V8s is a lot of fun for a lot of people, but most drivers just want to know how to turn on the car and have the same basic interface work for any workload: wheel, accelerator, brake. Disclaimer: I work for Pivotal, which is the majority donor of engineering effort to Cloud Foundry, a PaaS inspired in part by Google's experiences.
- melted 11y ago
  I wouldn't quite go as far as to call Borg a PaaS. It's at a somewhat lower level, closer to IaaS. You get a known quantity from it in the form of a stable, tuned, stripped down underlying Linux image, plus a relatively small set of services, but you can deploy pretty much whatever the hell you want without a lot of constraints a "true" PaaS would force you to accept.

MichaelBurge 11y ago

If it's a single Go binary, I imagine you can just compile it using the Makefile. I googled and found this: https://github.com/docker/distribution https://github.com/docker/distribution It has a dockerfile that just calls make. Everyone uses the usual unix tools to build software - Docker makes some sense for deployment, but it's not really suitable for development(what if I need to add profiling? Dwarf debugging information? Tweak the optimization settings? Disassemble one of the object files? Attach gdb to a process? Strace the process to understand it?). So there'll always be the basic build instructions, and the Dockerfile will probably wrap them: Adding Docker is way more abstraction than I'm willing to deal with when debugging a tricky problem.

j_mcnally 11y ago

I for one am excited to get to the point where it doesnt make sense to dockerize everything? Docker is like violence, if its not working you aren't using enough.

euroclydon 11y ago

If the only instructions or working distribution for a piece of software is a Docker image and you're not into Docker, than that is probably not an OSS project you should use. I learned this the hard way with Bosun. I should have just avoided it totally, and saved a bunch of time.

KirinDave 11y ago

> "The Dockerfile format is completely fucking useless at this kind of thing." Right... which is why we have Docker Compose. The point of the image is to provide the code and the harness for launching it WITHOUT those assumptions. > "But wait - how do we configure these services for multiple environments (test/prod clusters)? They don't read our ENVvars, nor do they know of our internal service discovery tools." This is why docker containers are composed out of other containers. You use an elasticsearch container as a basis and extend it out with your tools to make your unique flavor of deployable search unit. This is not a new technique to anyone, as even the es docker image itself is built off another base image. I get the impression the writer of this has yet to really internalize what docker containers are. > "Tools like pyinfra and Ansible are much more suitable for this kind of work (and don't install useless crap to generate a config file)." Are they though? This is said without really any justification. To me, I'd 100% rather do it via Docker. Next to actually locking everything into one big solid lump via Nix, Docker actually gives you reproducible and reusable chunks of code with nearly infinite and modular configurability, without any care about installations stepping over one another or even library conflicts. Sure, things like an unprunable stale image cache filling up small disks is annoying. But the alternative is a continuous and inscrutable agglomeration of code and configuration files onto a box, eventually leading to total disaster. But kinda typical of someone who wants to run Go. If you're building Go you've already accepted that you'll never ship the same executable twice.

jacques_chester 11y ago
> This is why docker containers are composed out of other containers. You use an elasticsearch container as a basis and extend it out My limited experience is that this recreated all the worst properties of single-inheritance subclassing. In particular, a lot of subclassing for construction.
- KirinDave 11y ago
  Docker containers shouldnt have substantial subclassing. In fact, for production work you should remake it from scratch for security. The benefit is the triviality and the orthogonality. Docker makes system components that can't interact and that can cleanly mesh with each other via simple contracts. As a means of retrofitting older software models into a new style of system assembly, it's excellent.

olalonde 11y ago

> These kind of systems have their own configs, be it elasticsearch.yml or my.cnf. The Dockerfile format is completely fucking useless at this kind of thing. confd is meant to solve this problem [0]. We use it at work to keep our bitcoind server configuration in sync with etcd [1]. Deis (the PaaS) also relies heavily on it, to generate nginx configuration files for example [2]. [0] https://github.com/kelseyhightower/confd https://github.com/kelseyhightower/confd [1] https://github.com/olalonde/coreos-bitcoind https://github.com/olalonde/coreos-bitcoind [2] https://github.com/deis/deis/tree/master/router/rootfs/etc/confd https://github.com/deis/deis/tree/master/router/rootfs/etc/c...

SevereOverfl0w 11y ago
I've always felt like config should be "COPY"'d into an image. Etcd/Confd looks really neat, in principle, but I feel like it's asking for trouble in terms of "immutable containers." I'm not overly familiar with the system though, so I may be misunderstanding.

willejs 11y ago

This raises some interesting points, and I agree in part. I think docker makes sense in some cases, single, compiled binaries that adhere to 12 factor app standards, I'm all for putting in a docker container. I would then run them on a PaaS, if the reasoning is sound. I am working on a project doing this currently. However, shoe horning something like a php-fpm & nginx stack in there, or anything that doesn't fit the aforementioned spec, seems and mostly is complicated. Doing something like this has caveats, becomes confusing and when you look into it, crazy. I am fed up of the hype and people thinking that docker is the silver bullet that solves all problems, and should be used for everything. Ultimately, I feel like a lot of people don't understand how docker works under the hood, and what it takes to deploy and operate applications in docker containers in production. The result of this is mostly scary. I feel like I don't have to go into details about security, entry point scripts, gosu, multi processes, logging, sketchy build processes, mounting config volumes, persistent storage, layer caches, networking, links, SDN and more. These are some of the things you have to work with with, around or avoid with docker, they are the issues people are not aware of, or don't yet understand.

dragonsh 11y ago
My feeling is the real things get lost in hype. I still feel given the current tool support for continuous integration docker containers are useful. But going from development to production is entirely different story. Moreover what docker calls single process (actually what they mean is single daemon, which might fork multiple processes). You can use LXD in those scenario, where you needed extra security and run unprivileged containers as lightweigh VM. So containers in itself has evolved to support different scenarios. not sure when docker will better join forces with LXD and LXC teams whose work is phenomenal and was foundation of docker.
jacques_chester 11y ago
> I am working on a project doing this currently. You would find OpenShift or Cloud Foundry interesting references. > However, shoe horning something like a php-fpm & nginx stack in there, or anything that doesn't fit the aforementioned spec, seems and mostly is complicated. Different stacks have different ecosystems with different patterns of historical evolution. PHP grew from a shared hosting world and pretty much everything written in PHP assumes some version of that. I used to work on Cloud Foundry buildpacks. "Shoehorning" is the entire point of the buildpacks abstraction. Ruby, Python, PHP, Java, Golang and NodeJS developers all have the same interface to a Heroku or Cloud Foundry installation.

dragonsh 11y ago

Real things get lost in hype cycle, as it is said right tool for right job. If you are looking for lightweight container VM use LXD. This let you use saltstack, ansible, chef or puppet etc. CM for system management. Same configuration can run on bare metal, VM based vagrant on desktop or cloud services like AWS, Azure, Google or many others. If you are looking for application containers running single daemmon use docker (I am not using the term process since many daemons fork multiple processes and docker still call it single process). Docker by default doesn't yet support unprivileged containers which poses security risks on multi-tenant system so can only be used with added overhead of virtual machine in AWS, Google, Azure etc. But its still good for continuous integration and development given hype resulted in integration of many tools around it.

zeta0134 11y ago

I use docker to build a particularly complex (at least to me) NDS project. I do this because I regularly develop on Windows or Linux, so do my friends, and none of us have simple working arrangements. The toolkits for NDS are a bit of a pain to set up, and I want the source for my project to be usable by anyone in the community, regardless of what platform they use, or what changes happen to the development tools over time. Thus, docker. It lets me figure out compile-time libraries and dependencies once, ever, on one platform. (Debian base.) Then magically everyone else on the team can just hit up the build script, which calls into the docker image (building it first if needed) and viola, project built. It's not nearly as efficient for compiling and making frequent changes, but in our case, the lack of complex setup and differences between build environments is worth the extra overhead. I think there's something to be said for Docker as a development tool in general; it's nice to be able to play around with development libraries without (a) cluttering up my main machine's list of installed packages, or (b) spinning up a virtual machine and sapping my workstation's RAM.

markbnj 11y ago

>> ... and I need none of Dockers scaling properties, so I'll run it direct on hardware. What is not "direct on hardware" about Docker containers? There's a bit of a misunderstanding here, and I wouldn't nitpick on it if I didn't think it betrayed something about the author's point. In some way or other he sees Docker as additional overhead, like a VM. While there obviously is _some_ overhead this isn't an accurate picture. As for the overall point of doing everything in a container, according to various sources that is exactly what Google does now, for example. The reason is that containers capture dependencies, and they make for much more fluid and manageable systems. As with most changes of this magnitude there are waves of adulation and revulsion, but overall I think this is the new world. On the elasticsearch point: you can use environment variables inside the elasticsearch.yml file, and you can set environment variables inside a container when you execute it so there is a complete pathway to pipe configuration information into the container. There are really only two things that cause an issue: discovery and disk volumes. Discovery is a problem because es uses udp multicast by default, but there are plugins that substitute other mechanisms for listing cluster members. On kubernetes/GKE we use a fabric8 plugin for this. Disk volumes are an issue just because most container platforms don't yet deal well with them. We had to roll our own solution for dynamically attaching replication controllers to GCE persistent disks, but there are some better solutions in the release pipe.

dschiptsov 11y ago

The same reason as with Why Always Java - availability bias, self-serving bias, attribution error and related mass hysteria.

NamPNQ 11y ago

> how do we configure these services for multiple environments (test/prod clusters)? Docker have config ENV varibale and VOLUME, just research about it

castell 11y ago

Run an Linux application in a container like FreeBSD jail or Sandboxie, that's what I want. I don't need the management overhead Give me Docker "light" or a good tutorial for LXC(?).