8 ms·

Docker for Beginners

326 points by vegasbrianc 11y ago

urs2102 11y ago
Prakhar, this is phenomenal. I'm a Columbia student as well and actually just came across your blog with this article. Hope to meet up at some point! This is definitely a great start to Docker and I like how you provided an application to allow the reader to just work through the process of deploying something. Will definitely recommend this, as Docker is something easier shown than explained. Great stuff man!
- krat0sprakhar 11y ago
  Thanks Uday! I'm planning to hold a workshop in the coming months on campus where I'll be going over this stuff in greater detail. Feel free to join in! PS: If you do come to the workshop, drop by and say Hi!
dijit 11y ago
I still don't know how to give docker an IP address or "physical" network card on my network. I was able to do this very easily with solaris zones, and even BSD jails, but every installation of docker that is any way integrated into packages seems to be unable to do this. perhaps I'm simply not using the right google search terms.
- mrala 11y ago
  https://docs.docker.com/engine/userguide/networking/dockernetworks/ https://docs.docker.com/engine/userguide/networking/dockerne...
  - dijit 11y ago
    This doesn't really help, exposing network ports is not the same as giving a container a seperate IP address.
    - mrala 11y ago
      Is this what you're looking for? https://github.com/docker/docker/pull/19001 https://github.com/docker/docker/pull/19001
      - dijit 11y ago
        Yep. Glad to see it was merged. I wonder when it will trickle down into my distro of choice. :)
        Patrick_Devine 11y ago
        You should check out get.docker.com. Most of the distro versions are pretty old/broken.
- general_failure 11y ago
  Is your intention to expose the ports of a docker container? If you want to do this search for "expose" ports. The docker run option is -p and -P for this. Or to create a subnet with custom IP for the docker daemon's bridge network? Use -b option to docker daemon. https://docs.docker.com/engine/userguide/networking/ https://docs.docker.com/engine/userguide/networking/ is a good place to start. If you have a specific question, feel free to ask and I can try to answer.
  - dijit 11y ago
    no, that's not my intention, my intention is to not have to deal with iptables mangling packets and adding netfilter tags to everything. not to mention port collisions with things that must run on predefined ports (think SMTP or pesky applications that keep redirecting you back to port 80) I'm looking to expose 'an IP' similar to a bridged/open network in KVM.
    - jdoliner 11y ago
      Would doing --network=host do this? It makes the container use the hosts networking stack so there's no funny business.
      - takee 11y ago
        Yes it would help here but would also expose new security loopholes.
- cgb_ 11y ago
  I've found pipework (https://github.com/jpetazzo/pipework https://github.com/jpetazzo/pipework) still to be the easiest way to manage networking in containers when you know what you want. Start up your container with --net=none and then something like: pipework mybr01 mycontainername 1.1.1.3/24@1.1.1.1 Docker 1.9's enhanced networking functionality is supposed to make this easier, although I've had a hard time understanding how to use it on my preferred RHEL/CentOS platform (the doco I've read either glosses over details or assumes you use docker-machine & swarm, neither of which I want to use in a simple deployment). Weave is supposed to make this easy to (and offers some simplicity over docker 1.9's requirement for a clustered key-value store), but I got frustrated at their implicit iptables nat rule for networks created by default and haven't worked out how to stop that.
- AnythingForDogs 11y ago
  Take a look at Calico - https://github.com/projectcalico/calico-containers https://github.com/projectcalico/calico-containers It's a networking solution that uses BGP to connect containers across multiple hosts, and so can very easily integrate with existing infrastructure.
- NetStrikeForce 11y ago
  I haven't tried it yet with Docker, but I'm starting a networking service (pretty much SoftEther as a service) that might work on your case. The idea is that your Docker containers will connect to a central server (443/TCP outgoing traffic) and have a virtual private network among them through this "hub", so they've got full access to each other. This is a layer 2 network and in my offering I have DHCP running by default to simplify things (100.64.0.0/24... naughty, I know :)). The communications are encrypted, so effectively you've got a sort of Virtual Private Network between your containers. As I said, I haven't tried yet with Docker, but it's worth a shot. My service simplifies the process of getting up and running. https://wormhole.network https://wormhole.network
- clinta 11y ago
  I've had the same struggle. I'm moving my network to use a vxlan overlay for VMs so I ended up building a vxlan driver for docker networking which lets me do the same thing for containers. That project is here: https://github.com/clinta/docker-vxlan-plugin https://github.com/clinta/docker-vxlan-plugin Docker seems really intent on NATing containers behind the host, which IMO is not acceptable from a security perspective when I want to firewall outbound access based on a containers role. Kubernetes is another option which creates a unique IP per container pod on the same network as the host. Not as flexible as a vxlan approach where containers can be micro-segmented into specific networks, but more like BSD jails that you're used to.
- NetStrikeForce 11y ago
  I did this now: https://github.com/pjperez/docker-wormhole https://github.com/pjperez/docker-wormhole Example Docker images for iPerf client and server: https://github.com/pjperez/docker-iperfserver https://github.com/pjperez/docker-iperfserver https://github.com/pjperez/docker-iperfclient https://github.com/pjperez/docker-iperfclient Edit to clarify: You can deploy each of the above images on different hosts located on different providers. There's no need for the hosts to have visibility between each other at all.
is_this_tinder 11y ago
awesome - very well written m8
nemothekid 11y ago
While the technical tutorial is good, I can't say anyone should choose Docker for the reasons outlines in the "Why should I use it?" section. "Because its popular" will only give you a headache when you are trying to deploy your simple rails app.
- krat0sprakhar 11y ago
  Great point! While the popular aspect is true to some extent, I've found that for my side projects (I'm still a student) Docker has vastly decreased the pain in deploying my projects. I used to have "but it works on my machine" moments but now using the same container in production is super awesome! I'm sure Docker is not a panacea to all your infrastructure problems but it surely is a worthwhile tool to learn :)
  - nemothekid 11y ago
    >I've found that for my side projects (I'm still a student) Docker has vastly decreased the pain in deploying my projects. I used to have "but it works on my machine" moments but now using the same container in production is super awesome! Thats a far stronger reason to use Docker - as I'm sure people have wrestled with this issue when trying to use Capistrano/Grunt/Ansible to deploy as well.
    - k__ 11y ago
      But "It's popular" seems to be one of the main reasons. Otherwise more people would use Nix.
      - tayo42 11y ago
        How does nix offer any of the advantages that docker does? Isn't it just another way to do config management? Plus the added complexity of using a very niche os. It seems like it's not popular for a reason...
        11y ago
krat0sprakhar 11y ago
Hi everyone, Author here. There are a bunch of awesome[0] tutorials on Docker so why another one? Well, my motivation was to have a guide (for myself and for others) on how to deploy dockerized apps on the cloud. So in this tutorial, apart from giving an intro to docker, I demonstrate how to use Elastic Beanstalk for single-container and ECS for multi-container deployments. Here's are the two apps we deploy on AWS for example 1. Catnip - A simple flask app (single-container): http://catnip.elasticbeanstalk.com/ http://catnip.elasticbeanstalk.com/ 2. Foodtrucks - A simple app to discover foodtrucks in SF (Flask + Elasticsearch in multi-containers): http://sf-foodtrucks.xyz/ http://sf-foodtrucks.xyz/ I'm new to Docker myself so I'm sure I've made mistakes. Let me know if you have ideas on how to improve this! [0] - http://docker.atbaker.me/ http://docker.atbaker.me/
- nzoschke 11y ago
  Thanks, this is really great! I learned a lot about how to use Elastic Beanstalk and the ECS cli from this guide.
- alainchabat 11y ago
  Awesome! I will try ECS :D 2 questions about ECS : I have few script I'd like to run on ECS. I created many images with the same dependencies, only the script code change. I was thinking attaching the script with volume . Is that a good way to do it ? Is ECS handle volume well ?
fideloper 11y ago
ECS is really the only way I'd go with Docker - So far in my experience, Docker in production is really a lot harder than the casual developer / devops person realizes based on the "getting started with docker" tutorials. My conclusion with Docker is that, in general™, you really need to have a justifiable reason to go whole-hog into Docker, especially if you're not on AWS / considering ECS. I'm glad the article covers ECS, as it makes a lot of the scheduling / config issues simpler!
- krat0sprakhar 11y ago
  Totally agree! To be honest I'm still a bit wary of using ECS via the web console. Too many (new) terms to understand! Thankfully the folks at AWS were smart enough to make a CLI that understands docker-compose. So that's how I ended up using it.
  - nzoschke 11y ago
    We're working on a whole ECS toolkit at http://convox.com/ http://convox.com/. There are similarities to the ECS CLI, but we are going for the whole "batteries included" approach. Builds, Logs, cluster and app scaling, encrypted environment, and more.
- deleted 11y ago
  [deleted]
- gtaylor 11y ago
  > ECS is really the only way I'd go with Docker ECS is AWS-specific, which is perfectly fine for some. But Kubernetes has been amazing for us. It abstracts many of the differences between AWS/Google Cloud, it's open source, and is far more powerful and flexible. The only issue right now is that setting the cluster up involves running some shell scripts (yuck). We use Google Container Engine (hosted Kubernetes on Google Cloud), so we don't have to deal with that, but the option is there should we ever need to go multi-cloud. Figured I'd toss that out there for anyone struggling with ECS (it can be a bit rigid) or keeping an eye on things beyond AWS. Kubernetes is still young and rough in areas, but it is a nice, opinionated way to orchestrate containers.
deleted 11y ago
[deleted]
yeukhon 11y ago
neat article. I didn't know now you can run Docker on EB!! I recommend start with Docker Machine (https://docs.docker.com/machine/ https://docs.docker.com/machine/) Another well-written beginner tutorial (but with a few hardcoded urls): http://stackengine.com/docker-101-01-docker-development-environments/ http://stackengine.com/docker-101-01-docker-development-envi...
siquick 11y ago
As a Docker noob, it would be so useful if there was a list of Docker use-cases that someone could link to?
- olalonde 11y ago
  If you're working on multiple projects simultaneously (e.g. multiple microservices within a company or multiple client projects if you are a freelancer), Docker is a godsend to quickly get your development environment setup. All you need to declare your "infrastructure" (e.g. PostgreSQL, Redis, etc.) in a docker-compose.yml and `docker-compose up` will get you up and running in no time.
- krat0sprakhar 11y ago
  At the risk of oversimplifying, you can use Docker wherever you might want to use a full-fledged VM. You can use it to run a job, host a webserver or any other scenario that you can think of. What makes Docker (rather containers) different from a VM is the way it sandboxes all your application's dependencies into an isolated sandbox. So let's say you have a python app that for relies on a dependency that needs C bindings (e.g. ImageMagick). Instead of running `./app.py` freshly downloaded from some Git <repo>, you would run `docker run <repo> ./app.py`. In the former case, you would need to care of, say, the C dependencies. In the second case, they are packaged in the image that Docker will download from <repo> prior to run the ./app.py process in it. (Note that the two <repo> are not the same things. One is a Git repo, the other is a Docker repo - called an Image.) Think of process of the building a container as taking a snapshot of the entire OS (such as VM images) but w/o the high overhead of running these images. Feel free to reach out to me if you need more clarification!
- drey 11y ago
  https://www.docker.com/products/use-cases https://www.docker.com/products/use-cases
systemz 11y ago
I still prefer using automation like ansible or saltstack over docker. You can't run it on very popular OpenVZ VPS and it's another useless layer of abstraction with security holes in it.
fpoling 11y ago
The weak part of Docker is volume/data management and it is interesting that the article almost never mentioned it. There are solutions that attach networking storage to containers, but that just adds a latency to the application. If one wants access to the fast local storage, then Docker requires significant extra efforts to set things up.
- stephenitis 11y ago
  I think you can do exactly what you are asking for with the host machine's local storage with docker today. Where do these docs fall short for you? https://docs.docker.com/engine/userguide/dockervolumes/ https://docs.docker.com/engine/userguide/dockervolumes/ I would first start with the use case, checkout what came out in docker 1.9 and the read up on what the various volume plugins provide. (I work for ClusterHQ and we read and hear a lot of feedback around the topic, also )
  - fpoling 11y ago
    My point is that most Docker tutorials and documentation underestimates the complexity of persistent data management. Docker is not going to solve that. However, once one takes care about the data, Docker is really nice for service isolation and deployment.
    - musha68k 11y ago
      The guys over at the great http://www.thecloudcast.net http://www.thecloudcast.net have been regularly following this topic recently. It's a treasure trove of devops information, anyone interested should subscribe IMHO.
musha68k 11y ago
Docker does great marketing but to my knowledge I can't use the toolbox to effortlessly (without Vagrant) set up a non-trivial microservice development flow on my Mac yet. On the deployment side there are also still lots of inconsistencies between the different tools but I can see it becoming the go-to pragmatic way to bootstrap your private "cloud" very soon. Again deployment is only part of the solution and I wish Docker would hire more good people to work on different development "best-practices" for Linux as well as OSX. If Docker eventually wants to become a major service provider they should complement that kind of tooling with the same vigorous level of documentation and blogging as Heroku, Digital Ocean, Codeship and the likes come up with consistently.
- benjaminwootton 11y ago
  Docker have a lot more than just the containers - they acquired Tutum for a cloud based deployment workflow and Universal Control Plane for on prem. The Docker hub and DTR also help with build and deployments of containers. It's a set of tooling and an ecosystem and not just the container technology.
  - musha68k 11y ago
    Thank you, I'm familiar with the various "merges and acquisitions" (fig, tutum cloud etc) but my point was that there are still a lot of unfulfilled promises and that Docker's admittedly great marketing can be misleading.
- fpoling 11y ago
  On my Mac I run docker under CoreOS VM, the same OS I use on production, and use lsyncd to transparently synchronize all the files that I touch in the editor into the VM. Works like charm.
  - musha68k 11y ago
    I bet you would help not only me but lots of people if you'd share that workflow. Which hypervisor are you using? Is Vagrant still part of that setup (a tool that should become obsolete with a pure docker development approach IMHO)? I didn't find anything like that on either the Docker nor CoreOS official docs unfortunately. Both of those companies are fighting for territory in this super lucrative space, one of them should provide it without relying on the greater community (like one of you or me doing what's arguably their job for free). Edit: I'm sorry if this comes off as the typical abrasive comment but I've been working with both those technologies for more than a year now and mind you not alone. A couple of good developers and sysadmins I work with are also still trying to figure out how to go about all of these issues and what I see is a big disconnect between what gets advertised vs. where we are at this point. As somebody who did FreeBSD jails and OpenVZ container system administration as well as general "distributed systems" development myself for many years I also admit to painfully miss the amazing simplicity of creating a monolith and effortlessly deploying it to Heroku. It's something I've become used to over the last couple of years and I miss it, even though I myself find the promise of the current "microservice" trend very interesting as well.
MichaelBurge 11y ago
I'm always surprised to see a comparison to Virtual Machines. The containers seem more like an enhanced chroot jail than a CPU emulator, and I've always used those for the similar purpose of isolating a tricky build environment. They even have some of the same restrictions(Docker needs root, as does chroot; they both work by making system calls lie to the process). Whenever I hear a comparison with VMs, I wonder for a second, "Wait, is there some clever way to invoke the virtualization instructions without evicting references to the OS from the CPU's context to provide isolation without a separate guest OS?"

326 points by vegasbrianc 11y ago

Prakhar, this is phenomenal. I'm a Columbia student as well and actually just came across your blog with this article. Hope to meet up at some point! This is definitely a great start to Docker and I like how you provided an application to allow the reader to just work through the process of deploying something. Will definitely recommend this, as Docker is something easier shown than explained. Great stuff man!

krat0sprakhar 11y ago
Thanks Uday! I'm planning to hold a workshop in the coming months on campus where I'll be going over this stuff in greater detail. Feel free to join in! PS: If you do come to the workshop, drop by and say Hi!

dijit 11y ago

I still don't know how to give docker an IP address or "physical" network card on my network. I was able to do this very easily with solaris zones, and even BSD jails, but every installation of docker that is any way integrated into packages seems to be unable to do this. perhaps I'm simply not using the right google search terms.

mrala 11y ago
https://docs.docker.com/engine/userguide/networking/dockernetworks/ https://docs.docker.com/engine/userguide/networking/dockerne...
- dijit 11y ago
  This doesn't really help, exposing network ports is not the same as giving a container a seperate IP address.
  - mrala 11y ago
    Is this what you're looking for? https://github.com/docker/docker/pull/19001 https://github.com/docker/docker/pull/19001
    - dijit 11y ago
      Yep. Glad to see it was merged. I wonder when it will trickle down into my distro of choice. :)
      - Patrick_Devine 11y ago
        You should check out get.docker.com. Most of the distro versions are pretty old/broken.
general_failure 11y ago
Is your intention to expose the ports of a docker container? If you want to do this search for "expose" ports. The docker run option is -p and -P for this. Or to create a subnet with custom IP for the docker daemon's bridge network? Use -b option to docker daemon. https://docs.docker.com/engine/userguide/networking/ https://docs.docker.com/engine/userguide/networking/ is a good place to start. If you have a specific question, feel free to ask and I can try to answer.
- dijit 11y ago
  no, that's not my intention, my intention is to not have to deal with iptables mangling packets and adding netfilter tags to everything. not to mention port collisions with things that must run on predefined ports (think SMTP or pesky applications that keep redirecting you back to port 80) I'm looking to expose 'an IP' similar to a bridged/open network in KVM.
  - jdoliner 11y ago
    Would doing --network=host do this? It makes the container use the hosts networking stack so there's no funny business.
    - takee 11y ago
      Yes it would help here but would also expose new security loopholes.
cgb_ 11y ago
I've found pipework (https://github.com/jpetazzo/pipework https://github.com/jpetazzo/pipework) still to be the easiest way to manage networking in containers when you know what you want. Start up your container with --net=none and then something like: pipework mybr01 mycontainername 1.1.1.3/24@1.1.1.1 Docker 1.9's enhanced networking functionality is supposed to make this easier, although I've had a hard time understanding how to use it on my preferred RHEL/CentOS platform (the doco I've read either glosses over details or assumes you use docker-machine & swarm, neither of which I want to use in a simple deployment). Weave is supposed to make this easy to (and offers some simplicity over docker 1.9's requirement for a clustered key-value store), but I got frustrated at their implicit iptables nat rule for networks created by default and haven't worked out how to stop that.
AnythingForDogs 11y ago
Take a look at Calico - https://github.com/projectcalico/calico-containers https://github.com/projectcalico/calico-containers It's a networking solution that uses BGP to connect containers across multiple hosts, and so can very easily integrate with existing infrastructure.
NetStrikeForce 11y ago
I haven't tried it yet with Docker, but I'm starting a networking service (pretty much SoftEther as a service) that might work on your case. The idea is that your Docker containers will connect to a central server (443/TCP outgoing traffic) and have a virtual private network among them through this "hub", so they've got full access to each other. This is a layer 2 network and in my offering I have DHCP running by default to simplify things (100.64.0.0/24... naughty, I know :)). The communications are encrypted, so effectively you've got a sort of Virtual Private Network between your containers. As I said, I haven't tried yet with Docker, but it's worth a shot. My service simplifies the process of getting up and running. https://wormhole.network https://wormhole.network
clinta 11y ago
I've had the same struggle. I'm moving my network to use a vxlan overlay for VMs so I ended up building a vxlan driver for docker networking which lets me do the same thing for containers. That project is here: https://github.com/clinta/docker-vxlan-plugin https://github.com/clinta/docker-vxlan-plugin Docker seems really intent on NATing containers behind the host, which IMO is not acceptable from a security perspective when I want to firewall outbound access based on a containers role. Kubernetes is another option which creates a unique IP per container pod on the same network as the host. Not as flexible as a vxlan approach where containers can be micro-segmented into specific networks, but more like BSD jails that you're used to.
NetStrikeForce 11y ago
I did this now: https://github.com/pjperez/docker-wormhole https://github.com/pjperez/docker-wormhole Example Docker images for iPerf client and server: https://github.com/pjperez/docker-iperfserver https://github.com/pjperez/docker-iperfserver https://github.com/pjperez/docker-iperfclient https://github.com/pjperez/docker-iperfclient Edit to clarify: You can deploy each of the above images on different hosts located on different providers. There's no need for the hosts to have visibility between each other at all.

is_this_tinder 11y ago

awesome - very well written m8

nemothekid 11y ago

While the technical tutorial is good, I can't say anyone should choose Docker for the reasons outlines in the "Why should I use it?" section. "Because its popular" will only give you a headache when you are trying to deploy your simple rails app.

krat0sprakhar 11y ago
Great point! While the popular aspect is true to some extent, I've found that for my side projects (I'm still a student) Docker has vastly decreased the pain in deploying my projects. I used to have "but it works on my machine" moments but now using the same container in production is super awesome! I'm sure Docker is not a panacea to all your infrastructure problems but it surely is a worthwhile tool to learn :)
- nemothekid 11y ago
  >I've found that for my side projects (I'm still a student) Docker has vastly decreased the pain in deploying my projects. I used to have "but it works on my machine" moments but now using the same container in production is super awesome! Thats a far stronger reason to use Docker - as I'm sure people have wrestled with this issue when trying to use Capistrano/Grunt/Ansible to deploy as well.
  - k__ 11y ago
    But "It's popular" seems to be one of the main reasons. Otherwise more people would use Nix.
    - tayo42 11y ago
      How does nix offer any of the advantages that docker does? Isn't it just another way to do config management? Plus the added complexity of using a very niche os. It seems like it's not popular for a reason...
      - 11y ago

krat0sprakhar 11y ago

Hi everyone, Author here. There are a bunch of awesome[0] tutorials on Docker so why another one? Well, my motivation was to have a guide (for myself and for others) on how to deploy dockerized apps on the cloud. So in this tutorial, apart from giving an intro to docker, I demonstrate how to use Elastic Beanstalk for single-container and ECS for multi-container deployments. Here's are the two apps we deploy on AWS for example 1. Catnip - A simple flask app (single-container): http://catnip.elasticbeanstalk.com/ http://catnip.elasticbeanstalk.com/ 2. Foodtrucks - A simple app to discover foodtrucks in SF (Flask + Elasticsearch in multi-containers): http://sf-foodtrucks.xyz/ http://sf-foodtrucks.xyz/ I'm new to Docker myself so I'm sure I've made mistakes. Let me know if you have ideas on how to improve this! [0] - http://docker.atbaker.me/ http://docker.atbaker.me/

nzoschke 11y ago
Thanks, this is really great! I learned a lot about how to use Elastic Beanstalk and the ECS cli from this guide.
alainchabat 11y ago
Awesome! I will try ECS :D 2 questions about ECS : I have few script I'd like to run on ECS. I created many images with the same dependencies, only the script code change. I was thinking attaching the script with volume . Is that a good way to do it ? Is ECS handle volume well ?

fideloper 11y ago

ECS is really the only way I'd go with Docker - So far in my experience, Docker in production is really a lot harder than the casual developer / devops person realizes based on the "getting started with docker" tutorials. My conclusion with Docker is that, in general™, you really need to have a justifiable reason to go whole-hog into Docker, especially if you're not on AWS / considering ECS. I'm glad the article covers ECS, as it makes a lot of the scheduling / config issues simpler!

krat0sprakhar 11y ago
Totally agree! To be honest I'm still a bit wary of using ECS via the web console. Too many (new) terms to understand! Thankfully the folks at AWS were smart enough to make a CLI that understands docker-compose. So that's how I ended up using it.
- nzoschke 11y ago
  We're working on a whole ECS toolkit at http://convox.com/ http://convox.com/. There are similarities to the ECS CLI, but we are going for the whole "batteries included" approach. Builds, Logs, cluster and app scaling, encrypted environment, and more.
deleted 11y ago
[deleted]
gtaylor 11y ago
> ECS is really the only way I'd go with Docker ECS is AWS-specific, which is perfectly fine for some. But Kubernetes has been amazing for us. It abstracts many of the differences between AWS/Google Cloud, it's open source, and is far more powerful and flexible. The only issue right now is that setting the cluster up involves running some shell scripts (yuck). We use Google Container Engine (hosted Kubernetes on Google Cloud), so we don't have to deal with that, but the option is there should we ever need to go multi-cloud. Figured I'd toss that out there for anyone struggling with ECS (it can be a bit rigid) or keeping an eye on things beyond AWS. Kubernetes is still young and rough in areas, but it is a nice, opinionated way to orchestrate containers.

deleted 11y ago

[deleted]

yeukhon 11y ago

neat article. I didn't know now you can run Docker on EB!! I recommend start with Docker Machine (https://docs.docker.com/machine/ https://docs.docker.com/machine/) Another well-written beginner tutorial (but with a few hardcoded urls): http://stackengine.com/docker-101-01-docker-development-environments/ http://stackengine.com/docker-101-01-docker-development-envi...

siquick 11y ago

As a Docker noob, it would be so useful if there was a list of Docker use-cases that someone could link to?

olalonde 11y ago
If you're working on multiple projects simultaneously (e.g. multiple microservices within a company or multiple client projects if you are a freelancer), Docker is a godsend to quickly get your development environment setup. All you need to declare your "infrastructure" (e.g. PostgreSQL, Redis, etc.) in a docker-compose.yml and `docker-compose up` will get you up and running in no time.
krat0sprakhar 11y ago
At the risk of oversimplifying, you can use Docker wherever you might want to use a full-fledged VM. You can use it to run a job, host a webserver or any other scenario that you can think of. What makes Docker (rather containers) different from a VM is the way it sandboxes all your application's dependencies into an isolated sandbox. So let's say you have a python app that for relies on a dependency that needs C bindings (e.g. ImageMagick). Instead of running `./app.py` freshly downloaded from some Git <repo>, you would run `docker run <repo> ./app.py`. In the former case, you would need to care of, say, the C dependencies. In the second case, they are packaged in the image that Docker will download from <repo> prior to run the ./app.py process in it. (Note that the two <repo> are not the same things. One is a Git repo, the other is a Docker repo - called an Image.) Think of process of the building a container as taking a snapshot of the entire OS (such as VM images) but w/o the high overhead of running these images. Feel free to reach out to me if you need more clarification!
drey 11y ago
https://www.docker.com/products/use-cases https://www.docker.com/products/use-cases

systemz 11y ago

I still prefer using automation like ansible or saltstack over docker. You can't run it on very popular OpenVZ VPS and it's another useless layer of abstraction with security holes in it.

fpoling 11y ago

The weak part of Docker is volume/data management and it is interesting that the article almost never mentioned it. There are solutions that attach networking storage to containers, but that just adds a latency to the application. If one wants access to the fast local storage, then Docker requires significant extra efforts to set things up.

stephenitis 11y ago
I think you can do exactly what you are asking for with the host machine's local storage with docker today. Where do these docs fall short for you? https://docs.docker.com/engine/userguide/dockervolumes/ https://docs.docker.com/engine/userguide/dockervolumes/ I would first start with the use case, checkout what came out in docker 1.9 and the read up on what the various volume plugins provide. (I work for ClusterHQ and we read and hear a lot of feedback around the topic, also )
- fpoling 11y ago
  My point is that most Docker tutorials and documentation underestimates the complexity of persistent data management. Docker is not going to solve that. However, once one takes care about the data, Docker is really nice for service isolation and deployment.
  - musha68k 11y ago
    The guys over at the great http://www.thecloudcast.net http://www.thecloudcast.net have been regularly following this topic recently. It's a treasure trove of devops information, anyone interested should subscribe IMHO.

musha68k 11y ago

Docker does great marketing but to my knowledge I can't use the toolbox to effortlessly (without Vagrant) set up a non-trivial microservice development flow on my Mac yet. On the deployment side there are also still lots of inconsistencies between the different tools but I can see it becoming the go-to pragmatic way to bootstrap your private "cloud" very soon. Again deployment is only part of the solution and I wish Docker would hire more good people to work on different development "best-practices" for Linux as well as OSX. If Docker eventually wants to become a major service provider they should complement that kind of tooling with the same vigorous level of documentation and blogging as Heroku, Digital Ocean, Codeship and the likes come up with consistently.

benjaminwootton 11y ago
Docker have a lot more than just the containers - they acquired Tutum for a cloud based deployment workflow and Universal Control Plane for on prem. The Docker hub and DTR also help with build and deployments of containers. It's a set of tooling and an ecosystem and not just the container technology.
- musha68k 11y ago
  Thank you, I'm familiar with the various "merges and acquisitions" (fig, tutum cloud etc) but my point was that there are still a lot of unfulfilled promises and that Docker's admittedly great marketing can be misleading.
fpoling 11y ago
On my Mac I run docker under CoreOS VM, the same OS I use on production, and use lsyncd to transparently synchronize all the files that I touch in the editor into the VM. Works like charm.
- musha68k 11y ago
  I bet you would help not only me but lots of people if you'd share that workflow. Which hypervisor are you using? Is Vagrant still part of that setup (a tool that should become obsolete with a pure docker development approach IMHO)? I didn't find anything like that on either the Docker nor CoreOS official docs unfortunately. Both of those companies are fighting for territory in this super lucrative space, one of them should provide it without relying on the greater community (like one of you or me doing what's arguably their job for free). Edit: I'm sorry if this comes off as the typical abrasive comment but I've been working with both those technologies for more than a year now and mind you not alone. A couple of good developers and sysadmins I work with are also still trying to figure out how to go about all of these issues and what I see is a big disconnect between what gets advertised vs. where we are at this point. As somebody who did FreeBSD jails and OpenVZ container system administration as well as general "distributed systems" development myself for many years I also admit to painfully miss the amazing simplicity of creating a monolith and effortlessly deploying it to Heroku. It's something I've become used to over the last couple of years and I miss it, even though I myself find the promise of the current "microservice" trend very interesting as well.

MichaelBurge 11y ago

I'm always surprised to see a comparison to Virtual Machines. The containers seem more like an enhanced chroot jail than a CPU emulator, and I've always used those for the similar purpose of isolating a tricky build environment. They even have some of the same restrictions(Docker needs root, as does chroot; they both work by making system calls lie to the process). Whenever I hear a comparison with VMs, I wonder for a second, "Wait, is there some clever way to invoke the virtualization instructions without evicting references to the OS from the CPU's context to provide isolation without a separate guest OS?"