5 ms·

There are many ways of using Docker and obviously different companies could come up with their own business cases for adopting the technology. So let me focus o

by osipov 11y ago

There are many ways of using Docker and obviously different companies could come up with their own business cases for adopting the technology. So let me focus on one scenario and we can talk about whether it makes sense for your environment. Software engineering is often difficult because programmers have to deal with inconsistent environments for development and for production execution of their products. Due to mismatches between these environments, developers often found bugs that surface in one environment but not in another. Hardware based virtualization (VMWare, HyperV, etc) helped with the inconsistency issue because it enabled developers to create dev/test environments that could later be replicated into production. However this category of virtualization requires more computational resources (esp. storage) than operating system virtualization like LXC used by Docker. In addition to requiring less resources than hardware virtualization, Docker defined a convenient container specification format (Dockerfile) and a way to share these specifications (DockerHub). When used together, these Docker technologies accelerate the process of defining a consistent environment for both development and production in your application. Dockerfiles are easy to maintain and help reduce the need for a dedicated operations team for your application. In buzzword speak, your team can become more "DevOps". Docker, by the virtue of relying on a thinner virtualization layer than hardware hypervisors like VMWare, also has higher performance for I/O intensive operations. When used on bare metal hardware, you going to be able to get better performance for many databases in Docker containers compared to databases running in a virtual machine guest. So to recap, Docker can help you - maintain consistent dev/test/prod environments - use less resources than hardware virtualization - free up the time your team spends on dev to ops handoffs - improve your app I/O performance compared to running in a hardware virtualization based virtual machine guest However if you are using AWS, note that Docker Container Service available from Amazon actually doesn't give you Docker on bare metal. That's because Docker Containers run in AWS virtual machine guests virtualized by Xen hypervisor. So with AWS you are paying a penalty in resources and performance when using Docker.

takeda 11y ago
Great, but what are the benefits of running Docker in AWS? You are still running VMs and you are being charged for running them. With Docker you are simply putting yet another layer of complexity, because now you have to run more beefier VMs, you now have problem with network communication between containers running on different hosts. So you will most likely need to use overlay network. You also decrease resiliency, because now when AWS terminated a single VM, all apps running on that node suddenly disappear. I also don't get the argument about running the same container in dev/test/prod. For example my company is working on going Docker and one of the problem with these environments is that app running there has different configuration. So the idea to solve it is to create three different versions of the same container. Genius! But now are you really running the same thing in dev/test/prod? How is it different to what we did in the past? Especially that before Docker through our continuous delivery we actually were using exact same artifact on machines set up with chef that were configured the same way as in prod, while with Docker now we plan to use three different containers.
- osipov 11y ago
  >what are the benefits of running Docker in AWS? I don't see benefits to running Docker in AWS. In my opinion, AWS implemented its Docker-based Container Service very poorly. I advise my customers against using AWS when want to use Docker. There are many bare metal as a service providers out in the marketplace. >the argument about running the same container in dev/test/prod Is this issue really caused by Docker because you said that you had consistent environments when built by chef?
  - notwedtm 11y ago
    Can you name a few bare metal as a service providers?
    - nickstinemates 11y ago
      Rackspace, SoftLayer/Bluemix
  - pavel_lishin 11y ago
    > In my opinion, AWS implemented its Docker-based Container Service very poorly. We looked into using it earlier this year. The web UI was flat-out busted in several ways - they only listed the first 100 security groups, and we have a magnitude more than that. The command line interface was poorly documented, and was missing some of the functionality. It was a total waste of a week; I wonder if they've fixed any of that.
  - vosper 11y ago
    Would you mind elaborating on some of the issues you see with Elastic Container Service? I'd hoped it would be something like an AWS-specific Mesos, but I haven't looked into it closely.
  - nnain 11y ago
    I installed the Amazon Linux Installation Image (AMI) with docker and Python 3.4 for a recent project. I know ElasticBeanstalk fairly well, but configuring this was a big headache. WASTED LOT OF TIME. Instead of making life easy, it just added unnecessary burden of learning Docker for the future project members. Documentation is poor, had to hunt for hours for solutions to simple queries. On AWS, I would suggest you stick with the basic Linux flavour that you know. Use their Docker build only IF you know docker very well.
hootener 11y ago
- maintain consistent dev/test/prod environments We're pretty heavily invested in Docker. To this point, it's really nice knowing that all the developers are operating in consistent environments. An additional bonus is that using Docker makes it really easy to propagate infrastructure changes to the rest of your team for use in development. And, more importantly, know those infrastructure changes are consistent across local dev setups. As an example, I recently incorporated Sphinx search into a project we were working on. I didn't want to require devs to install Sphinx on their own machines and get it up and running for search to function properly. I also didn't want devs to have the overhead of running Sphinx for search on their local boxes unless they were actively working on something related to search. Basically, I wanted search to be optionally configured to run on startup. I used a DockerFile to setup Sphinx in its own container, pushed the DockerFile to a Tools source repository we use, and then incorporated the build and running of that container into our startup scripts (just some simple orchestration stuff for the local machine written in bash). Now, if a dev wants a containerized search mechanism they run a simple bash script to build that container, then run another command to spin up our dockerized web app with a connection to a running sphinx search container. We do this for all of our services: mysql, redis, sphinx, the web app itself, and anything else we might need. As an added bonus, all the Docker CLI work and orchestration of our application is easily hidden behind a shell script. If a dev wants to run the webapp, they simply run: app server dev. If they want the app with search they run: app server dev search from the command line. Developers never need to know what's going on under the hood to get their job done. From their perspective, it just works.
antocv 11y ago
> Software engineering is often difficult because programmers have to deal with inconsistent environments for development and for production execution of their products. Due to mismatches between these environments, developers often found bugs that surface in one environment but not in another. Docker doesnt solve this, we already ran into problems with different docker client versions, different iptables and other tools used on the host. Docker does allow you to push the problem away for a while.

by osipov 11y ago

takeda 11y ago

Great, but what are the benefits of running Docker in AWS? You are still running VMs and you are being charged for running them. With Docker you are simply putting yet another layer of complexity, because now you have to run more beefier VMs, you now have problem with network communication between containers running on different hosts. So you will most likely need to use overlay network. You also decrease resiliency, because now when AWS terminated a single VM, all apps running on that node suddenly disappear. I also don't get the argument about running the same container in dev/test/prod. For example my company is working on going Docker and one of the problem with these environments is that app running there has different configuration. So the idea to solve it is to create three different versions of the same container. Genius! But now are you really running the same thing in dev/test/prod? How is it different to what we did in the past? Especially that before Docker through our continuous delivery we actually were using exact same artifact on machines set up with chef that were configured the same way as in prod, while with Docker now we plan to use three different containers.

osipov 11y ago
>what are the benefits of running Docker in AWS? I don't see benefits to running Docker in AWS. In my opinion, AWS implemented its Docker-based Container Service very poorly. I advise my customers against using AWS when want to use Docker. There are many bare metal as a service providers out in the marketplace. >the argument about running the same container in dev/test/prod Is this issue really caused by Docker because you said that you had consistent environments when built by chef?
- notwedtm 11y ago
  Can you name a few bare metal as a service providers?
  - nickstinemates 11y ago
    Rackspace, SoftLayer/Bluemix
- pavel_lishin 11y ago
  > In my opinion, AWS implemented its Docker-based Container Service very poorly. We looked into using it earlier this year. The web UI was flat-out busted in several ways - they only listed the first 100 security groups, and we have a magnitude more than that. The command line interface was poorly documented, and was missing some of the functionality. It was a total waste of a week; I wonder if they've fixed any of that.
- vosper 11y ago
  Would you mind elaborating on some of the issues you see with Elastic Container Service? I'd hoped it would be something like an AWS-specific Mesos, but I haven't looked into it closely.
- nnain 11y ago
  I installed the Amazon Linux Installation Image (AMI) with docker and Python 3.4 for a recent project. I know ElasticBeanstalk fairly well, but configuring this was a big headache. WASTED LOT OF TIME. Instead of making life easy, it just added unnecessary burden of learning Docker for the future project members. Documentation is poor, had to hunt for hours for solutions to simple queries. On AWS, I would suggest you stick with the basic Linux flavour that you know. Use their Docker build only IF you know docker very well.

hootener 11y ago

- maintain consistent dev/test/prod environments We're pretty heavily invested in Docker. To this point, it's really nice knowing that all the developers are operating in consistent environments. An additional bonus is that using Docker makes it really easy to propagate infrastructure changes to the rest of your team for use in development. And, more importantly, know those infrastructure changes are consistent across local dev setups. As an example, I recently incorporated Sphinx search into a project we were working on. I didn't want to require devs to install Sphinx on their own machines and get it up and running for search to function properly. I also didn't want devs to have the overhead of running Sphinx for search on their local boxes unless they were actively working on something related to search. Basically, I wanted search to be optionally configured to run on startup. I used a DockerFile to setup Sphinx in its own container, pushed the DockerFile to a Tools source repository we use, and then incorporated the build and running of that container into our startup scripts (just some simple orchestration stuff for the local machine written in bash). Now, if a dev wants a containerized search mechanism they run a simple bash script to build that container, then run another command to spin up our dockerized web app with a connection to a running sphinx search container. We do this for all of our services: mysql, redis, sphinx, the web app itself, and anything else we might need. As an added bonus, all the Docker CLI work and orchestration of our application is easily hidden behind a shell script. If a dev wants to run the webapp, they simply run: app server dev. If they want the app with search they run: app server dev search from the command line. Developers never need to know what's going on under the hood to get their job done. From their perspective, it just works.

antocv 11y ago

> Software engineering is often difficult because programmers have to deal with inconsistent environments for development and for production execution of their products. Due to mismatches between these environments, developers often found bugs that surface in one environment but not in another. Docker doesnt solve this, we already ran into problems with different docker client versions, different iptables and other tools used on the host. Docker does allow you to push the problem away for a while.