6 ms·

I went to a couple of events where Docker was a topic in Europe, the general feeling is that the solution looks great, but every single time a speaker/organizer

by bontoJR 12y ago

I went to a couple of events where Docker was a topic in Europe, the general feeling is that the solution looks great, but every single time a speaker/organizer asks to raise a hand if using it in production, nobody does. I mean, it's definitely a technology having a momentum, but I still have to find someone that heavily used it in production.

otterley 12y ago
That's because it isn't ready for production yet, and certainly not at scale. You're in for a world of hurt if you try to deploy hundreds of containers at once from a central registry. And there's not yet a way to bind containers to physical interfaces that DHCP themselves at runtime. Docker solves problems for developers, and soon it might solve problems in production. But for large deployments, that day is not today.
- piva00 12y ago
  Well, we're currently running 80+ instances in my company using Docker to deploy every app. I'm the responsible for the push to this Docker-only approach, given that we had a bunch of legacy apps that were running in Chef-managed nodes (that I've also implemented), so I don't think that it's a developers-only tool, it's already pretty mature and I've not yet run into requiring to bind a container to some physical interface or whatever, the virtual interface has been great thus far.
  - otterley 12y ago
    How large are your images? How long does it take you to fetch them from the registry? Binding to a physical interface is required if any of your apps will share port numbers on a host. If you don't have network ACLs that control traffic between VLANs, and you have a discovery service to manage rendezvous, then maybe you can get away without it. But many of us cannot.
    - piva00 12y ago
      Our images range from 600MiB to 2GiB in general. When you say share port numbers what exactly are you referring to? App X listening to port M in eth0 and app Y listening to port M in eth1? We don't really have this use case so for us it has been pretty simple. I think that your use case is a lot more complex than ours or I'm misunderstanding some part, are you using bare metal servers in your own data center? I'm trying to grasp what's your architecture like so I can see what parallel can I make to ours.
      - otterley 12y ago
        You didn't mention how long deploys take. 2GiB x 80 servers = 160GiB; assuming a 1Gb/s interface on the repository, it'll take a minimum of 22 minutes for all the servers to get their new bits.
        piva00 12y ago
        Not every server runs the same applications, we mostly do deploys on 5-6 servers at a time, it takes around 3 minutes to deploy our biggest apps, most of them deploy in under 30 seconds.
        curun1r 12y ago
        The nice part about Docker is the images are composed of layers, so you only have to re-download a layer when it changes. In practice, we pull all of our base layers when we bake our AMIs, so spinning up a new instance is pretty quick and each new deploy only has to update the top layer that contains our code. Honestly, the only issue when it comes to image size is the initial developer pull either when they onboard to a team or when they bork their boot2docker install (which, disturbingly, happens way too often).
        otterley 12y ago
        Many applications are very large (consider a Rails app with all of its prerequisite Gems). Granted, this isn't Docker's problem, but it presents practical distribution bandwidth constraints that Docker doesn't currently solve well.
  - bontoJR 12y ago
    Did you have to change anything during the implementation? What about habits compared to deploy on VMs/DSs? What's you experience regarding a long running? Did you face any issue?
    - piva00 12y ago
      We had to change how we managed services, I think it was for a better way, we were too much reliant on instances being available all the time, although I tried to always think with an HA/redundancy mind most of our team failed to do that. We're now using Docker with CoreOS (and fleet), so it has not been without headaches (mostly in part from CoreOS and btrfs and fleet scheduling going afoul), but we're much more resilient. Compared to VMs we adopted a lot more of a "this is disposable" mindset, we persist any data that must not be destroyed in a failure in an external volume and treat the instance and service as something unreliable, pretty much it's always "Game Day", before deploying something we've already tried to cover the basics of HA (what happens if this service suddenly stops? How is the rest of the system behave? How do we make it automatically restart or reprovision?). As I said, most of our issues regarded CoreOS (especially running with btrfs, getting out of space for metadata was a pain in the ass for months) but right now, after moving to ext4 + overlayfs we've been running without issues for a combined 28,800 hours of instance time (roughly NO issue in half a month in an environment running 30+ deploys a day). I've not stated that we're perfect, very far from it, but Docker itself has not been a pain in production. Our private registry is currently running in an Ubuntu-based instance for more than 6 months non-stop (and we push/pull a lot of data from it everyday).
      - vacri 12y ago
        Compared to VMs we adopted a lot more of a "this is disposable" mindset There is nothing particularly magical about Docker that means you can't consider VMs equally as disposable. It's a mindset change about how you build your product, not something inherent to Docker. It's the original concept behind AWS's EC2 - treat your VMs as disposable, and store state elsewhere.
        Huggernaut 12y ago
  - bmurphy1976 12y ago
    We're running it in production as well, very heavily (100+ servers running many more containers). The parent is right. Docker is not really ready for production yet. We've had to write (and continue to write) a lot of code to work around limitations and bugs. We've replaced the registry with custom code, written a lot of code to "manage" the Docker daemon, had to do some funny business to make logging work, are constantly integrating other components (such as weave) to work around Docker limitations, and my God the bugs... I can deal with the limitations. It's the fundamentals that are the real problem (bugs, memory leaks, descriptor problems, aufs issues, the fucking registry). Docker, Inc. keep chasing new features over fixing the fundamentals. That's their choice, but it's unfortunate for us because, well, I have to go restart a few Docker daemons now because of that memory leak, yet again. I expect these issues to be resolved eventually and there isn't a better alternative (the ones that exist have their own issues) so we take the pain and wait patiently for things to get better.
    - eropple 12y ago
      You're higher on the Docker product and team than I am, but with your discussion of new-and-shiny over working-and-bugfree you've pretty much nailed why I'm not happy with Docker and am hoping Rocket (or something out of left field) becomes a thing. While I'm kind of terrified in general about the current drive towards VC-backed companies providing core infrastructure, I think CoreOS's team cares much more about stability and correctness and I think I can at least come to grips with them. The usability of Docker gets so much worse as soon as you get outside the happy examples, too. Like, the one that most recently bit us: Docker is considered a "1.x product" and I have to hardcode a registry URI into the Dockerfile instead of something sane like a host-level search path--like all package managers have done for forevers now-and the suggested option by a Docker developer (and I'm not slagging him, he tried to help and I really appreciate that, it's not his fault this is stupid) was to muck around with DNS to redirect a single hostname to different places within my environment. Couple the constant friction of usage with the security questions throughout the system (single giant garbage executable that needs root for absolutely everything instead of principle of least privilege, and--oh!--looks like user namespaces slipped to 1.7, awesome!) and I'm not feeling confident in my use of Docker. I feel like they'd rather market than fix. - I am reminded of this article, and the dismissive replies by the author (a Docker employee and committer) here on HN: https://blog.jessfraz.com/posts/docker-containers-on-the-desktop.html https://blog.jessfraz.com/posts/docker-containers-on-the-des... https://news.ycombinator.com/item?id=9086751 https://news.ycombinator.com/item?id=9086751
- chimeracoder 12y ago
  > And there's not yet a way to bind containers to physical interfaces that DHCP themselves at runtime Sure there is. Systemd will actually do this for you. Docker recently eschewed LXC, and I'm still not sure I understand exactly why, but even after the move to libcontainer, systemd is capable of handling this.
  - otterley 12y ago
    I thought we were talking about Docker, not systemd. systemd is pretty powerful, but you have to write the glue Docker is providing yourself. If you mean a scenario where Docker launches systemd, we don't do that. In my view Docker is just a packaging system; each container's pid 1 is the app we're deploying.
    - chimeracoder 12y ago
      > If you mean a scenario where Docker launches systemd, we don't do that. In my view Docker is just a packaging system; each container's pid 1 is the app we're deploying. No, systemd is actually capable of running Docker containers directly (even without Docker). I definitely wouldn't recommend running systemd inside a container unless you're using systemd to launch the container (instead of Docker), since Docker doesn't handle init systems very well, whereas systemd has this built-in. Though you can also go with the one-app-process-per-container approach as well with systemd, which also works.
      - otterley 12y ago
        If you have any citations that describe how to make systemd run Docker containers without Docker, I'd like to see them.
        grosskur 12y ago
        Starting with systemd 219, you can download a Docker image from a registry with "machinectl pull-dkr", and run it with "machinectl start": http://www.freedesktop.org/software/systemd/man/machinectl.html http://www.freedesktop.org/software/systemd/man/machinectl.h...
- errordeveloper 12y ago
  There is a lot community effort being put into Docker extensions to enable production use cases like what you have just mentioned.
justincormack 12y ago
It is going through the same path that AWS did, people start with test and CI type work, then move towards production. Europe is behind Silicon Valley in terms of moving towards production.
- deleted 12y ago
  [deleted]
csomar 12y ago
I'm using it for a financial application in production. Granted I can go without it, but currently it makes lots of sense to use it since I need the ability to nicely handle dependencies. I'm using Grunt for handling the creation, destruction and running of the containers. Docker Compose didn't exist back then.

by bontoJR 12y ago

otterley 12y ago

That's because it isn't ready for production yet, and certainly not at scale. You're in for a world of hurt if you try to deploy hundreds of containers at once from a central registry. And there's not yet a way to bind containers to physical interfaces that DHCP themselves at runtime. Docker solves problems for developers, and soon it might solve problems in production. But for large deployments, that day is not today.

piva00 12y ago
Well, we're currently running 80+ instances in my company using Docker to deploy every app. I'm the responsible for the push to this Docker-only approach, given that we had a bunch of legacy apps that were running in Chef-managed nodes (that I've also implemented), so I don't think that it's a developers-only tool, it's already pretty mature and I've not yet run into requiring to bind a container to some physical interface or whatever, the virtual interface has been great thus far.
- otterley 12y ago
  How large are your images? How long does it take you to fetch them from the registry? Binding to a physical interface is required if any of your apps will share port numbers on a host. If you don't have network ACLs that control traffic between VLANs, and you have a discovery service to manage rendezvous, then maybe you can get away without it. But many of us cannot.
  - piva00 12y ago
    Our images range from 600MiB to 2GiB in general. When you say share port numbers what exactly are you referring to? App X listening to port M in eth0 and app Y listening to port M in eth1? We don't really have this use case so for us it has been pretty simple. I think that your use case is a lot more complex than ours or I'm misunderstanding some part, are you using bare metal servers in your own data center? I'm trying to grasp what's your architecture like so I can see what parallel can I make to ours.
    - otterley 12y ago
      You didn't mention how long deploys take. 2GiB x 80 servers = 160GiB; assuming a 1Gb/s interface on the repository, it'll take a minimum of 22 minutes for all the servers to get their new bits.
      - piva00 12y ago
        Not every server runs the same applications, we mostly do deploys on 5-6 servers at a time, it takes around 3 minutes to deploy our biggest apps, most of them deploy in under 30 seconds.
      - curun1r 12y ago
        The nice part about Docker is the images are composed of layers, so you only have to re-download a layer when it changes. In practice, we pull all of our base layers when we bake our AMIs, so spinning up a new instance is pretty quick and each new deploy only has to update the top layer that contains our code. Honestly, the only issue when it comes to image size is the initial developer pull either when they onboard to a team or when they bork their boot2docker install (which, disturbingly, happens way too often).
        otterley 12y ago
        Many applications are very large (consider a Rails app with all of its prerequisite Gems). Granted, this isn't Docker's problem, but it presents practical distribution bandwidth constraints that Docker doesn't currently solve well.
- bontoJR 12y ago
  Did you have to change anything during the implementation? What about habits compared to deploy on VMs/DSs? What's you experience regarding a long running? Did you face any issue?
  - piva00 12y ago
    We had to change how we managed services, I think it was for a better way, we were too much reliant on instances being available all the time, although I tried to always think with an HA/redundancy mind most of our team failed to do that. We're now using Docker with CoreOS (and fleet), so it has not been without headaches (mostly in part from CoreOS and btrfs and fleet scheduling going afoul), but we're much more resilient. Compared to VMs we adopted a lot more of a "this is disposable" mindset, we persist any data that must not be destroyed in a failure in an external volume and treat the instance and service as something unreliable, pretty much it's always "Game Day", before deploying something we've already tried to cover the basics of HA (what happens if this service suddenly stops? How is the rest of the system behave? How do we make it automatically restart or reprovision?). As I said, most of our issues regarded CoreOS (especially running with btrfs, getting out of space for metadata was a pain in the ass for months) but right now, after moving to ext4 + overlayfs we've been running without issues for a combined 28,800 hours of instance time (roughly NO issue in half a month in an environment running 30+ deploys a day). I've not stated that we're perfect, very far from it, but Docker itself has not been a pain in production. Our private registry is currently running in an Ubuntu-based instance for more than 6 months non-stop (and we push/pull a lot of data from it everyday).
    - vacri 12y ago
      Compared to VMs we adopted a lot more of a "this is disposable" mindset There is nothing particularly magical about Docker that means you can't consider VMs equally as disposable. It's a mindset change about how you build your product, not something inherent to Docker. It's the original concept behind AWS's EC2 - treat your VMs as disposable, and store state elsewhere.
      - Huggernaut 12y ago
- bmurphy1976 12y ago
  We're running it in production as well, very heavily (100+ servers running many more containers). The parent is right. Docker is not really ready for production yet. We've had to write (and continue to write) a lot of code to work around limitations and bugs. We've replaced the registry with custom code, written a lot of code to "manage" the Docker daemon, had to do some funny business to make logging work, are constantly integrating other components (such as weave) to work around Docker limitations, and my God the bugs... I can deal with the limitations. It's the fundamentals that are the real problem (bugs, memory leaks, descriptor problems, aufs issues, the fucking registry). Docker, Inc. keep chasing new features over fixing the fundamentals. That's their choice, but it's unfortunate for us because, well, I have to go restart a few Docker daemons now because of that memory leak, yet again. I expect these issues to be resolved eventually and there isn't a better alternative (the ones that exist have their own issues) so we take the pain and wait patiently for things to get better.
  - eropple 12y ago
    You're higher on the Docker product and team than I am, but with your discussion of new-and-shiny over working-and-bugfree you've pretty much nailed why I'm not happy with Docker and am hoping Rocket (or something out of left field) becomes a thing. While I'm kind of terrified in general about the current drive towards VC-backed companies providing core infrastructure, I think CoreOS's team cares much more about stability and correctness and I think I can at least come to grips with them. The usability of Docker gets so much worse as soon as you get outside the happy examples, too. Like, the one that most recently bit us: Docker is considered a "1.x product" and I have to hardcode a registry URI into the Dockerfile instead of something sane like a host-level search path--like all package managers have done for forevers now-and the suggested option by a Docker developer (and I'm not slagging him, he tried to help and I really appreciate that, it's not his fault this is stupid) was to muck around with DNS to redirect a single hostname to different places within my environment. Couple the constant friction of usage with the security questions throughout the system (single giant garbage executable that needs root for absolutely everything instead of principle of least privilege, and--oh!--looks like user namespaces slipped to 1.7, awesome!) and I'm not feeling confident in my use of Docker. I feel like they'd rather market than fix. - I am reminded of this article, and the dismissive replies by the author (a Docker employee and committer) here on HN: https://blog.jessfraz.com/posts/docker-containers-on-the-desktop.html https://blog.jessfraz.com/posts/docker-containers-on-the-des... https://news.ycombinator.com/item?id=9086751 https://news.ycombinator.com/item?id=9086751
chimeracoder 12y ago
> And there's not yet a way to bind containers to physical interfaces that DHCP themselves at runtime Sure there is. Systemd will actually do this for you. Docker recently eschewed LXC, and I'm still not sure I understand exactly why, but even after the move to libcontainer, systemd is capable of handling this.
- otterley 12y ago
  I thought we were talking about Docker, not systemd. systemd is pretty powerful, but you have to write the glue Docker is providing yourself. If you mean a scenario where Docker launches systemd, we don't do that. In my view Docker is just a packaging system; each container's pid 1 is the app we're deploying.
  - chimeracoder 12y ago
    > If you mean a scenario where Docker launches systemd, we don't do that. In my view Docker is just a packaging system; each container's pid 1 is the app we're deploying. No, systemd is actually capable of running Docker containers directly (even without Docker). I definitely wouldn't recommend running systemd inside a container unless you're using systemd to launch the container (instead of Docker), since Docker doesn't handle init systems very well, whereas systemd has this built-in. Though you can also go with the one-app-process-per-container approach as well with systemd, which also works.
    - otterley 12y ago
      If you have any citations that describe how to make systemd run Docker containers without Docker, I'd like to see them.
      - grosskur 12y ago
        Starting with systemd 219, you can download a Docker image from a registry with "machinectl pull-dkr", and run it with "machinectl start": http://www.freedesktop.org/software/systemd/man/machinectl.html http://www.freedesktop.org/software/systemd/man/machinectl.h...
errordeveloper 12y ago
There is a lot community effort being put into Docker extensions to enable production use cases like what you have just mentioned.

justincormack 12y ago

It is going through the same path that AWS did, people start with test and CI type work, then move towards production. Europe is behind Silicon Valley in terms of moving towards production.

deleted 12y ago
[deleted]

csomar 12y ago

I'm using it for a financial application in production. Granted I can go without it, but currently it makes lots of sense to use it since I need the ability to nicely handle dependencies. I'm using Grunt for handling the creation, destruction and running of the containers. Docker Compose didn't exist back then.