18 ms·

Docker not ready for primetime

490 points by erlend_sh 10y ago

ktamiola 10y ago
Fair enough! There is also performance penalty and annoyingly complicated setups in certain cloud environments.
mkagenius 10y ago
Would love to hear thought from people who use it in production?
- toomuchtodo 10y ago
  Unless you're using it to compartmentalize CI jobs on Travis or Jenkins, don't use it in production (do use it in development for standing up local dev environments quickly). We do, and it's a wasted layer of abstraction. I predict the Docker fad will end in 2-3 years, fingers crossed.
  - cgag 10y ago
    I think it's a pretty excellent layer of abstraction, as it was in the form of jails/zones. Even if we stuck to single node computing I think I think it'd be a powerful thing, but viewing a whole cluster of computers as a single compute resource is incredible. I think docker itself may fade but I view its popularity as jails finally catching on rather than a new bit of hype that's going to die down soon.
  - jagermo 10y ago
    maybe not end, but scale down to realistic usage scenarios.
  - eropple 10y ago
    > do use it in development for standing up local dev environments quickly You can do this, for sure, but, tbh, I'd still rather use Vagrant. I'm already provisioning things like my datastores using Chef (and not deploying those inside of Docker in production); I might as well repurpose the same code to build a dev environment. I do agree regarding using containers (I use rkt rather than Docker for a few reasons) for CI being a really good jumping-off point, though.
    - bdcravens 10y ago
      Vagrant is pretty heavy, but if you're already using Chef, makes sense. If you're deploying using Docker, then mirroring your environment in dev makes more sense. I've used both Vagrant and Docker for dev, and find Docker to be super fast and light. (though I've definitely run into some headaches with Docker Compose that I didn't see in using just Docker)
      - eropple 10y ago
        I think it's probably not worth worrying about "fast" when "slow" is "ten minutes once a week, maybe, if you're really destructive" and similarly worrying about "light" when you have 16GB of RAM and a quad-core processor; the heaviest dev environment/IDE I can find (IntelliJ and a couple hundred Chrome tabs) doesn't use eight of it. It comes off as super-premature optimization to me, and those optimizations that you would otherwise do in Docker (which are just building base images, whatever, Packer does this trivially) exist in a virtualized environment too. Running datastores in Docker is bonkers in the first place, which still leaves a significant place for an actual CM tool--so unless you are a particular strain of masochist, when you are replicating your local environment, you'll probably need it anyway.
        bdcravens 10y ago
        > ten minutes once a week, maybe, if you're really destructive If you do a lot of active machine development, where you're iterating on that machine setup, it's an extremely slow process. Perhaps it would make sense to iterate in Docker, and then convert that process to Vagrant once you finish. > similarly worrying about "light" when you have 16GB of RAM and a quad-core processor It's a concern when you're launching many machines at once to develop against. Docker tends to only eat what it needs. Also, while I think everyone should have a 16GB machine, there's quite a few developers out there running 8 or 4GB machines.
        eropple 10y ago
        > Perhaps it would make sense to iterate in Docker, and then convert that process to Vagrant once you finish. I disagree; writing convergent Chef is easy. (Actually convergent Ansible isn't, but that's just one more reason not to use it!) So I just re-run on the same machine. When I'm confident, I nuke and rebuild to make sure that the first pass, and not just later passes, work fine. When I'm doing this for other people, they don't have to rebuild at all unless I have to break compatibility (which is rare, if ever). Using Docker for this means wasting time using Docker instead of doing what I'm actually going to send to production. In a cloud environment, I get so vanishingly close to nothing for using Docker that it makes little sense to use it in development, at all. (I've tried. It's a slower, clumsier experience that results in a duplication of effort even before you have to buy into the gong show of k8s or whatever you want to use in order to actually manage your disaster.) > It's a concern when you're launching many machines at once to develop against. My standard VM flight for my current project is eight machines. It uses four gigs of RAM, and that's mostly because I decided to be pretty generous with RAM usage (none of my services really need half a gig of RAM when running in a development environment; there isn't enough data in a dev environment even to fill the caches). If you are doing the "oh, we have a billionty microservices" thing, well, sure, you might stick yourself in a rough place, but in the overwhelming majority of situations you're also probably engaged in masturbatory exercises that don't help you ship and I think you have much bigger problems on your hands. As far as developers with unsuitable hardware go--meh. "My boss is too cheap, so I need to use the wrong tools" is, and thankfully for the foreseeable future will be, someone else's problem. And that someone else should quit. (I was being pretty conservative with 16GB of RAM and a quad-core; that's just my laptop, I have a workstation for serious work.)
  - jacques_chester 10y ago
    > I predict the Docker fad will end in 2-3 years, fingers crossed. I work for what is, ostensibly, a competitor and I don't agree. Containers are here to stay. Docker will pretty much have a brand lock on that for a long time.
  - bdcravens 10y ago
    > I predict the Docker fad will end in 2-3 years, fingers crossed. Containers are definitely here to stay. Google, Heroku, and others have been using them for years. Obviously Docker is just a subset, and it might fade, but I think it's pretty well established as a defacto standard.
    - toomuchtodo 10y ago
      Containers have been around for more than a decade as lxc. I'm saying Docker itself. It's hard to make money when you're running on an open standard that anyone can build tooling around.
  - deleted 10y ago
    [deleted]
- DigitalJack 10y ago
  I use it in production by way of kubernetes. It's been great so far.
- olalonde 10y ago
  I used it in production with CoreOS and now Kubernetes. I almost never touch the Docker CLI and haven't tried Swarm. I think a lot of developers mistakenly think they can get to a production environment in a few Docker commands which is definitely not possible today.
  - Twirrim 10y ago
    From an ops side, that's worrying. Yes, in an absolutely ideal world no one ever should touch a production environment, and ideally in every situation you find yourself having to touch a production system you should then be creating at the very least a backlog item to figure out how to avoid doing it in the future, but the reality is some kinds of troubleshooting are hard to do without having access. Generally the kinds of situations where you need that access are ones when you want access the quickest and easiest, i.e. the world is burning somehow.
    - lobster_johnson 10y ago
      OP wasn't talking about access, but about the effort needed to set up ("get to") a production environment. Of course Kubernetes/Docker gives you full access to your production environment.
- TheHydroImpulse 10y ago
  We're using Docker pretty extensively at Segment with over 1,200 containers running in production with ECS (total count, not uniqueness). We haven't touched Swarm and are staying a couple versions behind. With that said, we're pretty happy with the setup right now.
  - NathanKP 10y ago
    Airtime is also using ECS to run docker containers in production. We ran into a few issues early on due to early adopter status, but the AWS team was very quick to add many features we needed. I have no complaints about ECS now, and in fact I'd say that ECS solves a lot of the problems that the author complains here. I'd say that Swarm is not production ready, but docker itself as an engine for running a container is fine. You just need to build your own orchestration layer, or use something like ECS. More details on Airtime's docker setup here: https://techblog.airtime.com/microservice-continuous-integration-made-easy-with-aws-ecs-10d470e31af0 https://techblog.airtime.com/microservice-continuous-integra...
    - hosh 10y ago
      Maybe things has changed since Apr 2015, but I found ECS pretty limited and K8S solving those limitations. One big issue was container-to-container addressing. It was not available last year when I put an ECS cluster into production. Has it been added since?
      - NathanKP 10y ago
        Direct container to container communication isn't the recommended pattern when using ECS. Instead you let ECS associate with either a legacy Elastic Load Balancer, or one of the new Application Load Balancers, which can direct requests to different classes of containers based on the request path, allowing you to take a set of containers of different types and meld their http resources together into one API on one domain. If one container needs to talk to another container of a certain type it just uses the load balancer, and the load balancer takes care of directing the request to the other container which may be on the same machine, or another machine, or maybe even in a different cluster. Obviously this forces you into a fully stateless design because the load balancer round robins all the requests across random containers, but its very fail proof, and gives you high resiliency.
- ceocoder 10y ago
  We use it via GKE in prod. Zero complaints so far.
- knz 10y ago
  I've posted about this before, but my work (non IT Department at a State Govt agency) has been using Docker in production for about a year. We are small by HN standards (~12 apps on 5 hosts that are a mix of API's, front ends, and several more complex GIS apps). Moving to Docker was a massive improvement for us in terms of deployment and development. Deploying with Docker is ridiculously easy (no more taking a node out of HAProxy for an update, rollbacks are painless, and we love that what you deploy is what you get - there is no question over what the container is actually doing since you define it in the Dockerfile). I can't remember the last time we had a bad deployment or even cared about making a change to production. It's that reliable. It's also been a massive improvement for QA and development because of the ability to standardize the container using the Dockerfile. We've been using CentOS for the production hosts and CentOS/OSX for developing on. There were a couple of minor issues when we started but the benefits outweighed the negatives then. It's been completely painless since Docker came out of beta. We have some containers that are deployed daily and others that run for months without being touched. I've never quite understood the negative attitude towards Docker. Perhaps we are too small to see many of the issues that people often complain about, but moving to Docker was a great decision in our office.
  - cpitman 10y ago
    Are you installing docker out of the CentOS repositories, or from Docker Inc's repository? The CentOS version is based on the work Red Hat does to backport patches and create more production ready versions of Docker, so that might be why you are having a better experience.
    - knz 10y ago
      We are indeed using the CentOS repositories. The current version on our production servers is 1.10.3.
anotherdpk 10y ago
> Each version of the CLI is incompatible with the last version of the CLI. Sure, but I don't think this is a show stopper. You can and should only carefully upgrade between versions of Docker (and other mission-critical software). The process is functionally identical to the process you'd use to perform a zero-downtime migration between versions of the Linux kernel -- bring up a new server with the version of Docker you want to use, start your services on that new server, stop them on the old server, shut down the old server, done. I don't mean to suggest this is trivial, only to suggest that it is no more complicated than tasks that you/we are already expected to perform.
- coldtea 10y ago
  >Sure, but I don't think this is a show stopper. You can and should only carefully upgrade between versions of Docker (and other mission-critical software). Whether I upgrade carefully or not, the Docker CLI should work with all versions of Docker.
  - anotherdpk 10y ago
    I don't get why this is a sticking point. There's no reason to update the Docker CLI independent of the Docker server, and it would be silly to run two versions of Docker on the same host, so why would it matter?
- StreamBright 10y ago
  I think you are missing the point. It is not about upgrading the production cluster, more like erasing muscle memory and re-learning docker cli again. How often do you re-learn grep, awk, ls, ps, or other command line tools? How mad would you be if these were developed the same manner as Docker CLI? I would be definitely not happy.
deleted 10y ago
[deleted]
okket 10y ago
Two days ago there was a fairly long discussion about a similar argument ("The sad state of Docker") https://news.ycombinator.com/item?id=12364123 https://news.ycombinator.com/item?id=12364123 (217 comments)
- deleted 10y ago
  [deleted]
perturbation 10y ago
Most of the complaints I've seen recently about using Docker are about the immaturity of Docker Swarm. Can this be mitigated by using Docker with Kubernetes / Mesos / Yarn? If it's truly a problem with the containerization format / stability with the core product, I'm not sure what a good alternative would be. I see a lot of praise for rkt but the ecosystem and tooling around it are so much smaller than that for Docker.
- lobster_johnson 10y ago
  Using it with Kubernetes definitely helps. One reason is that if you have enough surplus nodes, then Docker misbehaving on one of them shouldn't screw up anything; Kubernetes is really good at shuffling things around, and you can "cordon off" problem nodes to prevent them from being used for scheduling.
deleted 10y ago
[deleted]
joshka 10y ago
> Each version of the CLI is incompatible with the last version of the CLI. Run the previous version of the cli in a container on your local machine. https://hub.docker.com/_/docker/ https://hub.docker.com/_/docker/ $ docker run -it --rm docker:1.9 version
- deleted 10y ago
  [deleted]
- Chronic9q 10y ago
  You literally just proved his point.
  - jcoffland 10y ago
    I don't think he was arguing against it.
    - Twirrim 10y ago
      It's so hard to tell. I hope they were joking.
  - joshka 10y ago
    I'm not sure that I did. Let's expand out my point to see if I'm on the same page as the replies here. 1. TFA complains that versions of servers require corresponding client tools to be installed. 2. TFA says he loves docker in dev, but not in prod. 3. I state (omitting many steps): install the latest docker in your dev environment (or wherever you're connecting to prod), spin up a small docker image to run the appropriate version cli. This is cheap, easy, easy to understand why it works, consistent with points 1 and 2. What am I missing?
- inlined 10y ago
  This is both very clever and horrifying. This obviously works for development but doing a rolling update of your fleet is stressful without the additional wrinkle of progressively updating which tool you need to inspect or control that fleet.
asitdhal 10y ago
Do they speak English or the caller is expected to know French ?
- steveklabnik 10y ago
  Did you mean to comment on https://news.ycombinator.com/item?id=12376886 https://news.ycombinator.com/item?id=12376886 ?
  - asitdhal 10y ago
    yes, thanks
dawnerd 10y ago
I spent close to 12 hours yesterday trying to get a fairly simple node app to run on my mac yesterday. Turned out I had to wipe out docker completely and reinstall. Keep in mind this is their stable version thats no longer in beta. I've just run into too many documented bugs for me to consider it stable. I wouldn't even say it should be out of beta. The issues here are the real telling story: I spent close to 12 hours yesterday trying to get a fairly simple node app to run on my mac yesterday. Turned out I had to wipe out docker completely and reinstall. Keep in mind this is their stable version thats no longer in beta. I've just run into too many documented bugs for me to consider it stable. I wouldn't even say it should be out of beta. The issues here are the real telling story. https://github.com/docker/for-mac/issues https://github.com/docker/for-mac/issues I love docker, it's amazing when it works. It's just really not there yet. I get that their focus is on making money right now, but they need to nail their core product first. I honestly don't care about whatever cloud platform they're building if their core app doesn't even work reliably.
- Ninn 10y ago
  Heads up -- I think you messed up your copy/paste.
  - coldtea 10y ago
    That, or the issues here are the real telling story.
- serverholic 10y ago
  I'm a fan of Docker in general but I'm amazed by some of the poor choices they've made with Docker for Mac. - Stable? That's honestly laughable. It is nowhere near stable. As an example, I was trying to upload images to a third-party image registry but the upload speed was ridiculously slow. It took me forever to figure out but it turned out I needed to completely reinstall docker for mac. - They had a command-line tool called pinata for managing daemon settings in docker for mac. They chose to get rid of it. Not only did we lose a way to declaratively define and set configuration but the preferences window has no where near all of the daemon settings that are available. - The CPU usage is still crazy. I regularly get 100% CPU usage on 3/4 of my CPU cores while starting up just a few containers. Even after the containers have started it will idle at 100% 1/4 cores. - It needs to be reinstalled regularly if you are using it on a daily basis. Otherwise it will get slower and slower over time. See my first complaint. - The GUI (kitematic) will randomly disconnect from the daemon forcing me to restart the GUI repeatedly. - They really need some sort of garbage collector with adjustable settings. With the default settings the app will just keep building and building images and eventually fill up, crash, slow down, etc. How is that acceptable? What other apps do that? Like I said, I like docker in general. I think they are tackling some very hard problems and definitely experiencing some growing pains from such crazy growth. However, at some point they need to take a step back and focus on the core of what they offer and make it as simple, and rock solid as possible. As another example, they still haven't added a way to compress, and/or flatten docker images. No wonder docker for mac slows down after regular use when it's building 1GB+ images for simple things.
kozikow 10y ago
I am using kubernetes instead of docker swarm to orchestrate docker images and all points mentioned in this article do not apply. My cluster is small - I have <100 machines at peak, but so far it feels ready for prime time. There are parts of docker that are relatively stable, have many other companies involved, and have been around for a while. There are also "got VC money, gotta monetise" parts that damage the reputation of stable parts.
- hosh 10y ago
  I've used K8S before, and those things are smooth once installation happen. We have a project which is looking at using Docker Swarm on a two node set up. The idea is that this will give us a baseline, and since it is looking more and more we'll move to GKE, this sets us up for a later move into GKE. I thought Docker Swarm would be mature enough to handle two nodes, for a short time (maybe half a year) until we make the leap to GKE. But based on the reports coming in from the wild, that's looking less and less likely. We hired a contractor to do this part. My team is so resource constrained that I don't have time for this, so we farmed it out. But now I'm thinking, the risk of project failure is much higher than I thought, made worse that the contractor is also showing signs of having poor communication skills. (I would rather be updated on things going wrong than to have someone try to be the hero or cowboy and figure it all out).
  - sandGorgon 10y ago
    we are exactly in your boat. However, we are signing up for kubernetes since we want to keep the GKE option open. The only reason why we didnt go ahead is because of broken logging in k8s - https://github.com/kubernetes/kubernetes/issues/24677 https://github.com/kubernetes/kubernetes/issues/24677
    - kozikow 10y ago
      I went through a few iterations for logging, but now I settled on using built-in GKE logging. Stdout logs from my containers are picked up by kubernetes and forwarded to stackdriver. Since it's just stdout I do not create too much lock-in. I use stackdriver dashboard for investigating recent logs and BigQuery exporter for complex analysis. My stdout logs are jsons, so I can export extra metadata without relying on regexps for analysis - I use https://pypi.python.org/pypi/python-json-logger https://pypi.python.org/pypi/python-json-logger.
ycombinatorMan 10y ago
Aye, theres a lot of important bugs that are just sitting there
jcoffland 10y ago
One of Docker's biggest problems is that internally they have fomented a culture of "users are stupid" which is immediately apparent if you interact with their developers on GitHub.
- StreamBright 10y ago
  This is pretty dangerous path to failure. If you look at the most successful companies it is obvious that they respect the user base using their service. Software is no exception to that, you have to make it user friendly, regardless if your user base is mostly software and systems engineers.
  - enraged_camel 10y ago
    I don't think it's so cut-and-dry. I mean, does Oracle respect its users and is it user-friendly? What about Epic, the dominant software system in the healthcare industry? Or Microsoft SharePoint?
    - ghufran_syed 10y ago
      "If you look at the most successful companies it is obvious that they respect the user base using their service" - I think a better way of putting it would be that "successful companies respect their customers". In the case of software tools like Docker, the customers generally are the users. In the case of Epic, an electronic health record, the buyers are senior management, the users are the clinical staff such as the nurses, doctors and techs. So Epic does give the senior management what they want (buzzword compliance, a safe choice that it's hard to get fired for), but doesn't give clinical staff what they want..because they are not the ones making the buying decision.
    - StreamBright 10y ago
      Most, not all. I think you are still better of listening to your customers.
- danielrhodes 10y ago
jacques_chester 10y ago
Make no mistake: Docker Inc has a lot of excellent engineers. But there's also a landrush going on. Everyone has worked out that owning building blocks isn't where the money is. The money is in the platform. Businesses don't want to pay people to assemble a snowflake. They want a turnkey. CoreOS, Docker and Red Hat are in the mix. So too my employers, Pivotal, who are (disclosure) the majority donors of engineering for Cloud Foundry. IBM is also betting on Cloud Foundry with BlueMix, GE with Predix, HPE with Helion and SAP with HANA Cloud Platform. You're probably sick of me turning up in threads like these, resembling one of the beardier desert prophets, constantly muttering "Cloud Foundry, Cloud Foundry". It's because we've already built the platform. I feel like Mugatu pointing this out over and over. We're there! No need to wait! A distributed, in-place upgradeable, 12-factor oriented, containerising, log-draining, routing platform. The intellectual lovechild of Google and Heroku. Basically, it's like someone installed all the cool things (Docker, Kubernetes, some sort of distributed log system, a router, a service injection framework) for you. Done. Dusted. You can just focus on the apps and services you're building, because that's usually what end users actually care about. And we know it works really well. We know of stable 10k app instance scale production instances that are running right now. That's real app instances, by the way: fully loaded, fully staged, fully logging, fully routed, fully service injected, fully distributed across execution cells. Real, critical-path business workloads. Front-page-of-the-WSJ-if-it-dies workloads. Our next stretch goal is to benchmark 250k real app instances. If you need more than 250,000 copies of your apps and services running, then you probably have more engineers than we do. Though I guess you could probably stretch to running two copies of Cloud Foundry, if you really had to. OK, a big downside: BOSH is the deployment and upgrade system. It's not very approachable, in the way that an Abrams main battle tank is less approachable than a Honda Civic (and for a similar reason). We're working on that. The other downside: it's not sexy front-page-of-HN tech. We didn't use Docker, it didn't exist. Or Kubernetes, it didn't exist. We didn't use Terraform or CloudFormation ... they didn't exist. Docker will get all this stuff right. I mean that sincerely. They've got too many smart engineers to fail to hammer it down. More to the point, Docker have an unassailable brand position. Not to be discounted. Microsoft regularly pulled did this kind of thing for decades and made mad, crazy cash money the whole way along.
- sandGorgon 10y ago
urvader 10y ago
The title should say: Docker Swarm is not ready for primetime. We have used Docker in production for more than two years and there has been very few issues overall.
StreamBright 10y ago
I have the same experience. I am trying to set a new node where I accidentally installed 1.10 and CLI does not work. Looking for some thing on the internet how do X with Docker, only articles available for previous versions. I mean seriously, command line development is not supposed to be this hard, get a set of switches and stick to it unless something major forces you to change. If you shuffle around CLI switches between minor releases nobody is going to be happy.
exratione 10y ago
So far I'm feeling pretty good about the decision to skip the first generation containerization infrastructure. At the outset it had the look of something that wasn't an advance over standard issue virtualization, in that it just shuffled the complexity around a bit. It doesn't do enough to abstract away the ops complexity of setting up environments. I'm still of the mind, a few years later, that the time to move on from whatever virtualization approach you're currently using for infrastructure and development (cloud instances, virtualbox, etc), is when the second generation of serverless/aws-lambda-like platforms arrive. The first generation is a nice adjunct to virtual servers for wrapping small tools, but it is too limited and clunky in its surrounding ops-essential infrastructure to build real, entire applications easily. So the real leap I see ahead is the move from cloud servers to a server-free abstraction in which your codebase, from your perspective, is deployed to run as a matter of functions and compute time and you see nothing of what is under that layer, and need to do no meaningful ops management at all.
- dmourati 10y ago
  This sounds like I wrote it.
siliconc0w 10y ago
We use it in production. It generally works if: * you don't use it to store data * don't use 'ambassador', 'buddy', or 'data' container patterns. * use tooling available to quickly and easily nuke and rebuild docker hosts on a daily or more frequent basis. * use tooling available to 'orchestrate' what gets run where - if you're manually running containers you're doing it wrong. * wrap docker pulls with 'flock' so they don't deadlock * don't use swarm - use mesos, kube, or fleet(simpler, smaller clusters)
- bphogan 10y ago
  woah, wait..... It "geneerally works if" you " rebuild docker hosts on a daily or more frequent basis." Perhaps I'm misunderstanding, but needing to rebuild my prod env several times a day seems pretty "not ready for prime time" to me. That's like when we'd say that Rails ran great in production in 2005, as long as you had a cron task to bounce fastCGI processes every hour or so. So, can you elaborate on why rebuilding the containers is good advice?
  - jjn2009 10y ago
    The host isn't the container itself. They want to re-provision the host likely not because of something wrong with the application but instead docker is in some state which is non-recoverable, or at least not recoverable by automatic means.
  - jacques_chester 10y ago
    The security exec at Pivotal, where I work, has been talking about "repaving" servers as a security tactic (along with rotating keys and repairing vulnerabilities).[0] The theory runs that attackers need time to accrue and compound their incomplete positions into a successful compromise. But if you keep patching continuously, attackers have fewer vulnerabilities to work with. If you keep rotating keys frequently, the keys they do capture become useless in short order. And if you rebuild the servers frequently, any system they've taken control of simply vanishes and they have to start from scratch. I'm not completely sold on the difference between repair and repave, myself. And I expect that sophisticated attackers will begin to rely more on identifying local holes and quickly encoding those in automated tools so that they can re-establish their positions after a repaving happens. But it raises the cost for casual attackers, which is still worthy. [0] https://medium.com/built-to-adapt/the-three-r-s-of-enterprise-security-rotate-repave-and-repair-f64f6d6ba29d#.k3dj6wlp1 https://medium.com/built-to-adapt/the-three-r-s-of-enterpris...
dcosson 10y ago
> Each version of the CLI is incompatible with the last version of the CLI. I'm pretty sure that as long as the CLI version is >= the server version, you can set the DOCKER_VERSION env var to the server version and everything works. I haven't used this extensively, so maybe there are edge cases or some minimal supported version of backwards compatibility?
sandGorgon 10y ago
it seems that none of the container frameworks are generally ready. Take for example k8s - I just started exploring it as something we could move to. https://github.com/kubernetes/kubernetes/issues/24677 https://github.com/kubernetes/kubernetes/issues/24677 - logging of application logs is an unsolved problem. And most of the proposals talk about creating yet another logger...rather than patching journald or whatever exists out there. For those of you who are running k8s in production - how are you doing logging ? does everyone roll their own ?
- felixgallo 10y ago
  logging of application logs generally in the world is an unsolved problem, but I feel like k8s does better than most; fluentd is the default, and then logs are collected by whatever end system you like the best that's appropriate for your needs (e.g. GKE -> fluentd -> Cloud Logging, and I've heard of K8S -> fluentd -> ELK).
  - sandGorgon 10y ago
    since most people use k8s in the context of GKE.. there seems to be some magic that Google has put in. But if you read the bug, this doesnt work quite well in the wild. In fact, an entire spec is being written on it and right now the debate is whether to forward to journald and have it proxy further or reimplement a new logging format.
    - felixgallo 10y ago
      well, there's definitely systemd people pushing for journald, that's for sure. The first comment talks about how it doesn't fit, though. I wonder what in particular about the solution doesn't work for those people.
- lobster_johnson 10y ago
  Logging is definitely not an unsolved problem with K8s. It's trivial to set up Fluentd to snarf all container logs into some destination, such as Graylog, ElasticSearch or a simple centralized file system location. The Github issue talks a lot about "out of the box" setup via kube-up. If you're not using kube-up (and I wouldn't recommend using it for production setups), the problem is rather simple. The logging story could be better, but it's not "unsolved".
jbverschoor 10y ago
Docker is the new mongodb. Let's just use freebsd jails + postgresql.
- abz10 10y ago
  +1 I got sick of Docker and switched to FreeBSD. Far better experience.
  - X86BSD 10y ago
    Jails and Tredly.com or SmartOS. Anything else is busted by design. Enjoy your poop sandwich.
morgante 10y ago
This article should really be about Docker Swarm not being ready for production. It's much newer technology than Docker and is predictably brittle. The only points made against Docker proper are rather laughable. You shouldn't be remotely administering Docker clusters from the CLI (use a proper cluster tool like Kubernetes), and copying entire credentials files from machine to machine is extremely unlikely/esoteric. Docker, with Kubernetes or ECS, is totally suitable for production at this point. Lots and lots of companies are successfully running production workloads using it.
- atemerev 10y ago
  > You shouldn't be remotely administering Docker clusters from the CLI (use a proper cluster tool like Kubernetes) Docker Swarm is advertised as stable, production ready cluster management solution. Then, if you actually try to use it, it is very NOT. Kubernetes is great, but it feels like adding another layer to the system, and it is not always a good thing (especially if you are on AWS and have to work with _their_ infrastructure management too).
zwischenzug 10y ago
Can anyone using Swarm seriously in production post any account of their experiences here? Thanks.
- nostrebored 10y ago
  I doubt you'll get any replies. We tried using swarm in production, but could not ever get a reliable, working component. Failures of the worst kind would happen - networking would work 1/10 times meaning that a build could've conceivably gotten through CI.
markbnj 10y ago
I have run docker in production at past employers, and am getting ready to do so again at my current employer. However I don't run it as a native install on bare metal. I prefer to let a cloud provider such as Google deal with the infrastructure issues, and use a more mature orchestration platform (kubernetes). The author's complaints are valid, and the Docker team needs to do a better job on these issues. Personally I am going to be taking a close look at rkt and other technologies as they come along. Docker blew this technology open by making it more approachable but there is no reason to think they are going to own it. It's more like databases than operating systems.
- lugg 10y ago
  I'm trying to get started with rkt right now but it's (understandable) lack of maturity is a bit daunting, I think some usability issues need to be handled / offloaded to some other tools. And acbuild severly needs caching built in. Disclaimer, new to containers, if it sounds like I'm doing something wrong let me know, it certainly feels like I'm missing something right now.
  - jzelinskie 10y ago
    You'll see a maturity in this space after the Open Containers Initiative standardizes the container image format. Then, developers can focus on UX improvements rather than worrying about what the build tool should even be producing.
    - jmspring 10y ago
      Reliance on initiatives and standards bodies to provide a guidance for maturity, at least in recent years, is a fools errand. They usually end up rubber stamping what's been the norm and work back from there.
      - cesnja 10y ago
        That's true, OCF is more or less the same as the Docker container format.
bogomipz 10y ago
Docker not being ready for primetime and Swarm not being ready for primetime are two different things no? As for the cli compatibility issues, don't most people use an orchestration tool like ansible/chef/puppet etc to manage their fleet? I'm not sure I think the title of the post is accurate.
- Annatar 10y ago
  don't most people use an orchestration tool like ansible/chef/puppet etc to manage their fleet? I don't. All my configuration management is done with OS packages. And once you start making OS packages to do mass-scale configuration, the whole premise of Docker becomes pointless. If I have OS packages, and I can provision them using KickStart, or JumpStart, or imgadm(1M) + vmadm(1M), without using or needing any code, then what exactly do I need Docker for?
jasoncchild 10y ago
"Just imagine if someone emailed you a PDF or an Excel file and you didn't have the exact verion of the PDF reader or Excel that would open that file. Your head would blow clean off." Obviously this person has never spent a good deal of time dealing with AutoCAD...
- saidajigumi 10y ago
  Or more to the point early versions of the MS Office suite, where major updates could and did create backwards-incompatible saves. Users had to be careful that files intended for broader distribution were saved in a sufficiently old/compatible format version. Which is to say: this is a sign that the involved technologies are still rapidly maturing.
- mattdennewitz 10y ago
  or excel!
kev009 10y ago
I wonder how much suffering would be alleviated in most mid-level IT organizations if they just used Joyent/SmartDataCenter, and I say this as a FreeBSD developer with no affiliation.
- Annatar 10y ago
  A lot. zones in SmartOS provide full-blown UNIX servers running at the speed of the bare metal, but in complete isolation (no need for hacks like "runC", containers, or any other such nonsense). Packaging one's software into OS packages provides for repeatability: after the packages are installed in a zone, a ZFS image with a little bit of metadata can be created, and imported into a local or remote image repository with imgadm(1M). https://wiki.smartos.org/display/DOC/Managing+Images https://wiki.smartos.org/display/DOC/Managing+Images That's it. It really is that simple.
hellofunk 10y ago
I agree in general, and find Docker one of those technologies that does not (yet) live up to the hype.
brint 10y ago
For the versions issue, check out dvm: https://github.com/getcarina/dvm https://github.com/getcarina/dvm While versions are an issue, it's at least a reasonable way to work around it.
MrFurious 10y ago
Docker, container for hipsters that doesn't know deploy a simple linux virtual server.
ldehaan 10y ago
I have several past clients running docker in production just fine. At my current job we run nearly all our services in docker. I've replied to this type of comment on here at least a dozen times, it has nothing to do with docker, it is a lack of understanding how it all works. Understand the history, understand the underlying concepts and this is no more complex than an advanced chroot. Now on the tooling side, I personally stay away from any plug-ins and tools created by the docker team, they do docker best, let other tools manage dockers externalities. I've used weave since it came out, and it's perfect for network management and service discovery. I prefer to use mesos to manage container deploys. There is an entirely usable workflow with docker but I like to let the specialists specialize, and so I just use docker (.10.1 even), because all the extra stuff is just making it bloated. I'm testing newer versions on a case by case basis, but nothing new has come out that makes me want to upgrade yet. And I'll probably keep using docker as long as it stays separate from all the cruft being added to the ecosystem.
acd 10y ago
Here are alternatives to Docker One can use Ubuntu LXD which is Linux containers built on top of LXC but with ZFS as storage backend. LXD can also run Docker containers. http://www.ubuntu.com/cloud/lxd http://www.ubuntu.com/cloud/lxd One can also use Linux containers via Kubernetes by Google. http://kubernetes.io/ http://kubernetes.io/
20yrs_no_equity 10y ago
Network partitions really do happen! They are often short, but if you can't recover from them, then you shouldn't call yourself a distributed system. I am shocked at how fragile etcd is in this way. I was hoping docker swarm was better, but I'm not surprised (alas) to find out that it has the same problem. I'm about ready to build my own solution, because I know a way to do it that will be really robust in the face of partitions (and it doesn't use RAFT, you probably should not be using RAFT, I've seen lots of complaints about zookeeper too. I've done this before in other contexts so I know how to make it work, but so have others so why are people who don't know how to make it work reinventing the wheel all the time?)
- forktheweb 10y ago
  amazon efs + duplicity p.o.t. snapshots -> s3 + docker = win
- helloiamaperson 10y ago
  I'd love to hear more about your solution. Are you saying that you've created an algorithm distinct from paxos/raft/zab that's more robust?
- ldehaan 10y ago
  Check out weave, it's great for service discovery. Zookeeper is much better than etcd imo.
jokoon 10y ago
I watched again an explanation of what docker really is, it just seems to be this awesome thing that solve the very hard problem of inter-compatiblity between systems. I always tend to question how and why a developer had to use docker instead of making choice that would avoid it. It's not surprising docker can't always work, but it's nice to see that programmers are winning. I guess future OS designers and developers will try to encourage for more inter compatibility if possible. That's really a big nerve.
stevesun21 10y ago
As I worked for my ex-employer, we use docker based Elatic beanstalk serve over millions requests across three services per seconds.
rjurney 10y ago
Docker Swarm is definitely not production ready. Try to run any service that requires communication among nodes and you will agree. It works fine for web servers, but that is about it. DC/OS is emerging as the go-to way to deploy docker containers at scale in complex service combinations. It 'just works' with one simple config per service.
sergiotapia 10y ago
We've tried to use Docker a couple of times, but it was always much more trouble than it was worth. It was always some edge case that caused it to not work as expected one developer machine or another. After about 2 years of giving it another shot on and off, we just gave up. And it's not like we were doing something crazy, just a typical documented "run this rails app" type thing. I would definitely not use this in production for anything based on my experience.
- sjellis 10y ago
  Were you using Linux developer machines? Docker, Inc. provide ready-made installers for Windows and Mac - the Docker ToolBox makes setting up on those OSes very easy these days. I haven't tried the new products that replace ToolBox yet. I can imagine it being harder on some Linux distributions because your host system will have to have all of the kernel bits etc. Ubuntu 14.04 and above or current versions of Fedora should be good - the distributions explicitly support Docker, and Docker, Inc. provide packages so that you can run either the distribution packages or the Docker, Inc. ones.
  - sergiotapia 10y ago
    We are all on Macs.
deleted 10y ago
[deleted]
forktheweb 10y ago
I would say that my experience with Docker has been fantastic. I run over 10 Ubuntu Trusty instances on EC2 as 8G instances, mounted with NFS4 to EFS. This makes it super simple to manage data across multiple hosts. From that you can run as many containers as you like, and either mount them to the EFS folder, or just spawn them with data-containers, then export backups regularly with something like duplicity. I use rancher with it, and it's retarded simple using rancher/docker compose. For a quick run-down see: https://github.com/forktheweb/amazon-docker-devops https://github.com/forktheweb/amazon-docker-devops
_jezell_ 10y ago
The Docker team is doing more than anyone to move container technology forward, but orchestration is a much harder problem to solve than wrapping OS APIs. I wish they would stick to the core, and let others like the Kubernetes team handle the orchestration pieces. Swarm is hard to take seriously right now. I'm not sure bundling it into the core was the best way to handle it.
- coding123 10y ago
  I disagree. Kube, Mesos, etc.. are all great I'm sure. I had Kub documentation open in the background for about a week or two while doing other things, trying to get a handle on installation, deployment, how it will fit my use-cases... I kept wanting to dip in, but kept getting pounded by tons of heavy dependencies I would have had to learn. After Swarm Mode was released I had a cluster deploying things locally within 2 hours of reading the documentation. From there was able to map all of our use-cases on top (rolling updates, load balancing, etc...). I totally get that it's "hard to take seriously" perspective with some of the early glitching, but Kube, Mesos, etc.. needed this egg on their faces to realize how easy the installation and getting things up and running should be. If nothing else, that alone will make those other products better.
forktheweb 10y ago
I would say that my experience with Docker has been fantastic. I run over 10 Ubuntu Trusty instances on EC2 as 8G instances, mounted with NFS4 to EFS. This makes it super simple to manage data across multiple hosts. From that you can run as many containers as you like, and either mount them to the EFS folder, or just spawn them with data-containers, then export backups regularly with something like duplicity. I use rancher with it, and it's retarded simple using rancher/docker compose. For a quick run-down see: https://github.com/forktheweb/amazon-docker-devops https://github.com/forktheweb/amazon-docker-devops More advanced run-down of where I'm going with my setup: https://labs.stackfork.com:2003/dockistry-devexp/exp-stackerize-app https://labs.stackfork.com:2003/dockistry-devexp/exp-stacker...
- forktheweb 10y ago
  Just want to add that I also use it on windows without barely any issues (win 10 x64). I'm not sure how stable it is on Mac OSX but Kitematic is pretty sweet. The only problems I've had with Docker container are those where processes get stuck inside the container and the --restart=always flag is set. When this happens it means that if you can't force the container to stop, when you reboot the defunct container will restart anyway and cause you the same issue... My solution to this has been to just create a clean AMI image with ubuntu/rancher/docker and then nuke the old host when it gives me problems. This is made even easier if you use EFS because it's literally already 100% setup once you launch a replacement instance. Also, you can do automatic memory-limiting and cpu-limiting your nodes using rancher-compose and health-checks that re-route traffic with L7 & HAProxy: http://docs.rancher.com/rancher/v1.1/zh/cattle/health-checks/ http://docs.rancher.com/rancher/v1.1/zh/cattle/health-checks... The only thing even comparable to that in my mind would be Consul health checks with auto-discovery: https://www.consul.io/docs/guides/index.html https://www.consul.io/docs/guides/index.html
callumjones 10y ago
If you're truly running Docker in production you're probably not making use of both of these issues taken with Docker. No one would dare interact with a production cluster via the basic Docker CLI, instead you should be interacting with the orchestration technology like ECS, Mesos or Kubernetes. We are running ECS and we only interact with the Docker CLI to query specific containers or shut down specific containers that ECS has weirdly forgotten about. It definitely sounds like Swarm is not ready but I wouldn't say this is representative of running Docker in production: instead you should be running one of the many battle tested cluster tools like ECS, Mesos or Kubernetes (or GCE).
- drchiu 10y ago
  Agreed. Wouldn't use Docker CLI for production purposes. We use Cloud66 (disclaimer - not associated with them) to help with the deployment issues if any arise. Also we don't store DB in containers.
mrmondo 10y ago
We've been running in production across thousands of containers for long over a year now and it's been fantastic, not only a life saver of a deployment method but it's allowed for reliable, repeatable application builds.
coding123 10y ago
Just read through most of the thread, seems like a very large disconnect between people that are happy with Docker and those that are not. Personally, I've been extremely happy with it. We have one product in production using pre-1.12 swarm (will be upgrading in the next couple months) and most of our dev -> uat environments are now fully docker. It's been stable. On my personal projects I used Docker 1.12 and yes, after a few days things kerploded, but after upgrading to 1.12.1 things have been incredibly stable. For Nodejs apps I have been able to use Docker service replicas instead of Nodejs clustering, and been very happy with the results.
ilaksh 10y ago
Creating and maintaining a service cluster is hard. I dont think you should just take it back to the store if your magic wand has a hiccup.
damm 10y ago
Docker's far from ready from primetime. I'm sure everyone likes taking down their site to upgrade Docker to the latest release. Their mindset is very tool driven; if there's a problem let me just write a new tool to do that. Ease of use or KISS isn't a part of their philosophy
skrowl 10y ago
Is lxc any better? Are any of the issues in OP solved in lxc vs Docker?
bjt 10y ago
I think the underlying issue here is that no two people agree on what "Docker" is. Is it the CLI? Is it Docker Machine? Is it Docker Swarm? The container part of Docker works well. And they've ridden that hype wave to try to run a lot of other pieces of your infrastructure by writing another app and calling it Docker Something. Now everybody means a different subset when they say "Docker".
SilverSurfer972 10y ago
I think we should stop falling into the marketing greed of Docker to manage container at large scale. They want to get a grip of the corporate market and catch up with kubernetes at the cost of their containerization quality. Unfortunately with their current strategy they are loosing credibility as a containerization tool AND as the container orchestration tool they try to become. Using it solely as a containerization tool with k8s/ecs for the heavy lifting is the relevant way to go as for today IMO.
opHASnoName 10y ago
You can set the CLI via environment variables to use newer clients with older machines: export DOCKER_API_VERSION=1.23 You can export and import machines with this handy node js tool: https://www.npmjs.com/package/machine-share https://www.npmjs.com/package/machine-share
cerisier 10y ago
We have docker in production for about a year for our ENTIRE INFRASTRUCTURE, handling about 500 millions requests per month on 20 services including JVMS, Distributed Systems and so far it helped us spare so much time and money, I wouldn't consider going back without for a minute...
- bluecmd 10y ago
  How is that relevant? The author didn't complain about the performance - the complaint was about API stability. Which I agree with. Docker is really terrible at supporting their own APIs.

490 points by erlend_sh 10y ago

ktamiola 10y ago

Fair enough! There is also performance penalty and annoyingly complicated setups in certain cloud environments.

mkagenius 10y ago

Would love to hear thought from people who use it in production?

toomuchtodo 10y ago
Unless you're using it to compartmentalize CI jobs on Travis or Jenkins, don't use it in production (do use it in development for standing up local dev environments quickly). We do, and it's a wasted layer of abstraction. I predict the Docker fad will end in 2-3 years, fingers crossed.
- cgag 10y ago
  I think it's a pretty excellent layer of abstraction, as it was in the form of jails/zones. Even if we stuck to single node computing I think I think it'd be a powerful thing, but viewing a whole cluster of computers as a single compute resource is incredible. I think docker itself may fade but I view its popularity as jails finally catching on rather than a new bit of hype that's going to die down soon.
- jagermo 10y ago
  maybe not end, but scale down to realistic usage scenarios.
- eropple 10y ago
  > do use it in development for standing up local dev environments quickly You can do this, for sure, but, tbh, I'd still rather use Vagrant. I'm already provisioning things like my datastores using Chef (and not deploying those inside of Docker in production); I might as well repurpose the same code to build a dev environment. I do agree regarding using containers (I use rkt rather than Docker for a few reasons) for CI being a really good jumping-off point, though.
  - bdcravens 10y ago
    Vagrant is pretty heavy, but if you're already using Chef, makes sense. If you're deploying using Docker, then mirroring your environment in dev makes more sense. I've used both Vagrant and Docker for dev, and find Docker to be super fast and light. (though I've definitely run into some headaches with Docker Compose that I didn't see in using just Docker)
    - eropple 10y ago
      I think it's probably not worth worrying about "fast" when "slow" is "ten minutes once a week, maybe, if you're really destructive" and similarly worrying about "light" when you have 16GB of RAM and a quad-core processor; the heaviest dev environment/IDE I can find (IntelliJ and a couple hundred Chrome tabs) doesn't use eight of it. It comes off as super-premature optimization to me, and those optimizations that you would otherwise do in Docker (which are just building base images, whatever, Packer does this trivially) exist in a virtualized environment too. Running datastores in Docker is bonkers in the first place, which still leaves a significant place for an actual CM tool--so unless you are a particular strain of masochist, when you are replicating your local environment, you'll probably need it anyway.
      - bdcravens 10y ago
        > ten minutes once a week, maybe, if you're really destructive If you do a lot of active machine development, where you're iterating on that machine setup, it's an extremely slow process. Perhaps it would make sense to iterate in Docker, and then convert that process to Vagrant once you finish. > similarly worrying about "light" when you have 16GB of RAM and a quad-core processor It's a concern when you're launching many machines at once to develop against. Docker tends to only eat what it needs. Also, while I think everyone should have a 16GB machine, there's quite a few developers out there running 8 or 4GB machines.
        eropple 10y ago
        > Perhaps it would make sense to iterate in Docker, and then convert that process to Vagrant once you finish. I disagree; writing convergent Chef is easy. (Actually convergent Ansible isn't, but that's just one more reason not to use it!) So I just re-run on the same machine. When I'm confident, I nuke and rebuild to make sure that the first pass, and not just later passes, work fine. When I'm doing this for other people, they don't have to rebuild at all unless I have to break compatibility (which is rare, if ever). Using Docker for this means wasting time using Docker instead of doing what I'm actually going to send to production. In a cloud environment, I get so vanishingly close to nothing for using Docker that it makes little sense to use it in development, at all. (I've tried. It's a slower, clumsier experience that results in a duplication of effort even before you have to buy into the gong show of k8s or whatever you want to use in order to actually manage your disaster.) > It's a concern when you're launching many machines at once to develop against. My standard VM flight for my current project is eight machines. It uses four gigs of RAM, and that's mostly because I decided to be pretty generous with RAM usage (none of my services really need half a gig of RAM when running in a development environment; there isn't enough data in a dev environment even to fill the caches). If you are doing the "oh, we have a billionty microservices" thing, well, sure, you might stick yourself in a rough place, but in the overwhelming majority of situations you're also probably engaged in masturbatory exercises that don't help you ship and I think you have much bigger problems on your hands. As far as developers with unsuitable hardware go--meh. "My boss is too cheap, so I need to use the wrong tools" is, and thankfully for the foreseeable future will be, someone else's problem. And that someone else should quit. (I was being pretty conservative with 16GB of RAM and a quad-core; that's just my laptop, I have a workstation for serious work.)
- jacques_chester 10y ago
  > I predict the Docker fad will end in 2-3 years, fingers crossed. I work for what is, ostensibly, a competitor and I don't agree. Containers are here to stay. Docker will pretty much have a brand lock on that for a long time.
- bdcravens 10y ago
  > I predict the Docker fad will end in 2-3 years, fingers crossed. Containers are definitely here to stay. Google, Heroku, and others have been using them for years. Obviously Docker is just a subset, and it might fade, but I think it's pretty well established as a defacto standard.
  - toomuchtodo 10y ago
    Containers have been around for more than a decade as lxc. I'm saying Docker itself. It's hard to make money when you're running on an open standard that anyone can build tooling around.
- deleted 10y ago
  [deleted]
DigitalJack 10y ago
I use it in production by way of kubernetes. It's been great so far.
olalonde 10y ago
I used it in production with CoreOS and now Kubernetes. I almost never touch the Docker CLI and haven't tried Swarm. I think a lot of developers mistakenly think they can get to a production environment in a few Docker commands which is definitely not possible today.
- Twirrim 10y ago
  From an ops side, that's worrying. Yes, in an absolutely ideal world no one ever should touch a production environment, and ideally in every situation you find yourself having to touch a production system you should then be creating at the very least a backlog item to figure out how to avoid doing it in the future, but the reality is some kinds of troubleshooting are hard to do without having access. Generally the kinds of situations where you need that access are ones when you want access the quickest and easiest, i.e. the world is burning somehow.
  - lobster_johnson 10y ago
    OP wasn't talking about access, but about the effort needed to set up ("get to") a production environment. Of course Kubernetes/Docker gives you full access to your production environment.
TheHydroImpulse 10y ago
We're using Docker pretty extensively at Segment with over 1,200 containers running in production with ECS (total count, not uniqueness). We haven't touched Swarm and are staying a couple versions behind. With that said, we're pretty happy with the setup right now.
- NathanKP 10y ago
  Airtime is also using ECS to run docker containers in production. We ran into a few issues early on due to early adopter status, but the AWS team was very quick to add many features we needed. I have no complaints about ECS now, and in fact I'd say that ECS solves a lot of the problems that the author complains here. I'd say that Swarm is not production ready, but docker itself as an engine for running a container is fine. You just need to build your own orchestration layer, or use something like ECS. More details on Airtime's docker setup here: https://techblog.airtime.com/microservice-continuous-integration-made-easy-with-aws-ecs-10d470e31af0 https://techblog.airtime.com/microservice-continuous-integra...
  - hosh 10y ago
    Maybe things has changed since Apr 2015, but I found ECS pretty limited and K8S solving those limitations. One big issue was container-to-container addressing. It was not available last year when I put an ECS cluster into production. Has it been added since?
    - NathanKP 10y ago
      Direct container to container communication isn't the recommended pattern when using ECS. Instead you let ECS associate with either a legacy Elastic Load Balancer, or one of the new Application Load Balancers, which can direct requests to different classes of containers based on the request path, allowing you to take a set of containers of different types and meld their http resources together into one API on one domain. If one container needs to talk to another container of a certain type it just uses the load balancer, and the load balancer takes care of directing the request to the other container which may be on the same machine, or another machine, or maybe even in a different cluster. Obviously this forces you into a fully stateless design because the load balancer round robins all the requests across random containers, but its very fail proof, and gives you high resiliency.
ceocoder 10y ago
We use it via GKE in prod. Zero complaints so far.
knz 10y ago
I've posted about this before, but my work (non IT Department at a State Govt agency) has been using Docker in production for about a year. We are small by HN standards (~12 apps on 5 hosts that are a mix of API's, front ends, and several more complex GIS apps). Moving to Docker was a massive improvement for us in terms of deployment and development. Deploying with Docker is ridiculously easy (no more taking a node out of HAProxy for an update, rollbacks are painless, and we love that what you deploy is what you get - there is no question over what the container is actually doing since you define it in the Dockerfile). I can't remember the last time we had a bad deployment or even cared about making a change to production. It's that reliable. It's also been a massive improvement for QA and development because of the ability to standardize the container using the Dockerfile. We've been using CentOS for the production hosts and CentOS/OSX for developing on. There were a couple of minor issues when we started but the benefits outweighed the negatives then. It's been completely painless since Docker came out of beta. We have some containers that are deployed daily and others that run for months without being touched. I've never quite understood the negative attitude towards Docker. Perhaps we are too small to see many of the issues that people often complain about, but moving to Docker was a great decision in our office.
- cpitman 10y ago
  Are you installing docker out of the CentOS repositories, or from Docker Inc's repository? The CentOS version is based on the work Red Hat does to backport patches and create more production ready versions of Docker, so that might be why you are having a better experience.
  - knz 10y ago
    We are indeed using the CentOS repositories. The current version on our production servers is 1.10.3.

anotherdpk 10y ago

> Each version of the CLI is incompatible with the last version of the CLI. Sure, but I don't think this is a show stopper. You can and should only carefully upgrade between versions of Docker (and other mission-critical software). The process is functionally identical to the process you'd use to perform a zero-downtime migration between versions of the Linux kernel -- bring up a new server with the version of Docker you want to use, start your services on that new server, stop them on the old server, shut down the old server, done. I don't mean to suggest this is trivial, only to suggest that it is no more complicated than tasks that you/we are already expected to perform.

coldtea 10y ago
>Sure, but I don't think this is a show stopper. You can and should only carefully upgrade between versions of Docker (and other mission-critical software). Whether I upgrade carefully or not, the Docker CLI should work with all versions of Docker.
- anotherdpk 10y ago
  I don't get why this is a sticking point. There's no reason to update the Docker CLI independent of the Docker server, and it would be silly to run two versions of Docker on the same host, so why would it matter?
StreamBright 10y ago
I think you are missing the point. It is not about upgrading the production cluster, more like erasing muscle memory and re-learning docker cli again. How often do you re-learn grep, awk, ls, ps, or other command line tools? How mad would you be if these were developed the same manner as Docker CLI? I would be definitely not happy.

deleted 10y ago

[deleted]

okket 10y ago

Two days ago there was a fairly long discussion about a similar argument ("The sad state of Docker") https://news.ycombinator.com/item?id=12364123 https://news.ycombinator.com/item?id=12364123 (217 comments)

deleted 10y ago
[deleted]

perturbation 10y ago

Most of the complaints I've seen recently about using Docker are about the immaturity of Docker Swarm. Can this be mitigated by using Docker with Kubernetes / Mesos / Yarn? If it's truly a problem with the containerization format / stability with the core product, I'm not sure what a good alternative would be. I see a lot of praise for rkt but the ecosystem and tooling around it are so much smaller than that for Docker.

lobster_johnson 10y ago
Using it with Kubernetes definitely helps. One reason is that if you have enough surplus nodes, then Docker misbehaving on one of them shouldn't screw up anything; Kubernetes is really good at shuffling things around, and you can "cordon off" problem nodes to prevent them from being used for scheduling.

deleted 10y ago

[deleted]

joshka 10y ago

> Each version of the CLI is incompatible with the last version of the CLI. Run the previous version of the cli in a container on your local machine. https://hub.docker.com/_/docker/ https://hub.docker.com/_/docker/ $ docker run -it --rm docker:1.9 version

deleted 10y ago
[deleted]
Chronic9q 10y ago
You literally just proved his point.
- jcoffland 10y ago
  I don't think he was arguing against it.
  - Twirrim 10y ago
    It's so hard to tell. I hope they were joking.
- joshka 10y ago
  I'm not sure that I did. Let's expand out my point to see if I'm on the same page as the replies here. 1. TFA complains that versions of servers require corresponding client tools to be installed. 2. TFA says he loves docker in dev, but not in prod. 3. I state (omitting many steps): install the latest docker in your dev environment (or wherever you're connecting to prod), spin up a small docker image to run the appropriate version cli. This is cheap, easy, easy to understand why it works, consistent with points 1 and 2. What am I missing?
inlined 10y ago
This is both very clever and horrifying. This obviously works for development but doing a rolling update of your fleet is stressful without the additional wrinkle of progressively updating which tool you need to inspect or control that fleet.

asitdhal 10y ago

Do they speak English or the caller is expected to know French ?

steveklabnik 10y ago
Did you mean to comment on https://news.ycombinator.com/item?id=12376886 https://news.ycombinator.com/item?id=12376886 ?
- asitdhal 10y ago
  yes, thanks

dawnerd 10y ago

I spent close to 12 hours yesterday trying to get a fairly simple node app to run on my mac yesterday. Turned out I had to wipe out docker completely and reinstall. Keep in mind this is their stable version thats no longer in beta. I've just run into too many documented bugs for me to consider it stable. I wouldn't even say it should be out of beta. The issues here are the real telling story: I spent close to 12 hours yesterday trying to get a fairly simple node app to run on my mac yesterday. Turned out I had to wipe out docker completely and reinstall. Keep in mind this is their stable version thats no longer in beta. I've just run into too many documented bugs for me to consider it stable. I wouldn't even say it should be out of beta. The issues here are the real telling story. https://github.com/docker/for-mac/issues https://github.com/docker/for-mac/issues I love docker, it's amazing when it works. It's just really not there yet. I get that their focus is on making money right now, but they need to nail their core product first. I honestly don't care about whatever cloud platform they're building if their core app doesn't even work reliably.

Ninn 10y ago
Heads up -- I think you messed up your copy/paste.
- coldtea 10y ago
  That, or the issues here are the real telling story.
serverholic 10y ago
I'm a fan of Docker in general but I'm amazed by some of the poor choices they've made with Docker for Mac. - Stable? That's honestly laughable. It is nowhere near stable. As an example, I was trying to upload images to a third-party image registry but the upload speed was ridiculously slow. It took me forever to figure out but it turned out I needed to completely reinstall docker for mac. - They had a command-line tool called pinata for managing daemon settings in docker for mac. They chose to get rid of it. Not only did we lose a way to declaratively define and set configuration but the preferences window has no where near all of the daemon settings that are available. - The CPU usage is still crazy. I regularly get 100% CPU usage on 3/4 of my CPU cores while starting up just a few containers. Even after the containers have started it will idle at 100% 1/4 cores. - It needs to be reinstalled regularly if you are using it on a daily basis. Otherwise it will get slower and slower over time. See my first complaint. - The GUI (kitematic) will randomly disconnect from the daemon forcing me to restart the GUI repeatedly. - They really need some sort of garbage collector with adjustable settings. With the default settings the app will just keep building and building images and eventually fill up, crash, slow down, etc. How is that acceptable? What other apps do that? Like I said, I like docker in general. I think they are tackling some very hard problems and definitely experiencing some growing pains from such crazy growth. However, at some point they need to take a step back and focus on the core of what they offer and make it as simple, and rock solid as possible. As another example, they still haven't added a way to compress, and/or flatten docker images. No wonder docker for mac slows down after regular use when it's building 1GB+ images for simple things.

kozikow 10y ago

I am using kubernetes instead of docker swarm to orchestrate docker images and all points mentioned in this article do not apply. My cluster is small - I have <100 machines at peak, but so far it feels ready for prime time. There are parts of docker that are relatively stable, have many other companies involved, and have been around for a while. There are also "got VC money, gotta monetise" parts that damage the reputation of stable parts.

hosh 10y ago
I've used K8S before, and those things are smooth once installation happen. We have a project which is looking at using Docker Swarm on a two node set up. The idea is that this will give us a baseline, and since it is looking more and more we'll move to GKE, this sets us up for a later move into GKE. I thought Docker Swarm would be mature enough to handle two nodes, for a short time (maybe half a year) until we make the leap to GKE. But based on the reports coming in from the wild, that's looking less and less likely. We hired a contractor to do this part. My team is so resource constrained that I don't have time for this, so we farmed it out. But now I'm thinking, the risk of project failure is much higher than I thought, made worse that the contractor is also showing signs of having poor communication skills. (I would rather be updated on things going wrong than to have someone try to be the hero or cowboy and figure it all out).
- sandGorgon 10y ago
  we are exactly in your boat. However, we are signing up for kubernetes since we want to keep the GKE option open. The only reason why we didnt go ahead is because of broken logging in k8s - https://github.com/kubernetes/kubernetes/issues/24677 https://github.com/kubernetes/kubernetes/issues/24677
  - kozikow 10y ago
    I went through a few iterations for logging, but now I settled on using built-in GKE logging. Stdout logs from my containers are picked up by kubernetes and forwarded to stackdriver. Since it's just stdout I do not create too much lock-in. I use stackdriver dashboard for investigating recent logs and BigQuery exporter for complex analysis. My stdout logs are jsons, so I can export extra metadata without relying on regexps for analysis - I use https://pypi.python.org/pypi/python-json-logger https://pypi.python.org/pypi/python-json-logger.

ycombinatorMan 10y ago

Aye, theres a lot of important bugs that are just sitting there

jcoffland 10y ago

One of Docker's biggest problems is that internally they have fomented a culture of "users are stupid" which is immediately apparent if you interact with their developers on GitHub.

StreamBright 10y ago
This is pretty dangerous path to failure. If you look at the most successful companies it is obvious that they respect the user base using their service. Software is no exception to that, you have to make it user friendly, regardless if your user base is mostly software and systems engineers.
- enraged_camel 10y ago
  I don't think it's so cut-and-dry. I mean, does Oracle respect its users and is it user-friendly? What about Epic, the dominant software system in the healthcare industry? Or Microsoft SharePoint?
  - ghufran_syed 10y ago
    "If you look at the most successful companies it is obvious that they respect the user base using their service" - I think a better way of putting it would be that "successful companies respect their customers". In the case of software tools like Docker, the customers generally are the users. In the case of Epic, an electronic health record, the buyers are senior management, the users are the clinical staff such as the nurses, doctors and techs. So Epic does give the senior management what they want (buzzword compliance, a safe choice that it's hard to get fired for), but doesn't give clinical staff what they want..because they are not the ones making the buying decision.
  - StreamBright 10y ago
    Most, not all. I think you are still better of listening to your customers.
danielrhodes 10y ago

jacques_chester 10y ago

Make no mistake: Docker Inc has a lot of excellent engineers. But there's also a landrush going on. Everyone has worked out that owning building blocks isn't where the money is. The money is in the platform. Businesses don't want to pay people to assemble a snowflake. They want a turnkey. CoreOS, Docker and Red Hat are in the mix. So too my employers, Pivotal, who are (disclosure) the majority donors of engineering for Cloud Foundry. IBM is also betting on Cloud Foundry with BlueMix, GE with Predix, HPE with Helion and SAP with HANA Cloud Platform. You're probably sick of me turning up in threads like these, resembling one of the beardier desert prophets, constantly muttering "Cloud Foundry, Cloud Foundry". It's because we've already built the platform. I feel like Mugatu pointing this out over and over. We're there! No need to wait! A distributed, in-place upgradeable, 12-factor oriented, containerising, log-draining, routing platform. The intellectual lovechild of Google and Heroku. Basically, it's like someone installed all the cool things (Docker, Kubernetes, some sort of distributed log system, a router, a service injection framework) for you. Done. Dusted. You can just focus on the apps and services you're building, because that's usually what end users actually care about. And we know it works really well. We know of stable 10k app instance scale production instances that are running right now. That's real app instances, by the way: fully loaded, fully staged, fully logging, fully routed, fully service injected, fully distributed across execution cells. Real, critical-path business workloads. Front-page-of-the-WSJ-if-it-dies workloads. Our next stretch goal is to benchmark 250k real app instances. If you need more than 250,000 copies of your apps and services running, then you probably have more engineers than we do. Though I guess you could probably stretch to running two copies of Cloud Foundry, if you really had to. OK, a big downside: BOSH is the deployment and upgrade system. It's not very approachable, in the way that an Abrams main battle tank is less approachable than a Honda Civic (and for a similar reason). We're working on that. The other downside: it's not sexy front-page-of-HN tech. We didn't use Docker, it didn't exist. Or Kubernetes, it didn't exist. We didn't use Terraform or CloudFormation ... they didn't exist. Docker will get all this stuff right. I mean that sincerely. They've got too many smart engineers to fail to hammer it down. More to the point, Docker have an unassailable brand position. Not to be discounted. Microsoft regularly pulled did this kind of thing for decades and made mad, crazy cash money the whole way along.

sandGorgon 10y ago

urvader 10y ago

The title should say: Docker Swarm is not ready for primetime. We have used Docker in production for more than two years and there has been very few issues overall.

StreamBright 10y ago

I have the same experience. I am trying to set a new node where I accidentally installed 1.10 and CLI does not work. Looking for some thing on the internet how do X with Docker, only articles available for previous versions. I mean seriously, command line development is not supposed to be this hard, get a set of switches and stick to it unless something major forces you to change. If you shuffle around CLI switches between minor releases nobody is going to be happy.

exratione 10y ago

So far I'm feeling pretty good about the decision to skip the first generation containerization infrastructure. At the outset it had the look of something that wasn't an advance over standard issue virtualization, in that it just shuffled the complexity around a bit. It doesn't do enough to abstract away the ops complexity of setting up environments. I'm still of the mind, a few years later, that the time to move on from whatever virtualization approach you're currently using for infrastructure and development (cloud instances, virtualbox, etc), is when the second generation of serverless/aws-lambda-like platforms arrive. The first generation is a nice adjunct to virtual servers for wrapping small tools, but it is too limited and clunky in its surrounding ops-essential infrastructure to build real, entire applications easily. So the real leap I see ahead is the move from cloud servers to a server-free abstraction in which your codebase, from your perspective, is deployed to run as a matter of functions and compute time and you see nothing of what is under that layer, and need to do no meaningful ops management at all.

dmourati 10y ago
This sounds like I wrote it.

siliconc0w 10y ago

We use it in production. It generally works if: * you don't use it to store data * don't use 'ambassador', 'buddy', or 'data' container patterns. * use tooling available to quickly and easily nuke and rebuild docker hosts on a daily or more frequent basis. * use tooling available to 'orchestrate' what gets run where - if you're manually running containers you're doing it wrong. * wrap docker pulls with 'flock' so they don't deadlock * don't use swarm - use mesos, kube, or fleet(simpler, smaller clusters)

bphogan 10y ago
woah, wait..... It "geneerally works if" you " rebuild docker hosts on a daily or more frequent basis." Perhaps I'm misunderstanding, but needing to rebuild my prod env several times a day seems pretty "not ready for prime time" to me. That's like when we'd say that Rails ran great in production in 2005, as long as you had a cron task to bounce fastCGI processes every hour or so. So, can you elaborate on why rebuilding the containers is good advice?
- jjn2009 10y ago
  The host isn't the container itself. They want to re-provision the host likely not because of something wrong with the application but instead docker is in some state which is non-recoverable, or at least not recoverable by automatic means.
- jacques_chester 10y ago
  The security exec at Pivotal, where I work, has been talking about "repaving" servers as a security tactic (along with rotating keys and repairing vulnerabilities).[0] The theory runs that attackers need time to accrue and compound their incomplete positions into a successful compromise. But if you keep patching continuously, attackers have fewer vulnerabilities to work with. If you keep rotating keys frequently, the keys they do capture become useless in short order. And if you rebuild the servers frequently, any system they've taken control of simply vanishes and they have to start from scratch. I'm not completely sold on the difference between repair and repave, myself. And I expect that sophisticated attackers will begin to rely more on identifying local holes and quickly encoding those in automated tools so that they can re-establish their positions after a repaving happens. But it raises the cost for casual attackers, which is still worthy. [0] https://medium.com/built-to-adapt/the-three-r-s-of-enterprise-security-rotate-repave-and-repair-f64f6d6ba29d#.k3dj6wlp1 https://medium.com/built-to-adapt/the-three-r-s-of-enterpris...

dcosson 10y ago

> Each version of the CLI is incompatible with the last version of the CLI. I'm pretty sure that as long as the CLI version is >= the server version, you can set the DOCKER_VERSION env var to the server version and everything works. I haven't used this extensively, so maybe there are edge cases or some minimal supported version of backwards compatibility?

sandGorgon 10y ago

it seems that none of the container frameworks are generally ready. Take for example k8s - I just started exploring it as something we could move to. https://github.com/kubernetes/kubernetes/issues/24677 https://github.com/kubernetes/kubernetes/issues/24677 - logging of application logs is an unsolved problem. And most of the proposals talk about creating yet another logger...rather than patching journald or whatever exists out there. For those of you who are running k8s in production - how are you doing logging ? does everyone roll their own ?

felixgallo 10y ago
logging of application logs generally in the world is an unsolved problem, but I feel like k8s does better than most; fluentd is the default, and then logs are collected by whatever end system you like the best that's appropriate for your needs (e.g. GKE -> fluentd -> Cloud Logging, and I've heard of K8S -> fluentd -> ELK).
- sandGorgon 10y ago
  since most people use k8s in the context of GKE.. there seems to be some magic that Google has put in. But if you read the bug, this doesnt work quite well in the wild. In fact, an entire spec is being written on it and right now the debate is whether to forward to journald and have it proxy further or reimplement a new logging format.
  - felixgallo 10y ago
    well, there's definitely systemd people pushing for journald, that's for sure. The first comment talks about how it doesn't fit, though. I wonder what in particular about the solution doesn't work for those people.
lobster_johnson 10y ago
Logging is definitely not an unsolved problem with K8s. It's trivial to set up Fluentd to snarf all container logs into some destination, such as Graylog, ElasticSearch or a simple centralized file system location. The Github issue talks a lot about "out of the box" setup via kube-up. If you're not using kube-up (and I wouldn't recommend using it for production setups), the problem is rather simple. The logging story could be better, but it's not "unsolved".

jbverschoor 10y ago

Docker is the new mongodb. Let's just use freebsd jails + postgresql.

abz10 10y ago
+1 I got sick of Docker and switched to FreeBSD. Far better experience.
- X86BSD 10y ago
  Jails and Tredly.com or SmartOS. Anything else is busted by design. Enjoy your poop sandwich.

morgante 10y ago

This article should really be about Docker Swarm not being ready for production. It's much newer technology than Docker and is predictably brittle. The only points made against Docker proper are rather laughable. You shouldn't be remotely administering Docker clusters from the CLI (use a proper cluster tool like Kubernetes), and copying entire credentials files from machine to machine is extremely unlikely/esoteric. Docker, with Kubernetes or ECS, is totally suitable for production at this point. Lots and lots of companies are successfully running production workloads using it.

atemerev 10y ago
> You shouldn't be remotely administering Docker clusters from the CLI (use a proper cluster tool like Kubernetes) Docker Swarm is advertised as stable, production ready cluster management solution. Then, if you actually try to use it, it is very NOT. Kubernetes is great, but it feels like adding another layer to the system, and it is not always a good thing (especially if you are on AWS and have to work with _their_ infrastructure management too).

zwischenzug 10y ago

Can anyone using Swarm seriously in production post any account of their experiences here? Thanks.

nostrebored 10y ago
I doubt you'll get any replies. We tried using swarm in production, but could not ever get a reliable, working component. Failures of the worst kind would happen - networking would work 1/10 times meaning that a build could've conceivably gotten through CI.

markbnj 10y ago

I have run docker in production at past employers, and am getting ready to do so again at my current employer. However I don't run it as a native install on bare metal. I prefer to let a cloud provider such as Google deal with the infrastructure issues, and use a more mature orchestration platform (kubernetes). The author's complaints are valid, and the Docker team needs to do a better job on these issues. Personally I am going to be taking a close look at rkt and other technologies as they come along. Docker blew this technology open by making it more approachable but there is no reason to think they are going to own it. It's more like databases than operating systems.

lugg 10y ago
I'm trying to get started with rkt right now but it's (understandable) lack of maturity is a bit daunting, I think some usability issues need to be handled / offloaded to some other tools. And acbuild severly needs caching built in. Disclaimer, new to containers, if it sounds like I'm doing something wrong let me know, it certainly feels like I'm missing something right now.
- jzelinskie 10y ago
  You'll see a maturity in this space after the Open Containers Initiative standardizes the container image format. Then, developers can focus on UX improvements rather than worrying about what the build tool should even be producing.
  - jmspring 10y ago
    Reliance on initiatives and standards bodies to provide a guidance for maturity, at least in recent years, is a fools errand. They usually end up rubber stamping what's been the norm and work back from there.
    - cesnja 10y ago
      That's true, OCF is more or less the same as the Docker container format.

bogomipz 10y ago

Docker not being ready for primetime and Swarm not being ready for primetime are two different things no? As for the cli compatibility issues, don't most people use an orchestration tool like ansible/chef/puppet etc to manage their fleet? I'm not sure I think the title of the post is accurate.

Annatar 10y ago
don't most people use an orchestration tool like ansible/chef/puppet etc to manage their fleet? I don't. All my configuration management is done with OS packages. And once you start making OS packages to do mass-scale configuration, the whole premise of Docker becomes pointless. If I have OS packages, and I can provision them using KickStart, or JumpStart, or imgadm(1M) + vmadm(1M), without using or needing any code, then what exactly do I need Docker for?

jasoncchild 10y ago

"Just imagine if someone emailed you a PDF or an Excel file and you didn't have the exact verion of the PDF reader or Excel that would open that file. Your head would blow clean off." Obviously this person has never spent a good deal of time dealing with AutoCAD...

saidajigumi 10y ago
Or more to the point early versions of the MS Office suite, where major updates could and did create backwards-incompatible saves. Users had to be careful that files intended for broader distribution were saved in a sufficiently old/compatible format version. Which is to say: this is a sign that the involved technologies are still rapidly maturing.
mattdennewitz 10y ago
or excel!

kev009 10y ago

I wonder how much suffering would be alleviated in most mid-level IT organizations if they just used Joyent/SmartDataCenter, and I say this as a FreeBSD developer with no affiliation.

Annatar 10y ago
A lot. zones in SmartOS provide full-blown UNIX servers running at the speed of the bare metal, but in complete isolation (no need for hacks like "runC", containers, or any other such nonsense). Packaging one's software into OS packages provides for repeatability: after the packages are installed in a zone, a ZFS image with a little bit of metadata can be created, and imported into a local or remote image repository with imgadm(1M). https://wiki.smartos.org/display/DOC/Managing+Images https://wiki.smartos.org/display/DOC/Managing+Images That's it. It really is that simple.

hellofunk 10y ago

I agree in general, and find Docker one of those technologies that does not (yet) live up to the hype.

brint 10y ago

For the versions issue, check out dvm: https://github.com/getcarina/dvm https://github.com/getcarina/dvm While versions are an issue, it's at least a reasonable way to work around it.

MrFurious 10y ago

Docker, container for hipsters that doesn't know deploy a simple linux virtual server.

ldehaan 10y ago

I have several past clients running docker in production just fine. At my current job we run nearly all our services in docker. I've replied to this type of comment on here at least a dozen times, it has nothing to do with docker, it is a lack of understanding how it all works. Understand the history, understand the underlying concepts and this is no more complex than an advanced chroot. Now on the tooling side, I personally stay away from any plug-ins and tools created by the docker team, they do docker best, let other tools manage dockers externalities. I've used weave since it came out, and it's perfect for network management and service discovery. I prefer to use mesos to manage container deploys. There is an entirely usable workflow with docker but I like to let the specialists specialize, and so I just use docker (.10.1 even), because all the extra stuff is just making it bloated. I'm testing newer versions on a case by case basis, but nothing new has come out that makes me want to upgrade yet. And I'll probably keep using docker as long as it stays separate from all the cruft being added to the ecosystem.

acd 10y ago

Here are alternatives to Docker One can use Ubuntu LXD which is Linux containers built on top of LXC but with ZFS as storage backend. LXD can also run Docker containers. http://www.ubuntu.com/cloud/lxd http://www.ubuntu.com/cloud/lxd One can also use Linux containers via Kubernetes by Google. http://kubernetes.io/ http://kubernetes.io/

20yrs_no_equity 10y ago

Network partitions really do happen! They are often short, but if you can't recover from them, then you shouldn't call yourself a distributed system. I am shocked at how fragile etcd is in this way. I was hoping docker swarm was better, but I'm not surprised (alas) to find out that it has the same problem. I'm about ready to build my own solution, because I know a way to do it that will be really robust in the face of partitions (and it doesn't use RAFT, you probably should not be using RAFT, I've seen lots of complaints about zookeeper too. I've done this before in other contexts so I know how to make it work, but so have others so why are people who don't know how to make it work reinventing the wheel all the time?)

forktheweb 10y ago
amazon efs + duplicity p.o.t. snapshots -> s3 + docker = win
helloiamaperson 10y ago
I'd love to hear more about your solution. Are you saying that you've created an algorithm distinct from paxos/raft/zab that's more robust?
ldehaan 10y ago
Check out weave, it's great for service discovery. Zookeeper is much better than etcd imo.

jokoon 10y ago

I watched again an explanation of what docker really is, it just seems to be this awesome thing that solve the very hard problem of inter-compatiblity between systems. I always tend to question how and why a developer had to use docker instead of making choice that would avoid it. It's not surprising docker can't always work, but it's nice to see that programmers are winning. I guess future OS designers and developers will try to encourage for more inter compatibility if possible. That's really a big nerve.

stevesun21 10y ago

As I worked for my ex-employer, we use docker based Elatic beanstalk serve over millions requests across three services per seconds.

rjurney 10y ago

Docker Swarm is definitely not production ready. Try to run any service that requires communication among nodes and you will agree. It works fine for web servers, but that is about it. DC/OS is emerging as the go-to way to deploy docker containers at scale in complex service combinations. It 'just works' with one simple config per service.

sergiotapia 10y ago

We've tried to use Docker a couple of times, but it was always much more trouble than it was worth. It was always some edge case that caused it to not work as expected one developer machine or another. After about 2 years of giving it another shot on and off, we just gave up. And it's not like we were doing something crazy, just a typical documented "run this rails app" type thing. I would definitely not use this in production for anything based on my experience.

sjellis 10y ago
Were you using Linux developer machines? Docker, Inc. provide ready-made installers for Windows and Mac - the Docker ToolBox makes setting up on those OSes very easy these days. I haven't tried the new products that replace ToolBox yet. I can imagine it being harder on some Linux distributions because your host system will have to have all of the kernel bits etc. Ubuntu 14.04 and above or current versions of Fedora should be good - the distributions explicitly support Docker, and Docker, Inc. provide packages so that you can run either the distribution packages or the Docker, Inc. ones.
- sergiotapia 10y ago
  We are all on Macs.

deleted 10y ago

[deleted]

forktheweb 10y ago

I would say that my experience with Docker has been fantastic. I run over 10 Ubuntu Trusty instances on EC2 as 8G instances, mounted with NFS4 to EFS. This makes it super simple to manage data across multiple hosts. From that you can run as many containers as you like, and either mount them to the EFS folder, or just spawn them with data-containers, then export backups regularly with something like duplicity. I use rancher with it, and it's retarded simple using rancher/docker compose. For a quick run-down see: https://github.com/forktheweb/amazon-docker-devops https://github.com/forktheweb/amazon-docker-devops

_jezell_ 10y ago

The Docker team is doing more than anyone to move container technology forward, but orchestration is a much harder problem to solve than wrapping OS APIs. I wish they would stick to the core, and let others like the Kubernetes team handle the orchestration pieces. Swarm is hard to take seriously right now. I'm not sure bundling it into the core was the best way to handle it.

coding123 10y ago
I disagree. Kube, Mesos, etc.. are all great I'm sure. I had Kub documentation open in the background for about a week or two while doing other things, trying to get a handle on installation, deployment, how it will fit my use-cases... I kept wanting to dip in, but kept getting pounded by tons of heavy dependencies I would have had to learn. After Swarm Mode was released I had a cluster deploying things locally within 2 hours of reading the documentation. From there was able to map all of our use-cases on top (rolling updates, load balancing, etc...). I totally get that it's "hard to take seriously" perspective with some of the early glitching, but Kube, Mesos, etc.. needed this egg on their faces to realize how easy the installation and getting things up and running should be. If nothing else, that alone will make those other products better.

forktheweb 10y ago

forktheweb 10y ago
Just want to add that I also use it on windows without barely any issues (win 10 x64). I'm not sure how stable it is on Mac OSX but Kitematic is pretty sweet. The only problems I've had with Docker container are those where processes get stuck inside the container and the --restart=always flag is set. When this happens it means that if you can't force the container to stop, when you reboot the defunct container will restart anyway and cause you the same issue... My solution to this has been to just create a clean AMI image with ubuntu/rancher/docker and then nuke the old host when it gives me problems. This is made even easier if you use EFS because it's literally already 100% setup once you launch a replacement instance. Also, you can do automatic memory-limiting and cpu-limiting your nodes using rancher-compose and health-checks that re-route traffic with L7 & HAProxy: http://docs.rancher.com/rancher/v1.1/zh/cattle/health-checks/ http://docs.rancher.com/rancher/v1.1/zh/cattle/health-checks... The only thing even comparable to that in my mind would be Consul health checks with auto-discovery: https://www.consul.io/docs/guides/index.html https://www.consul.io/docs/guides/index.html

callumjones 10y ago

If you're truly running Docker in production you're probably not making use of both of these issues taken with Docker. No one would dare interact with a production cluster via the basic Docker CLI, instead you should be interacting with the orchestration technology like ECS, Mesos or Kubernetes. We are running ECS and we only interact with the Docker CLI to query specific containers or shut down specific containers that ECS has weirdly forgotten about. It definitely sounds like Swarm is not ready but I wouldn't say this is representative of running Docker in production: instead you should be running one of the many battle tested cluster tools like ECS, Mesos or Kubernetes (or GCE).

drchiu 10y ago
Agreed. Wouldn't use Docker CLI for production purposes. We use Cloud66 (disclaimer - not associated with them) to help with the deployment issues if any arise. Also we don't store DB in containers.

mrmondo 10y ago

We've been running in production across thousands of containers for long over a year now and it's been fantastic, not only a life saver of a deployment method but it's allowed for reliable, repeatable application builds.

coding123 10y ago

Just read through most of the thread, seems like a very large disconnect between people that are happy with Docker and those that are not. Personally, I've been extremely happy with it. We have one product in production using pre-1.12 swarm (will be upgrading in the next couple months) and most of our dev -> uat environments are now fully docker. It's been stable. On my personal projects I used Docker 1.12 and yes, after a few days things kerploded, but after upgrading to 1.12.1 things have been incredibly stable. For Nodejs apps I have been able to use Docker service replicas instead of Nodejs clustering, and been very happy with the results.

ilaksh 10y ago

Creating and maintaining a service cluster is hard. I dont think you should just take it back to the store if your magic wand has a hiccup.

damm 10y ago

Docker's far from ready from primetime. I'm sure everyone likes taking down their site to upgrade Docker to the latest release. Their mindset is very tool driven; if there's a problem let me just write a new tool to do that. Ease of use or KISS isn't a part of their philosophy

skrowl 10y ago

Is lxc any better? Are any of the issues in OP solved in lxc vs Docker?

bjt 10y ago

I think the underlying issue here is that no two people agree on what "Docker" is. Is it the CLI? Is it Docker Machine? Is it Docker Swarm? The container part of Docker works well. And they've ridden that hype wave to try to run a lot of other pieces of your infrastructure by writing another app and calling it Docker Something. Now everybody means a different subset when they say "Docker".

SilverSurfer972 10y ago

I think we should stop falling into the marketing greed of Docker to manage container at large scale. They want to get a grip of the corporate market and catch up with kubernetes at the cost of their containerization quality. Unfortunately with their current strategy they are loosing credibility as a containerization tool AND as the container orchestration tool they try to become. Using it solely as a containerization tool with k8s/ecs for the heavy lifting is the relevant way to go as for today IMO.

opHASnoName 10y ago

You can set the CLI via environment variables to use newer clients with older machines: export DOCKER_API_VERSION=1.23 You can export and import machines with this handy node js tool: https://www.npmjs.com/package/machine-share https://www.npmjs.com/package/machine-share

cerisier 10y ago

We have docker in production for about a year for our ENTIRE INFRASTRUCTURE, handling about 500 millions requests per month on 20 services including JVMS, Distributed Systems and so far it helped us spare so much time and money, I wouldn't consider going back without for a minute...

bluecmd 10y ago
How is that relevant? The author didn't complain about the performance - the complaint was about API stability. Which I agree with. Docker is really terrible at supporting their own APIs.