20 ms·

What I found wrong in Docker 1.12

390 points by rusher81572 10y ago

CSDude 10y ago
I also assumed the new easy swarm would be easier, but the Multi-host network only works with the newly introduced service command only, it does not work with regular docker run command, which is disappointing because you still neeed a third party key-value store for it, but not for docker service. It literally took me hours to realize that was missing. I think 1.12 was just rushed to show it in DockerCon, normally major releases were mostly bug-free and worked as intended.
- rusher81572 10y ago
  I feel the pain for I was in the same boat.
epberry 10y ago
I share the pain of this post - except my run-in with 1.12 occurred with Docker for Windows. The shared volumes and host networking were totally nondeterministic. I don't think I had really ever experienced software which seemed to fail so randomly. On the other hand I think Docker can probably be forgiven for my particular frustration. For one, the software was in beta. And two, working with Windows and all the different flavors must be a nightmare.
- djs55 10y ago
  (I work at Docker) If you haven't already and can spare the time, could you file a pair of issues: perhaps one for the shared volume problem and another for the networking on https://github.com/docker/for-win https://github.com/docker/for-win ? We've been fixing bugs in both areas and the fixes should arrive in the beta channel over the next few updates. Thanks for your patience!
hackits 10y ago
Don't know what all the fuss is about when essentially you're getting Docker for free?
- rusher81572 10y ago
  It is not about cost. It is about marketing and generating all the hype over something that clearly does not function properly. They act like they are laying golden eggs when in fact they are broken eggs.
  - hackits 10y ago
    There is a reason why they call it the bleeding edge. It's up to you to evaluate the technology to see if it fits within your infrastructure. Calling docker out and saying it's broken when its a free product seems to be miss guided.
    - tedunangst 10y ago
      Free and broken are not mutually exclusive.
      - hackits 10y ago
        It nothing about that. The main point is the whole community bitching a moaning about software that in all intent and purposes is given away free without any contribution by the users. The source code of Docker is open source so why doesn't the guy go in there and contribute his time and or his own money to support the project that to him is so vital? No doubt the guys working on Docker are super smart guys, though they too have bill's to pay and lives to live. So if you want to see Docker improve how about contributing back and or supporting their efforts with extra funding. Maybe they will have the funds to bring on more full time developers to hunt those obscure bugs or add those features that everyone wants? Discussing and putting down the developer with article titles such as 'Sad state of docker' to me shows self entitled the community around these projects are.
        tedunangst 10y ago
        Extra funding? Because $100 million isn't enough?
boomstik 10y ago
I haven't played enough with Docker Swarm to run into any of these issues, but Rancher (www.rancher.com) does all of this, and more, very well. I am not affiliated with them - just a happy user and contributor of github issues.
- sschueller 10y ago
  Rancher is very cool but I wish that rancher OS was easier to lockdown. If you run it outside of AWS or Exoscale you don't get a firewall so you have to do that on the hosts.
- jondubois 10y ago
  I agree, Rancher just ties everything together really nicely and it works awesome with Kubernetes. I'm building a hosted Rancher cluster manager for my OSS project http://baasil.io/ http://baasil.io/.
- drdaeman 10y ago
  Went with them last week, as it seemed to be one of the most sane options out there (I've tried a lot). Still, has its share of issues. Few ones I've encountered so far: - Their built-in load balancing (HAProxy-based) is nearly impossible to debug. Literally no logging there. - No locality awareness. DNS queries always return all addresses they know about (including those that don't even work - https://github.com/rancher/rancher/issues/5792 https://github.com/rancher/rancher/issues/5792) and I haven't yet found any good way to prioritize containers co-running on the local host to the more distant ones (https://github.com/rancher/rancher/issues/5798 https://github.com/rancher/rancher/issues/5798 - if someone been to this situation and has some ideas, would appreciate any suggestions!). - Storage management was advertised, but can't find anything besides NFS (which is SPOF) and Amazon EFS (which I don't use). There was GlusterFS support, but it seems it was too broken so they had removed it or something like that. If one wants persistent storage, they'd better pin containers to hosts.
  - InTheArena 10y ago
    GLuster is a disaster.
vacri 10y ago
> please take it slow and make Docker great again! Perhaps they should create some sort of firewall? I mean, the network packets coming through... they bring malware... they're spam... and some, I assume, are good traffic.
- MichaelGG 10y ago
  The good traffic should be already permitted in the firewall. If you're still having issues the firewall rules are probably not being enforced. I've seen this happen like when an any any allow rule is left in from debugging ("just to get things working").
  - wastedhours 10y ago
    But when your entire service is built on the notion of accepting all types of traffic, it's insulting to block it based on nothing more than opinion (I would point out for the avoidance of doubt, OP was making a Trump reference...)
- m_mueller 10y ago
  I agree very much. Trying to figure out how Docker hacks your host's iptables and how to deal with it in a production network is a pain.
  - andrewguenther 10y ago
    We never let Docker set its own iptables rules. It is a pain at first, but it forces you to understand how the rules work, what is going on, and has the added bonus of keeping those rules consistent across Docker releases and in your own version control.
    - m_mueller 10y ago
      so as I understand you add and remove the port mapping and other FW entries with your own scripts whenever you spin up / stop a container?
gnur 10y ago
The new swarm mode is great when it works, but it is monster to debug. It takes care of so many things that it is nearly impossible to pin point what component isn't working. The issues I have encountered with swarm mode: * Some containers could not use hostnames to connect to other containers. * Sometimes, in a 3 node swarm, containers on A could be reached from B, but not from C. * After every reboot these issues could be fixed or start occurring. * It automatically adds firewall rules for every service you port map to be exposed to the internet, without warning In the end I switched to nomad, it isn't perfect either but at least it is consistent.
- dexterbt1 10y ago
  Do you have the links to the issue tracker for each of these issues? For me/us to subscribe/follow.
rusher81572 10y ago
I am looking more and more about using the Apcera Platform. They have everything I need in a container management platform for free: https://www.apcera.com/community-edition https://www.apcera.com/community-edition
- jacques_chester 10y ago
  Do you work for Apcera? Your submission page is heavy on the linux-toys.com domain, implying that it's yours. That same site identifies the author as working at Apcera. There is also a submission in your account that points directly to an Apcera blogpost, with the same author name. Disclosure: I work for Pivotal, we donate the majority of engineering to Cloud Foundry. Apcera has beef with us.
  - rusher81572 10y ago
    Yes, I work for Apcera and linux-toys.com is my personal blog where I talk about open source technologies (mainly Docker lately) and software that I develop. Like the article states, I am a Docker fanboy and want them to be great again. The blog website is actually running on a four node Raspberry pi cluster that I referenced in the blog as a link. The reason I joined Apcera is because they are also developing cool container technology. Until recently, Apcera was only available to enterprises and didn’t have a solution for “homeduction” users like me to host and run applications like my blog and a few other services. With the community edition of the software, I can gather a few spare x86 boxes and make the switch. I will still be running the same Docker images but using Apcera’s orchestration software instead of swarm.
    - jacques_chester 10y ago
      Thanks. I'm glad to hear Apcera is branching out and that you're excited to work there.
colemickens 10y ago
Swarm Mode is "great". Assuming you've never heard of or used Kubernetes. In which case, Docker Swarm is too little, and a year+ too late. As for marketing, it does seem a bit funny that a product would announce "deep integration with underlying infrastructure" for a cloud provider when they haven't written a single line of (public) code to actually support that cloud provider. The fun thing is this arguably critical blog post praises features in "swarm mode" that have long been present in Kubernetes/Mesos/Nomad: [labels/constraints]. There could be a lot written about the fact that Docker ships "Swarm Mode" as stable in 1.12 despite virtually everyone's actual first-hand experiences. I would argue that if "Swarm Mode" were not shipped inside of Docker 1.12 and didn't benefit from riding along with the normal `docker` package, few would be talking about it.
- rusher81572 10y ago
  Yeah, Docker has a lot of catching up to do. They bragged a lot lately about how Swarm is faster than K8s but you really can not compare the two when you look at features and stability of k8s.
  - sjellis 10y ago
    It's amazing (and not in a good way) that this stuff is being shipped in a point release.
gbrayut 10y ago
So you are saying that a new major software release that is less than a month old still has some bugs? I am shocked. Shocked! Well, not that shocked. If you can't take the bleeding part of bleeding edge, wait for things to mature before using them. Bitching and whining that they didn't create a perfect product out the gate only belittles the hard work it took to get a new product out the door. This is how software releases work in the real world!
- rusher81572 10y ago
  Some bugs are ok and expected like you said but major functionality issues that make the new feature useless should be addressed prior to a release and huge marketing efforts.
- merb 10y ago
  https://blog.docker.com/2016/08/software-security-docker-containers/ https://blog.docker.com/2016/08/software-security-docker-con...
- colemickens 10y ago
  It's one thing to knowingly release beta software as "beta". It's another thing to bundle beta software into an existing package and label it as stable in order to stay relevant in the orchestration game.
andrewguenther 10y ago
Disclaimer: I work at AWS, but on a product which does not compete with Docker or its orchestration tools in any way shape or form. My opinions are my own. I wouldn't even limit this to just the swarm feature. We've been running Docker in production for a year, using it in dev environments a year before that, and we've had major problems nearly every release. We had to upgrade directly from Docker 1.7 to 1.11 because every release in between was too unstable or had glaring performance regressions. We ended up running a custom build and backporting features which were worth the risk. Speaking of 1.12, my heart sank when I saw the announcement. Native swarm adds a huge level of complexity to an already unstable piece of software. Dockercon this year was just a spectacle to shove these new tools down everyone's throats and really made it feel like they saw the container parts of Docker as "complete." One of the keynote slides literally read "No one cares about containers." I get the feeling we'll be running 1.11 for quite some time...
- tinco 10y ago
  To provide some weight the other way, we've been using Docker in production for about 3 years now, and have not had any big issues. Obviously, you guys probably have a bit more extreme use cases at AWS. Things that bug us are generally missing features, but those gradually get added in over the course of the years, though some get less love than others. For example for some reason it's still not possible to ADD something and change its permissions/ownership in one layer, resulting basically in a doubling of the size of such layers. I wouldn't go as far as saying it's in any kind of a 'sad' state though. It's a neat wrapper over some cool Linux kernel features, and it's been that way since before 1.0. I'm curious how you even get performance issues from Docker, what feature did cause performance issues for you?
  - andrewguenther 10y ago
    Always fun to hear experiences from other production veterans. Glad to hear things are working well for you guys. Our use case involves rapid creation and destruction of containers. Granted, this use case was pretty unheard of when we first adopted Docker, but it is becoming much more common. Before Docker moved over to containerd, the docker daemon was riddled with locks which resulted in frequent dead-locking scenarios under load as well as poor performance. Thankfully, containerd now uses an event loop and is lock-free. This was a huge motivating factor for us to move forward to Docker 1.11. To me, the sad state has more to do with Docker the company pushing new features out as quickly as possible and leaving stabilization to contributors. There are some days where it really feels like Docker is open-source so that Docker Inc can get free QA. To most users things may not feel in a sad state, but it can really suck for contributors.
wrong_variable 10y ago
# Market-Driven-Development Docker promises too much and delivers too little. Story of every software project in the last 50 years.
initdaemon 10y ago
IMHO, nobody should care / worry about the underlying orchestration. The industry is moving toward the public service model, and all these management problem is solved by the cloud platform, not app developers. Check out hyper.sh, that's what container service would be.
- deleted 10y ago
  [deleted]
- mixedCase 10y ago
  Or I could manage my own network, which isn't that hard and for a lot of (if not most) cases, less expensive.
- majewsky 10y ago
  Every time I hear "The industry is moving toward X", I can't help but imagine a car racing off a cliff at full speed.
deleted 10y ago
[deleted]
jondubois 10y ago
Yes maybe the Docker team should just have joined forces with Kubernetes (K8s) instead of going out on their own and building Swarm from scratch. K8s is far ahead of Swarm - K8s has practically built its own language using YAML files - Swarm is still at a stage that all the configs for a service have to fit into a single command (and the options are much more restrictive than K8s). To be fair, I do like some things about Swarm better than K8s (based on the docs), but in practice, Swarm is behind and they should tell you that up front. When I was just starting out, I literally had to install all of them; Swarm, Mesos and K8s to be able to make an informed decision because, in the case of Docker, the docs are like 6 months ahead of reality. I didn't realize that the Docker 'service' command didn't even exist until v1.12 and I couldn't install v1.12 on my machine (last time I tried, installation was failing - Obviously not yet stable). I think Swarm has potential but they need to accept that they're just not going to be the first to market.
- drdaeman 10y ago
  Swarm works (more or less) if one has just 2-3 hosts, use Docker for packaging, and want a semi-unified view of those machines. Kubernetes seems to be quite highly opinionated toward "clouds" and "microservices". I just wasn't able to wrap my head over its concepts' applicability to my "I just have one server that uses Docker for packaging, and now want to throw in another, for resiliency" case.
  - lobster_johnson 10y ago
    Kubernetes isn't particularly opinionated at all. It runs containers, and doesn't care what those containers are or how they behave. Microservices and clouds not required. Its core data model, simplified, that of pods. A pod specifies one or more named containers that should run together as a unit. A pod's config can specify many things, such as dependencies (volumes, secrets, configs), resource limits and ports (including how to perform health checks). You can deploy single-container pods, and this is the norm, but it's entirely feasible to run a whole bunch of containers that conceptually belong together. To expose a pod's ports to the world or to other pods, you define services. These simplify specify what ports should go which pods, and Kubernetes will assign a persistent, internal IP address to it. Kubernetes will (typically) configure iptables so that the service is round-robin-balanced at the network level across all containers that it serves; the idea is that the pod should be reachable from any other pod in the cluster. Together with KubeDNS, which resolves service names, you can do things like call http://mylittlepod/ http://mylittlepod/ to reach a pod. To achieve resilience, Kubernetes lets you define replica sets, which are rules that says "this pod should run with N replicas". K8s will use the scheduler to enforce this rule, ensuring that a pod is restarted if it dies and always has N replicas running, and it can automatically ensure that pods are spread evenly out across the cluster. Replica sets can be scaled up and down, automatically or manually. There are other objects, such as deployments (handle rolling upgrades between one version of a pod and another), ingresses (configures load-balancers to expose HTTP paths/hosts on public IPs), secrets (encrypted data that pods can mount as files or envvars), persistent volumes (e.g. AWS EBS volumes that be mounted into a pod), and so on, but you can get by with just pods and services, at least to start. Kubernetes is a bit pointless with a single server, but adds convenience even if you have just two or three.
kozikow 10y ago
I wish that docker would adapt more of a Unix philosophy and focus on doing one thing well. Why does everyone have to compete with everyone rather than create set of tools that work well together? I see docker-machine and docker-swarm as distractions. Reasons why doing all those other things, instead of focusing on containerisation and packaging may be harm-full for docker itself: - Bundling-in the orchestration with docker make k8s or Mesos more inclined to fork docker and pull out unnecessary cruft. - Churning out half-ready features causes Docker to be known as unreliable and leads to posts with titles like this. Such reputation tends to stay long after bugs are fixed. SV-esque launch and iterate works for web apps, but IMO not for back-end software.
- solatic 10y ago
  Because shipping features is king. When a single vendor ships many products, those products are more likely to work well together than with products from other vendors because they were built under the same roof. Better integration = saving time while reducing risk = more time to build and ship features. The downside, of course, is vendor lock-in. But that's only a problem if a) the vendor stops updating their products, which is unlikely if those products are popular, like Docker, or b) the vendor raises prices beyond what can be justified to remain that vendor's customer. But that's a problem for whoever takes over your project next year, not for you.
  - sjellis 10y ago
    I would say that the worst problem is the vendor changing the product in some incompatible way as part of a monetization strategy, so that Product X still exists, but the install-base splits. Docker Inc. themselves make a big play of the facts that containers have to be standardized to be portable, and the portability is the key value. We could have done a lot of this stuff years ago if the virtualization vendors had a totally portable format for transferring VMs between different systems. Swarm etc. is part of the monetization strategy - other vendors in the ecosystem have already backed Kubernetes, or MesoSphere, or whatever, and do not want or need this stuff tied to the Docker run-time itself. Fortunately, Docker Inc. can add these without breaking compatibility of images or damaging the core features enough that a fork becomes necessary, but it does create market confusion.
systemz 10y ago
Docker team, please focus on one thing - packing apps to containers. Leave everything else to other projects, don't try to do everything wrong, just do one thing good. Thank you.
- sjellis 10y ago
  The problem is that they literally can't do that - Docker Inc. took a huge pile of VC money, and containerization is a commodity feature. To succeed they have to build higher-level products and services, which put them in direct competition with the other vendors in the ecosystem.
  - adamc 10y ago
    Sounds like a high-risk ploy. Then again, VC money.
    - sjellis 10y ago
      What annoys me about it is that it requires everyone else to lose for Docker Inc. to win. It's rather like if Linus invented Git, then formed a VC-backed company on the back of it, and then tried to figure out how to extract enough money from the ecosystem so that Git, Inc. could be flipped or IPO'ed for a really big pile of cash.
      - corford 10y ago
        Agree strongly. I think they will lose this fight which is why I've stayed out of the docker race and am currently pinning my hopes on k8s + rkt.
  - dominotw 10y ago
    >To succeed they have to build higher-level products and services I wish they didn't have to cram it all into single product docker though.
    - 10y ago
nickbauman 10y ago
If you look at the design of Kubernetes you'll find a very strong opinion on how networking is done. Kubernetes does not allow the use of Network Address Translation (NAT) for container-to-container or for container-to-node traffic. The internal container IP address must match the IP address that is used to communicate with it. Swarm plays faster and looser than this. I think Google's age and experience shows that Docker went the wrong way.
- InTheArena 10y ago
  If by google's age and experience, you mean the requirements of the Google Cloud, then i agree. Docker is doing exactly what they should be, but in a manner that is destructive. Getting a built in consistent p2p routing mesh under every container is brilliant, and fixes one of the biggest problems with k8s and swarm (it's not really possible for these technologies to interoperate because of incompatibilities with the network model). The big problem is the stability hit. 1.12 had no business loosing the rc label.
  - dward 10y ago
    > If by google's age and experience, you mean the requirements of the Google Cloud, then i agree. Why do you think this is a requirement of Google Cloud?
    - InTheArena 10y ago
      Kubernetes and Google Cloud were both informed by the design and implemention of Borg. Kubernetes is basically for all intents and purposes Google Container Engine. That's fine, but it;s highly tuned for how google sees the world.
      - nickbauman 10y ago
        Huh? Kubernetes can run just fine outside of Google's Cloud. It'll work on any TCP/IP IaaS offering out there. If you mean it demands a clear end-to-end connection model for the important moving parts, then, yeah; because of hard-won experience of what works. They found out that you want to spend your time on bugs in your app, not bugs in your networking infra.
graffitici 10y ago
It seems that using Kubernetes is definitely more mature and usable than Swarm. But how would you rate the other Docker projects, like docker-machine and docker-compose. Does Kubernetes also subsume those projects? These seem to be way more mature than Swarm.
- fideloper 10y ago
  I don't know if Kubernetes is more usable - it's a MONSTER to host yourself. Other docker projects suffer from various levels of similar issues. Docker-machine is nice, but has a ton of rough edges, especially when spinning up host on AWS. It feels like the programmer(s) of machine never used AWS beyond the simplest use case.
  - ownagefool 10y ago
    Kubernetes isn't particularly hard to host. Bootstrap etcd start kubelet place api, proxy, scheduler and controller definitions manifests folder. If you're running infrastructure that deals with problems such as maintaining the health of applications then it's going to generally be a whole lot more complex than that.
  - iso-8859-1 10y ago
    Did you see https://github.com/kubernetes/minikube https://github.com/kubernetes/minikube ?
- deleted 10y ago
  [deleted]
- cyphar 10y ago
  Kubernetes is definitely the technically correct solution. The only really hard part is getting started, but if you have a cloud service provider that runs a Kubernetes cluster for you then you don't need to worry about that. :P
bullen 10y ago
If you need a distributed PaaS alternative for RPi and others, you can look at https://github.com/tinspin/rupy https://github.com/tinspin/rupy. It's simple and stable!
erikb 10y ago
Quality is something long term, but because the world becomes faster and faster people care less about it. Hype can generate a lot of money in the short term. And that is what is valued the most in the current economy. Being a quality guy myself I also feel that this is painful, but I think it's hard to blame anybody for that. I don't know anybody who goes for the short term success because they want to live in that kind of world. It's just about the only thing that gets rewarded.
agentgt 10y ago
I wish Docker had some more open source competitors particularly some non-profit competitors. Yes I know Docker is open source but I have this terribly gut feeling about becoming too reliant on their technology. I feel like I'm going to screwed some day. I guess I just know some day Docker's investors are going to want their money back. Yes sure there are other quasi opensource products that are super critical that I use but they all have alternatives (for example Java has plenty of alternatives). Hopefully I don't get downvoted to oblivion for this comment. I am sure my trust issues are illogical and I would really like to remove the inhibition to use docker but articles like this do not help.
- sjellis 10y ago
  I don't think that you are being completely illogical. One thing about the Docker stack, though, is that it is supported by every major vendor, and you may well be getting your server installations through a vendor (Red Hat, AWS etc.), so in that case you are insulated from problems. As other commenters have discussed, the vendors test and patch their Docker distributions themselves already, as well as contributing to development. We also already have a functional replacement with rkt. It can use Docker images, and Kubernetes can use rkt as a run-time in place of Docker, so Docker is not irreplaceable. I think that the most valuable bits of Docker today are the developer tooling - easy Windows and Mac installers, Docker Compose, the online documentation, and the Hub for grabbing ready-made images. None of which, AFAIK, makes much money for Docker Inc.
dominotw 10y ago
is swarm mode same as docker swarm [1]? Curious, Why they choose the same name if that's not the case. Why would one use docker swarm[1] vs just docker swarm mode. 1.https://github.com/docker/swarm https://github.com/docker/swarm
- johnhenry 10y ago
  Originally, there was the stand alone version of docker swarm, invoked using "docker-swarm". This was more difficult to set up than version 1.12's new swarm mode, invoked using "docker swarm", which does almost the same thing, but more succinctly. Presumably, they chose the same name because they serve the same purpose and, if the latter is set to replace the former, why come up with a new name for basically the same concept?
  - dominotw 10y ago
    >if the latter is set to replace the former Do you know if there was any guidance by docker team on this? Seems like there is active development going on for both docker-swarm and swarm-mode.
    - justincormack 10y ago
      Development is mainly in swarm mode, but yes the older swarm is still supported, it is just an application that uses docker so will always work.
      - dominotw 10y ago
        I guess my main confusion is when would someone choose to use docker-swarm over swarm-mode.
- eppsilon 10y ago
  It's not the same. Strangely the docker-swarm tutorial doesn't mention swarm mode. I wasn't able to follow it successfully with Docker 1.12. That was when I realized docker-swarm != swarm mode, so I tried the swarm mode tutorial instead, and it worked as expected. Maybe docker-swarm is not supported anymore with 1.12? It was all pretty confusing.
InTheArena 10y ago
There are a couple of things here. 1) Right now everyone is afraid that Docker will emulate VMware, and crowd them out of the container space, much like VMware killed most of their competitors. 2) To this end, I have heard that Google and Redhat have massive marketing budgets, and that the marching orders have been over and over - don't say docker, say k8s. 3) The real battle is where the money is - large scale distributed systems. Companies want to freeze docker out, because Docker controls the lowest point of access - the container runtime itself. 4) hence google is trying to push "docker compatible" ideas that are just the OCI standard - nothing to do with Docker itself. AWS doesn't want to support Swarm, because it gives people portability off of their cloud. Google doesn't want to support swarm, because K8s is a trojan for Google Cloud. No one else wants to support swarm because it competes with their products. That said, what's happening right now, if we are not careful, will fragment the container ecosystem, and it make it impossible for single containers to target multiple runtimes. Docker is the only one who can deliver a universal set of functionality that is leveraged by all. From a technology point of view, Docker is going in the right direction. We got burned with Redhat in Openshift 1 & 2 land, and that's left us with a point of view that the only thing we can depend on is a container runtime itself, and 12fa applications. K8s does not really work that way. It's huge and it's heavy, and it expects every app to be written it's way. The technical direction here for Docker is good. But the implementation and early release is ridiculous. I was impressed by the first RC release, and then terrified that they released a RC as production.
- wstrange 10y ago
  > Docker is the only one who can deliver a universal set of functionality that is leveraged by all. Why do you say that? I have quite a bit more faith in the design chops of the folks behind Kubernetes (Google, Redhat, CoreOS, and many others) than Docker Inc. Swarm really only touches the surface of the requirements for large scale distributed container orchestration. Kubernetes is complex because the problem it attempts to solve is complex. I'd also add that Kubernetes is dead simple to use. The difficulty is in setting it up - but even that is getting much better.
  - InTheArena 10y ago
finid 10y ago
Trying to run Docker on your own to do anything meaningful can be a very painful exercise. I gave up after trying to spin up a cluster on DigitalOcean using v1.12. Like the author, I couldn't get my containers to see each other, something that worked before v1.12.
SparkyMcUnicorn 10y ago
rekt.. I mean, rkt
moondev 10y ago
This should be renamed to "the sad state of swarm mode"
nullcipher 10y ago
Negative comments here seem to be a gross overreaction. This is primarily a code release. Code is always going to keep moving. I have used swarm and it's nowhere close to unstable as people say it is. Is it production ready? Probably not. But which software is production ready in every big release? Unnecessary fear mongering by people who have an outside agenda.
- nullcipher 10y ago
  OK downvoters, I get it. Everyone hates docker but uses it anyway :-)
- cpitman 10y ago
  Which release is going to be production ready, and which features are production ready in each release? If Docker Inc documented which features are production ready, I'd have more sympathy for that point of view. What we've had to do at Red Hat is always stay a couple releases behind (we're shipping 1.10+patches right now) and backport all the fixes from upstream releases to make it stable (ie production ready). Docker keeps shipping new versions, with fixes for old issues but then a whole new set of issues added in.
daxorid 10y ago
We've experimented with docker in a few places, and the deployment workflow is just painful: 1. sudo docker build -t quay.io/foo/bar 2. sudo docker push quay.io/foo/bar 3. <login to production> 4. sudo docker pull quay.io/foo/bar 5. sudo docker kill foobar 6. sudo docker rm foobar 7. sudo docker run -p 80:80 -p 443:443 -e FOO=bar --name foobar --net=host -d quay.io/foo/bar I can never understand how people talk about docker making deployments somehow easier.
- kinghajj 10y ago
  ElasticBeanstalk streamlines things greatly, and when it works everything is pretty nice. When it fails, though... for instance, just a few days ago, one of the containers failed with an OOM error. For some reason--still unclear--the ECS and/or Docker daemons weren't able to start new containers to replace them, leaving the instance broken for hours. Auto-scaling groups will mitigate this, but it's still unnerving. Still, I'm liking many aspects our tools. Using Docker with Rocker (https://github.com/grammarly/rocker/ https://github.com/grammarly/rocker/) has greatly sped-up CI builds by caching results when the source hasn't changed (especially important in multi-language shops; the Python guys don't want to wait on the Java code to build every time.) Just upload a tagged image to ECR, generate an "application version" referencing those images, and deploy via the Slack bot ("@bula deploy develop develop-XXX-e83fc3bd").
- justinsaccount 10y ago
  How is that painful? I run a service using docker and have your steps 4 through 7 as part of a systemd unit file. Updating the application requires a single systemd restart command.
  - sandGorgon 10y ago
    could you talk about your deployment scripts ? i am trying to deploy a single flask app which uses redis. I'm not sure how to set up logging, etc. and whether redis and flask will go in the same VM.
    - 10y ago
dang 10y ago
I changed the baity, over-general title "The Sad State of Docker" to what the first paragraph of the article says it's actually about. Generally we moderate HN stories/threads less when a YC startup or YC itself is at issue. But we do still moderate them some, because the standards of the site still apply. (We haven't done anything here besides this title edit, though, in case anybody is wondering.)
kennysipe 10y ago
Disclaimer: I work for Mesosphere This is the reason the latest Mesos and DCOS has the universal containerizer. Docker is great for development but currently doesn't make sense for production. The latest DCOS uses the docker images without the docker process and provides the high scale production quality needed for a large datacenter.
Nano2rad 10y ago
One reason for release of Open source software is testing.
- majewsky 10y ago
  Yes, but that's why "alpha", "beta" and "RC" labels exist.
sandGorgon 10y ago
Question: we are looking to deploy a flask + redis based application to a single server on AWS using docker. No load balancing, no multiple server what is the current best practice to do this (with logging, etc.) ? should I even be considering something like marathon/k8s, etc ? currently I have a fat docker VM with supervisord and all services running in a single VM with highly fragile logging. I dont think this can last. getting started seems very intimidating in the Docker world.
- kozikow 10y ago
  I recently moved my django running on docker to google container engine. Basically by following this tutorial you can be up in 15 minutes: https://cloud.google.com/python/django/container-engine https://cloud.google.com/python/django/container-engine . k8s picks up your stdout logs and sends them to stackdriver and you don't need to do anything to set it up. I was running nginx+gunicorn inside my docker image, but nginx part have been taken care of by k8s.
- josegonzalez 10y ago
  Dokku should be able to handle all of your concerns. We do load-balancing on the server via nginx, use Docker's plumbing (the docker command/api) to build nice porcelain (our own cli tool) for stuff like restart policies etc. It's targeted specifically at single-server solutions, and migrating once you are large enough to another platform is easy as we tend to not build in platform lock-in. Feel free to jump on our slack or irc: http://dokku.viewdocs.io/dokku/getting-started/where-to-get-help/ http://dokku.viewdocs.io/dokku/getting-started/where-to-get-... Disclaimer: I am a maintainer of Dokku
radarsat1 10y ago
I personally just getting into Docker and am looking at the swarm feature for deploying containerized compute nodes to our cluster. People here seem to be complaining specifically about Swarm mode, is there anything I should watch out for? What does Kubernetes provide that Docker swarm doesn't? I've tested Docker Swarm a bit and it seems to work as advertised. Is it just about node selection not being sophisticated enough? In that case I don't need to worry since all my nodes are the same, but if there's any words of warning I'm all ears. Thanks.
api 10y ago
Am I the only person around here who is skeptical of all this added complexity? Some of it is clearly very useful, but some of it strikes me as a tower of babel built upon workarounds to problems that have simpler solutions. What I see here are several different vendors vying to make sure they remain relevant. While I understand this and even the need for it, I also understand that it can drive poor engineering decisions in the long run.
maxamillion 10y ago
I've been using docker since version 0.6 and followed almost every version upgrade since and it's an absolute mess. This blog post is spot on. My old team had a production environment running in docker containers for about a year (this was pre-swarm, pre-kubernetes) and then transitioned to just using ansible for application deployment in a more "traditional" manner because we spent more time trying to fix broken things with docker than it was worth.

390 points by rusher81572 10y ago

I also assumed the new easy swarm would be easier, but the Multi-host network only works with the newly introduced service command only, it does not work with regular docker run command, which is disappointing because you still neeed a third party key-value store for it, but not for docker service. It literally took me hours to realize that was missing. I think 1.12 was just rushed to show it in DockerCon, normally major releases were mostly bug-free and worked as intended.

rusher81572 10y ago
I feel the pain for I was in the same boat.

epberry 10y ago

I share the pain of this post - except my run-in with 1.12 occurred with Docker for Windows. The shared volumes and host networking were totally nondeterministic. I don't think I had really ever experienced software which seemed to fail so randomly. On the other hand I think Docker can probably be forgiven for my particular frustration. For one, the software was in beta. And two, working with Windows and all the different flavors must be a nightmare.

djs55 10y ago
(I work at Docker) If you haven't already and can spare the time, could you file a pair of issues: perhaps one for the shared volume problem and another for the networking on https://github.com/docker/for-win https://github.com/docker/for-win ? We've been fixing bugs in both areas and the fixes should arrive in the beta channel over the next few updates. Thanks for your patience!

hackits 10y ago

Don't know what all the fuss is about when essentially you're getting Docker for free?

rusher81572 10y ago
It is not about cost. It is about marketing and generating all the hype over something that clearly does not function properly. They act like they are laying golden eggs when in fact they are broken eggs.
- hackits 10y ago
  There is a reason why they call it the bleeding edge. It's up to you to evaluate the technology to see if it fits within your infrastructure. Calling docker out and saying it's broken when its a free product seems to be miss guided.
  - tedunangst 10y ago
    Free and broken are not mutually exclusive.
    - hackits 10y ago
      It nothing about that. The main point is the whole community bitching a moaning about software that in all intent and purposes is given away free without any contribution by the users. The source code of Docker is open source so why doesn't the guy go in there and contribute his time and or his own money to support the project that to him is so vital? No doubt the guys working on Docker are super smart guys, though they too have bill's to pay and lives to live. So if you want to see Docker improve how about contributing back and or supporting their efforts with extra funding. Maybe they will have the funds to bring on more full time developers to hunt those obscure bugs or add those features that everyone wants? Discussing and putting down the developer with article titles such as 'Sad state of docker' to me shows self entitled the community around these projects are.
      - tedunangst 10y ago
        Extra funding? Because $100 million isn't enough?

boomstik 10y ago

I haven't played enough with Docker Swarm to run into any of these issues, but Rancher (www.rancher.com) does all of this, and more, very well. I am not affiliated with them - just a happy user and contributor of github issues.

sschueller 10y ago
Rancher is very cool but I wish that rancher OS was easier to lockdown. If you run it outside of AWS or Exoscale you don't get a firewall so you have to do that on the hosts.
jondubois 10y ago
I agree, Rancher just ties everything together really nicely and it works awesome with Kubernetes. I'm building a hosted Rancher cluster manager for my OSS project http://baasil.io/ http://baasil.io/.
drdaeman 10y ago
Went with them last week, as it seemed to be one of the most sane options out there (I've tried a lot). Still, has its share of issues. Few ones I've encountered so far: - Their built-in load balancing (HAProxy-based) is nearly impossible to debug. Literally no logging there. - No locality awareness. DNS queries always return all addresses they know about (including those that don't even work - https://github.com/rancher/rancher/issues/5792 https://github.com/rancher/rancher/issues/5792) and I haven't yet found any good way to prioritize containers co-running on the local host to the more distant ones (https://github.com/rancher/rancher/issues/5798 https://github.com/rancher/rancher/issues/5798 - if someone been to this situation and has some ideas, would appreciate any suggestions!). - Storage management was advertised, but can't find anything besides NFS (which is SPOF) and Amazon EFS (which I don't use). There was GlusterFS support, but it seems it was too broken so they had removed it or something like that. If one wants persistent storage, they'd better pin containers to hosts.
- InTheArena 10y ago
  GLuster is a disaster.

vacri 10y ago

> please take it slow and make Docker great again! Perhaps they should create some sort of firewall? I mean, the network packets coming through... they bring malware... they're spam... and some, I assume, are good traffic.

MichaelGG 10y ago
The good traffic should be already permitted in the firewall. If you're still having issues the firewall rules are probably not being enforced. I've seen this happen like when an any any allow rule is left in from debugging ("just to get things working").
- wastedhours 10y ago
  But when your entire service is built on the notion of accepting all types of traffic, it's insulting to block it based on nothing more than opinion (I would point out for the avoidance of doubt, OP was making a Trump reference...)
m_mueller 10y ago
I agree very much. Trying to figure out how Docker hacks your host's iptables and how to deal with it in a production network is a pain.
- andrewguenther 10y ago
  We never let Docker set its own iptables rules. It is a pain at first, but it forces you to understand how the rules work, what is going on, and has the added bonus of keeping those rules consistent across Docker releases and in your own version control.
  - m_mueller 10y ago
    so as I understand you add and remove the port mapping and other FW entries with your own scripts whenever you spin up / stop a container?

gnur 10y ago

The new swarm mode is great when it works, but it is monster to debug. It takes care of so many things that it is nearly impossible to pin point what component isn't working. The issues I have encountered with swarm mode: * Some containers could not use hostnames to connect to other containers. * Sometimes, in a 3 node swarm, containers on A could be reached from B, but not from C. * After every reboot these issues could be fixed or start occurring. * It automatically adds firewall rules for every service you port map to be exposed to the internet, without warning In the end I switched to nomad, it isn't perfect either but at least it is consistent.

dexterbt1 10y ago
Do you have the links to the issue tracker for each of these issues? For me/us to subscribe/follow.

rusher81572 10y ago

I am looking more and more about using the Apcera Platform. They have everything I need in a container management platform for free: https://www.apcera.com/community-edition https://www.apcera.com/community-edition

jacques_chester 10y ago
Do you work for Apcera? Your submission page is heavy on the linux-toys.com domain, implying that it's yours. That same site identifies the author as working at Apcera. There is also a submission in your account that points directly to an Apcera blogpost, with the same author name. Disclosure: I work for Pivotal, we donate the majority of engineering to Cloud Foundry. Apcera has beef with us.
- rusher81572 10y ago
  Yes, I work for Apcera and linux-toys.com is my personal blog where I talk about open source technologies (mainly Docker lately) and software that I develop. Like the article states, I am a Docker fanboy and want them to be great again. The blog website is actually running on a four node Raspberry pi cluster that I referenced in the blog as a link. The reason I joined Apcera is because they are also developing cool container technology. Until recently, Apcera was only available to enterprises and didn’t have a solution for “homeduction” users like me to host and run applications like my blog and a few other services. With the community edition of the software, I can gather a few spare x86 boxes and make the switch. I will still be running the same Docker images but using Apcera’s orchestration software instead of swarm.
  - jacques_chester 10y ago
    Thanks. I'm glad to hear Apcera is branching out and that you're excited to work there.

colemickens 10y ago

Swarm Mode is "great". Assuming you've never heard of or used Kubernetes. In which case, Docker Swarm is too little, and a year+ too late. As for marketing, it does seem a bit funny that a product would announce "deep integration with underlying infrastructure" for a cloud provider when they haven't written a single line of (public) code to actually support that cloud provider. The fun thing is this arguably critical blog post praises features in "swarm mode" that have long been present in Kubernetes/Mesos/Nomad: [labels/constraints]. There could be a lot written about the fact that Docker ships "Swarm Mode" as stable in 1.12 despite virtually everyone's actual first-hand experiences. I would argue that if "Swarm Mode" were not shipped inside of Docker 1.12 and didn't benefit from riding along with the normal `docker` package, few would be talking about it.

rusher81572 10y ago
Yeah, Docker has a lot of catching up to do. They bragged a lot lately about how Swarm is faster than K8s but you really can not compare the two when you look at features and stability of k8s.
- sjellis 10y ago
  It's amazing (and not in a good way) that this stuff is being shipped in a point release.

gbrayut 10y ago

So you are saying that a new major software release that is less than a month old still has some bugs? I am shocked. Shocked! Well, not that shocked. If you can't take the bleeding part of bleeding edge, wait for things to mature before using them. Bitching and whining that they didn't create a perfect product out the gate only belittles the hard work it took to get a new product out the door. This is how software releases work in the real world!

rusher81572 10y ago
Some bugs are ok and expected like you said but major functionality issues that make the new feature useless should be addressed prior to a release and huge marketing efforts.
merb 10y ago
https://blog.docker.com/2016/08/software-security-docker-containers/ https://blog.docker.com/2016/08/software-security-docker-con...
colemickens 10y ago
It's one thing to knowingly release beta software as "beta". It's another thing to bundle beta software into an existing package and label it as stable in order to stay relevant in the orchestration game.

andrewguenther 10y ago

Disclaimer: I work at AWS, but on a product which does not compete with Docker or its orchestration tools in any way shape or form. My opinions are my own. I wouldn't even limit this to just the swarm feature. We've been running Docker in production for a year, using it in dev environments a year before that, and we've had major problems nearly every release. We had to upgrade directly from Docker 1.7 to 1.11 because every release in between was too unstable or had glaring performance regressions. We ended up running a custom build and backporting features which were worth the risk. Speaking of 1.12, my heart sank when I saw the announcement. Native swarm adds a huge level of complexity to an already unstable piece of software. Dockercon this year was just a spectacle to shove these new tools down everyone's throats and really made it feel like they saw the container parts of Docker as "complete." One of the keynote slides literally read "No one cares about containers." I get the feeling we'll be running 1.11 for quite some time...

tinco 10y ago
To provide some weight the other way, we've been using Docker in production for about 3 years now, and have not had any big issues. Obviously, you guys probably have a bit more extreme use cases at AWS. Things that bug us are generally missing features, but those gradually get added in over the course of the years, though some get less love than others. For example for some reason it's still not possible to ADD something and change its permissions/ownership in one layer, resulting basically in a doubling of the size of such layers. I wouldn't go as far as saying it's in any kind of a 'sad' state though. It's a neat wrapper over some cool Linux kernel features, and it's been that way since before 1.0. I'm curious how you even get performance issues from Docker, what feature did cause performance issues for you?
- andrewguenther 10y ago
  Always fun to hear experiences from other production veterans. Glad to hear things are working well for you guys. Our use case involves rapid creation and destruction of containers. Granted, this use case was pretty unheard of when we first adopted Docker, but it is becoming much more common. Before Docker moved over to containerd, the docker daemon was riddled with locks which resulted in frequent dead-locking scenarios under load as well as poor performance. Thankfully, containerd now uses an event loop and is lock-free. This was a huge motivating factor for us to move forward to Docker 1.11. To me, the sad state has more to do with Docker the company pushing new features out as quickly as possible and leaving stabilization to contributors. There are some days where it really feels like Docker is open-source so that Docker Inc can get free QA. To most users things may not feel in a sad state, but it can really suck for contributors.

wrong_variable 10y ago

# Market-Driven-Development Docker promises too much and delivers too little. Story of every software project in the last 50 years.

initdaemon 10y ago

IMHO, nobody should care / worry about the underlying orchestration. The industry is moving toward the public service model, and all these management problem is solved by the cloud platform, not app developers. Check out hyper.sh, that's what container service would be.

deleted 10y ago
[deleted]
mixedCase 10y ago
Or I could manage my own network, which isn't that hard and for a lot of (if not most) cases, less expensive.
majewsky 10y ago
Every time I hear "The industry is moving toward X", I can't help but imagine a car racing off a cliff at full speed.

deleted 10y ago

[deleted]

jondubois 10y ago

Yes maybe the Docker team should just have joined forces with Kubernetes (K8s) instead of going out on their own and building Swarm from scratch. K8s is far ahead of Swarm - K8s has practically built its own language using YAML files - Swarm is still at a stage that all the configs for a service have to fit into a single command (and the options are much more restrictive than K8s). To be fair, I do like some things about Swarm better than K8s (based on the docs), but in practice, Swarm is behind and they should tell you that up front. When I was just starting out, I literally had to install all of them; Swarm, Mesos and K8s to be able to make an informed decision because, in the case of Docker, the docs are like 6 months ahead of reality. I didn't realize that the Docker 'service' command didn't even exist until v1.12 and I couldn't install v1.12 on my machine (last time I tried, installation was failing - Obviously not yet stable). I think Swarm has potential but they need to accept that they're just not going to be the first to market.

drdaeman 10y ago
Swarm works (more or less) if one has just 2-3 hosts, use Docker for packaging, and want a semi-unified view of those machines. Kubernetes seems to be quite highly opinionated toward "clouds" and "microservices". I just wasn't able to wrap my head over its concepts' applicability to my "I just have one server that uses Docker for packaging, and now want to throw in another, for resiliency" case.
- lobster_johnson 10y ago
  Kubernetes isn't particularly opinionated at all. It runs containers, and doesn't care what those containers are or how they behave. Microservices and clouds not required. Its core data model, simplified, that of pods. A pod specifies one or more named containers that should run together as a unit. A pod's config can specify many things, such as dependencies (volumes, secrets, configs), resource limits and ports (including how to perform health checks). You can deploy single-container pods, and this is the norm, but it's entirely feasible to run a whole bunch of containers that conceptually belong together. To expose a pod's ports to the world or to other pods, you define services. These simplify specify what ports should go which pods, and Kubernetes will assign a persistent, internal IP address to it. Kubernetes will (typically) configure iptables so that the service is round-robin-balanced at the network level across all containers that it serves; the idea is that the pod should be reachable from any other pod in the cluster. Together with KubeDNS, which resolves service names, you can do things like call http://mylittlepod/ http://mylittlepod/ to reach a pod. To achieve resilience, Kubernetes lets you define replica sets, which are rules that says "this pod should run with N replicas". K8s will use the scheduler to enforce this rule, ensuring that a pod is restarted if it dies and always has N replicas running, and it can automatically ensure that pods are spread evenly out across the cluster. Replica sets can be scaled up and down, automatically or manually. There are other objects, such as deployments (handle rolling upgrades between one version of a pod and another), ingresses (configures load-balancers to expose HTTP paths/hosts on public IPs), secrets (encrypted data that pods can mount as files or envvars), persistent volumes (e.g. AWS EBS volumes that be mounted into a pod), and so on, but you can get by with just pods and services, at least to start. Kubernetes is a bit pointless with a single server, but adds convenience even if you have just two or three.

kozikow 10y ago

I wish that docker would adapt more of a Unix philosophy and focus on doing one thing well. Why does everyone have to compete with everyone rather than create set of tools that work well together? I see docker-machine and docker-swarm as distractions. Reasons why doing all those other things, instead of focusing on containerisation and packaging may be harm-full for docker itself: - Bundling-in the orchestration with docker make k8s or Mesos more inclined to fork docker and pull out unnecessary cruft. - Churning out half-ready features causes Docker to be known as unreliable and leads to posts with titles like this. Such reputation tends to stay long after bugs are fixed. SV-esque launch and iterate works for web apps, but IMO not for back-end software.

solatic 10y ago
Because shipping features is king. When a single vendor ships many products, those products are more likely to work well together than with products from other vendors because they were built under the same roof. Better integration = saving time while reducing risk = more time to build and ship features. The downside, of course, is vendor lock-in. But that's only a problem if a) the vendor stops updating their products, which is unlikely if those products are popular, like Docker, or b) the vendor raises prices beyond what can be justified to remain that vendor's customer. But that's a problem for whoever takes over your project next year, not for you.
- sjellis 10y ago
  I would say that the worst problem is the vendor changing the product in some incompatible way as part of a monetization strategy, so that Product X still exists, but the install-base splits. Docker Inc. themselves make a big play of the facts that containers have to be standardized to be portable, and the portability is the key value. We could have done a lot of this stuff years ago if the virtualization vendors had a totally portable format for transferring VMs between different systems. Swarm etc. is part of the monetization strategy - other vendors in the ecosystem have already backed Kubernetes, or MesoSphere, or whatever, and do not want or need this stuff tied to the Docker run-time itself. Fortunately, Docker Inc. can add these without breaking compatibility of images or damaging the core features enough that a fork becomes necessary, but it does create market confusion.

systemz 10y ago

Docker team, please focus on one thing - packing apps to containers. Leave everything else to other projects, don't try to do everything wrong, just do one thing good. Thank you.

sjellis 10y ago
The problem is that they literally can't do that - Docker Inc. took a huge pile of VC money, and containerization is a commodity feature. To succeed they have to build higher-level products and services, which put them in direct competition with the other vendors in the ecosystem.
- adamc 10y ago
  Sounds like a high-risk ploy. Then again, VC money.
  - sjellis 10y ago
    What annoys me about it is that it requires everyone else to lose for Docker Inc. to win. It's rather like if Linus invented Git, then formed a VC-backed company on the back of it, and then tried to figure out how to extract enough money from the ecosystem so that Git, Inc. could be flipped or IPO'ed for a really big pile of cash.
    - corford 10y ago
      Agree strongly. I think they will lose this fight which is why I've stayed out of the docker race and am currently pinning my hopes on k8s + rkt.
- dominotw 10y ago
  >To succeed they have to build higher-level products and services I wish they didn't have to cram it all into single product docker though.
  - 10y ago

nickbauman 10y ago

If you look at the design of Kubernetes you'll find a very strong opinion on how networking is done. Kubernetes does not allow the use of Network Address Translation (NAT) for container-to-container or for container-to-node traffic. The internal container IP address must match the IP address that is used to communicate with it. Swarm plays faster and looser than this. I think Google's age and experience shows that Docker went the wrong way.

InTheArena 10y ago
If by google's age and experience, you mean the requirements of the Google Cloud, then i agree. Docker is doing exactly what they should be, but in a manner that is destructive. Getting a built in consistent p2p routing mesh under every container is brilliant, and fixes one of the biggest problems with k8s and swarm (it's not really possible for these technologies to interoperate because of incompatibilities with the network model). The big problem is the stability hit. 1.12 had no business loosing the rc label.
- dward 10y ago
  > If by google's age and experience, you mean the requirements of the Google Cloud, then i agree. Why do you think this is a requirement of Google Cloud?
  - InTheArena 10y ago
    Kubernetes and Google Cloud were both informed by the design and implemention of Borg. Kubernetes is basically for all intents and purposes Google Container Engine. That's fine, but it;s highly tuned for how google sees the world.
    - nickbauman 10y ago
      Huh? Kubernetes can run just fine outside of Google's Cloud. It'll work on any TCP/IP IaaS offering out there. If you mean it demands a clear end-to-end connection model for the important moving parts, then, yeah; because of hard-won experience of what works. They found out that you want to spend your time on bugs in your app, not bugs in your networking infra.

graffitici 10y ago

It seems that using Kubernetes is definitely more mature and usable than Swarm. But how would you rate the other Docker projects, like docker-machine and docker-compose. Does Kubernetes also subsume those projects? These seem to be way more mature than Swarm.

fideloper 10y ago
I don't know if Kubernetes is more usable - it's a MONSTER to host yourself. Other docker projects suffer from various levels of similar issues. Docker-machine is nice, but has a ton of rough edges, especially when spinning up host on AWS. It feels like the programmer(s) of machine never used AWS beyond the simplest use case.
- ownagefool 10y ago
  Kubernetes isn't particularly hard to host. Bootstrap etcd start kubelet place api, proxy, scheduler and controller definitions manifests folder. If you're running infrastructure that deals with problems such as maintaining the health of applications then it's going to generally be a whole lot more complex than that.
- iso-8859-1 10y ago
  Did you see https://github.com/kubernetes/minikube https://github.com/kubernetes/minikube ?
deleted 10y ago
[deleted]
cyphar 10y ago
Kubernetes is definitely the technically correct solution. The only really hard part is getting started, but if you have a cloud service provider that runs a Kubernetes cluster for you then you don't need to worry about that. :P

bullen 10y ago

If you need a distributed PaaS alternative for RPi and others, you can look at https://github.com/tinspin/rupy https://github.com/tinspin/rupy. It's simple and stable!

erikb 10y ago

Quality is something long term, but because the world becomes faster and faster people care less about it. Hype can generate a lot of money in the short term. And that is what is valued the most in the current economy. Being a quality guy myself I also feel that this is painful, but I think it's hard to blame anybody for that. I don't know anybody who goes for the short term success because they want to live in that kind of world. It's just about the only thing that gets rewarded.

agentgt 10y ago

I wish Docker had some more open source competitors particularly some non-profit competitors. Yes I know Docker is open source but I have this terribly gut feeling about becoming too reliant on their technology. I feel like I'm going to screwed some day. I guess I just know some day Docker's investors are going to want their money back. Yes sure there are other quasi opensource products that are super critical that I use but they all have alternatives (for example Java has plenty of alternatives). Hopefully I don't get downvoted to oblivion for this comment. I am sure my trust issues are illogical and I would really like to remove the inhibition to use docker but articles like this do not help.

sjellis 10y ago
I don't think that you are being completely illogical. One thing about the Docker stack, though, is that it is supported by every major vendor, and you may well be getting your server installations through a vendor (Red Hat, AWS etc.), so in that case you are insulated from problems. As other commenters have discussed, the vendors test and patch their Docker distributions themselves already, as well as contributing to development. We also already have a functional replacement with rkt. It can use Docker images, and Kubernetes can use rkt as a run-time in place of Docker, so Docker is not irreplaceable. I think that the most valuable bits of Docker today are the developer tooling - easy Windows and Mac installers, Docker Compose, the online documentation, and the Hub for grabbing ready-made images. None of which, AFAIK, makes much money for Docker Inc.

dominotw 10y ago

is swarm mode same as docker swarm [1]? Curious, Why they choose the same name if that's not the case. Why would one use docker swarm[1] vs just docker swarm mode. 1.https://github.com/docker/swarm https://github.com/docker/swarm

johnhenry 10y ago
Originally, there was the stand alone version of docker swarm, invoked using "docker-swarm". This was more difficult to set up than version 1.12's new swarm mode, invoked using "docker swarm", which does almost the same thing, but more succinctly. Presumably, they chose the same name because they serve the same purpose and, if the latter is set to replace the former, why come up with a new name for basically the same concept?
- dominotw 10y ago
  >if the latter is set to replace the former Do you know if there was any guidance by docker team on this? Seems like there is active development going on for both docker-swarm and swarm-mode.
  - justincormack 10y ago
    Development is mainly in swarm mode, but yes the older swarm is still supported, it is just an application that uses docker so will always work.
    - dominotw 10y ago
      I guess my main confusion is when would someone choose to use docker-swarm over swarm-mode.
eppsilon 10y ago
It's not the same. Strangely the docker-swarm tutorial doesn't mention swarm mode. I wasn't able to follow it successfully with Docker 1.12. That was when I realized docker-swarm != swarm mode, so I tried the swarm mode tutorial instead, and it worked as expected. Maybe docker-swarm is not supported anymore with 1.12? It was all pretty confusing.

InTheArena 10y ago

There are a couple of things here. 1) Right now everyone is afraid that Docker will emulate VMware, and crowd them out of the container space, much like VMware killed most of their competitors. 2) To this end, I have heard that Google and Redhat have massive marketing budgets, and that the marching orders have been over and over - don't say docker, say k8s. 3) The real battle is where the money is - large scale distributed systems. Companies want to freeze docker out, because Docker controls the lowest point of access - the container runtime itself. 4) hence google is trying to push "docker compatible" ideas that are just the OCI standard - nothing to do with Docker itself. AWS doesn't want to support Swarm, because it gives people portability off of their cloud. Google doesn't want to support swarm, because K8s is a trojan for Google Cloud. No one else wants to support swarm because it competes with their products. That said, what's happening right now, if we are not careful, will fragment the container ecosystem, and it make it impossible for single containers to target multiple runtimes. Docker is the only one who can deliver a universal set of functionality that is leveraged by all. From a technology point of view, Docker is going in the right direction. We got burned with Redhat in Openshift 1 & 2 land, and that's left us with a point of view that the only thing we can depend on is a container runtime itself, and 12fa applications. K8s does not really work that way. It's huge and it's heavy, and it expects every app to be written it's way. The technical direction here for Docker is good. But the implementation and early release is ridiculous. I was impressed by the first RC release, and then terrified that they released a RC as production.

wstrange 10y ago
> Docker is the only one who can deliver a universal set of functionality that is leveraged by all. Why do you say that? I have quite a bit more faith in the design chops of the folks behind Kubernetes (Google, Redhat, CoreOS, and many others) than Docker Inc. Swarm really only touches the surface of the requirements for large scale distributed container orchestration. Kubernetes is complex because the problem it attempts to solve is complex. I'd also add that Kubernetes is dead simple to use. The difficulty is in setting it up - but even that is getting much better.
- InTheArena 10y ago

finid 10y ago

Trying to run Docker on your own to do anything meaningful can be a very painful exercise. I gave up after trying to spin up a cluster on DigitalOcean using v1.12. Like the author, I couldn't get my containers to see each other, something that worked before v1.12.

SparkyMcUnicorn 10y ago

rekt.. I mean, rkt

moondev 10y ago

This should be renamed to "the sad state of swarm mode"

nullcipher 10y ago

Negative comments here seem to be a gross overreaction. This is primarily a code release. Code is always going to keep moving. I have used swarm and it's nowhere close to unstable as people say it is. Is it production ready? Probably not. But which software is production ready in every big release? Unnecessary fear mongering by people who have an outside agenda.

nullcipher 10y ago
OK downvoters, I get it. Everyone hates docker but uses it anyway :-)
cpitman 10y ago
Which release is going to be production ready, and which features are production ready in each release? If Docker Inc documented which features are production ready, I'd have more sympathy for that point of view. What we've had to do at Red Hat is always stay a couple releases behind (we're shipping 1.10+patches right now) and backport all the fixes from upstream releases to make it stable (ie production ready). Docker keeps shipping new versions, with fixes for old issues but then a whole new set of issues added in.

daxorid 10y ago

We've experimented with docker in a few places, and the deployment workflow is just painful: 1. sudo docker build -t quay.io/foo/bar 2. sudo docker push quay.io/foo/bar 3. <login to production> 4. sudo docker pull quay.io/foo/bar 5. sudo docker kill foobar 6. sudo docker rm foobar 7. sudo docker run -p 80:80 -p 443:443 -e FOO=bar --name foobar --net=host -d quay.io/foo/bar I can never understand how people talk about docker making deployments somehow easier.

kinghajj 10y ago
ElasticBeanstalk streamlines things greatly, and when it works everything is pretty nice. When it fails, though... for instance, just a few days ago, one of the containers failed with an OOM error. For some reason--still unclear--the ECS and/or Docker daemons weren't able to start new containers to replace them, leaving the instance broken for hours. Auto-scaling groups will mitigate this, but it's still unnerving. Still, I'm liking many aspects our tools. Using Docker with Rocker (https://github.com/grammarly/rocker/ https://github.com/grammarly/rocker/) has greatly sped-up CI builds by caching results when the source hasn't changed (especially important in multi-language shops; the Python guys don't want to wait on the Java code to build every time.) Just upload a tagged image to ECR, generate an "application version" referencing those images, and deploy via the Slack bot ("@bula deploy develop develop-XXX-e83fc3bd").
justinsaccount 10y ago
How is that painful? I run a service using docker and have your steps 4 through 7 as part of a systemd unit file. Updating the application requires a single systemd restart command.
- sandGorgon 10y ago
  could you talk about your deployment scripts ? i am trying to deploy a single flask app which uses redis. I'm not sure how to set up logging, etc. and whether redis and flask will go in the same VM.
  - 10y ago

dang 10y ago

I changed the baity, over-general title "The Sad State of Docker" to what the first paragraph of the article says it's actually about. Generally we moderate HN stories/threads less when a YC startup or YC itself is at issue. But we do still moderate them some, because the standards of the site still apply. (We haven't done anything here besides this title edit, though, in case anybody is wondering.)

kennysipe 10y ago

Disclaimer: I work for Mesosphere This is the reason the latest Mesos and DCOS has the universal containerizer. Docker is great for development but currently doesn't make sense for production. The latest DCOS uses the docker images without the docker process and provides the high scale production quality needed for a large datacenter.

Nano2rad 10y ago

One reason for release of Open source software is testing.

majewsky 10y ago
Yes, but that's why "alpha", "beta" and "RC" labels exist.

sandGorgon 10y ago

Question: we are looking to deploy a flask + redis based application to a single server on AWS using docker. No load balancing, no multiple server what is the current best practice to do this (with logging, etc.) ? should I even be considering something like marathon/k8s, etc ? currently I have a fat docker VM with supervisord and all services running in a single VM with highly fragile logging. I dont think this can last. getting started seems very intimidating in the Docker world.

kozikow 10y ago
I recently moved my django running on docker to google container engine. Basically by following this tutorial you can be up in 15 minutes: https://cloud.google.com/python/django/container-engine https://cloud.google.com/python/django/container-engine . k8s picks up your stdout logs and sends them to stackdriver and you don't need to do anything to set it up. I was running nginx+gunicorn inside my docker image, but nginx part have been taken care of by k8s.
josegonzalez 10y ago
Dokku should be able to handle all of your concerns. We do load-balancing on the server via nginx, use Docker's plumbing (the docker command/api) to build nice porcelain (our own cli tool) for stuff like restart policies etc. It's targeted specifically at single-server solutions, and migrating once you are large enough to another platform is easy as we tend to not build in platform lock-in. Feel free to jump on our slack or irc: http://dokku.viewdocs.io/dokku/getting-started/where-to-get-help/ http://dokku.viewdocs.io/dokku/getting-started/where-to-get-... Disclaimer: I am a maintainer of Dokku

radarsat1 10y ago

I personally just getting into Docker and am looking at the swarm feature for deploying containerized compute nodes to our cluster. People here seem to be complaining specifically about Swarm mode, is there anything I should watch out for? What does Kubernetes provide that Docker swarm doesn't? I've tested Docker Swarm a bit and it seems to work as advertised. Is it just about node selection not being sophisticated enough? In that case I don't need to worry since all my nodes are the same, but if there's any words of warning I'm all ears. Thanks.

api 10y ago

Am I the only person around here who is skeptical of all this added complexity? Some of it is clearly very useful, but some of it strikes me as a tower of babel built upon workarounds to problems that have simpler solutions. What I see here are several different vendors vying to make sure they remain relevant. While I understand this and even the need for it, I also understand that it can drive poor engineering decisions in the long run.

maxamillion 10y ago

I've been using docker since version 0.6 and followed almost every version upgrade since and it's an absolute mess. This blog post is spot on. My old team had a production environment running in docker containers for about a year (this was pre-swarm, pre-kubernetes) and then transitioned to just using ansible for application deployment in a more "traditional" manner because we spent more time trying to fix broken things with docker than it was worth.