9 ms·

Docker is such a mess. As a tool for building containers its cumbersome, has bizarre and frustrating limitations and has issues that haven’t been addressed in

Docker is such a mess. As a tool for building containers its cumbersome, has bizarre and frustrating limitations and has issues that haven’t been addressed in years. You can’t use semver for tags, it eats up all your disk space and you have to manually GC it, etc… Multi-build support is basically useless and invariably you end up writing convoluted bash scripts to get the thing to work. Caching is terrible pretty much across the board. How many petabytes of data are wasted every day re-syncing apt-get? As a runtime engine its basically dead for production use. If you ever try to use it you’ll quickly discover that it has tons of problems. It locks up, orphans processes, stops responding to commands, forgets about containers, etc. And its not safe to run arbitrary containers. You’d be surprised by how many companies using Kubernetes gave up using docker a long time ago. Please save yourself a lot of heartache and just use containerd. As a concept, containers never really lived up to their potential. "A Docker container image is a lightweight, standalone, executable package of software that includes everything needed to run an application." As long as that application runs in linux. As a company Docker is a failure. All their cloud stuff is clunky, poorly thought out and at this point largely irrelevent. I'll be surprised if it will last 5 years. Kubernetes won. Those cute, cuddly characters and the Docker name are basically all they have going for them, and all these things, the Moby rename, the requiring login to download, etc... are the death throes of a dying company. Docker is the next jQuery. People don't realize just how easy all this stuff will be to replace. Google already did it. Like 5 times. (gVisor, jib, kaniko, ...) It was probably some intern's side project. Microsoft should just get it over with and acquire the company. It would be the perfect cherry on top to their Github cake.

deleted 8y ago
[deleted]
notamy 8y ago
> You’d be surprised by how many companies using Kubernetes gave up using docker a long time ago. What are they using instead? cri-o? rkt?
- cdoxsey 8y ago
  At Datadog we are using containerd. We've heard of other companies using cri-o. There's also a lot of mixed deployments of container vs native out there. It seems Kubernetes is popular but not many companies with a large number of servers are willing to bet the farm on it, so they may only run a subset of their services with it (stateless, or test environments) You can use Docker, just be ready to account for the instability. With proper detection and strategies to evict bad nodes you can build a reliable platform out of it. (though for stateful things you may end up with a real mess on your hands) Weve seen issues with cointainerd too, but at least so far it seems more stable. Basically every piece of docker is being replaced. The runtime (containerd, crio), the tools to build containers (google has several), the server to host images (ecr, gcr, etc)... it's weird to call it docker when none of the components actually use docker anymore. The gui and installer is nice for local development, so I guess it has that going for it.
  - ibn_ibid 8y ago
    Do you have any good pointers on using containerd and building with it? I'm using Docker with the runc runtime but it still isn't the best experience.
  - hardwaresofton 8y ago
    +1 for containerd (+ cri-containerd, which is included by default now), it's served really wonderfully (meaning I didn't have to touch it much) as the base for k8s machines I've boostrapped recently -- I don't even install Docker anymore (as in literally docker the daemon is not installed or running, just runc + containerd). The replacement of docker is a good thing IMO (though probably not a good thing for Docker the company) -- it's one of the main benefits of the kubernetes hypetrain, the development of C*I (Container <something> Interface) has been great for the ecosystem. I personally find docker's CLI way more ergonomic however.
  - 7ewis 8y ago
    How do Dockerfiles work if you aren't using Docker? Is there another way to define what the container should look like?
politelemon 8y ago
> Please save yourself a lot of heartache and just use containerd. Part of the appeal of Docker is the ease with which developers new to the concept of containers can pick it up. I cannot say the same for containerd. Try a search for containerd tutorials and compare what you find with docker tutorials. > People don't realize just how easy all this stuff will be to replace. Think carefully about why that is. If it were easy, it would have been understood. So, no - understanding the replaceability of the container components is not easy. That is why people have not realized it. The issue with many docker's competitors is it appeals to docker experts. The abstraction and terminology itself is not straightforward. A lot of hyperbole in your comment is unnecessary and/or undeserved. Of course I realize that this is HN where k8s, rkt and containerd are used largely by the visiting audience. But don't forget the dark matter developers.
- majewsky 8y ago
  > Try a search for containerd tutorials and compare what you find with docker tutorials. containerd is just a building block. You'll actually be looking for a Kubernetes tutorial, which the internet is sprawling with.
  - oblio 8y ago
    Yeah, but how's the dev experience for Kubernetes, even with all those tutorial? How easy is it to install on Windows, for example?
    - rutthenut 8y ago
      If you have Docker for Windows, it includes K8S too, out of the box
    - tapoxi 8y ago
      choco install minikube choco install kubernetes-cli minikube start (You'll need Hyper-V, however.)
  - raesene9 8y ago
blablabla123 8y ago
I wouldn't go so far to call it a mess but there is some truth to it. Naively, one might expect the following use cases: - test-drive even the most complex deployments on your laptop - spin up all the cool admin tools you ever wanted without wasting months of precious life time - reproducible setups - everything is always up to date But as you say, caching is at best in a mediocre state. If you indeed want to run heavy-weight servers (like ELK), be ready to have a Terabyte space just for images unless you want to GC all the time when tweaking the configuration. Also the laptop thing doesn't work unless you either have a very beefy machine or just work with very lightweight software. In reality Docker can be used for these use cases: - Test-driving some heavy weight or complex server software - CI to some degree I'm starting to think good old Unix tools in combination with automation tools like Chef, Ansible, ... are the way to go - or even just plain .deb files...
IceDane 8y ago
Wow I'm so glad that I'm not the only one with this opinion. Every single time I've had to use docker I've had to slog through inane design decisions which seem to make everything harder than it needs to be. Everything is inconsistent and there are several commands which seem to essentially do the same thing but subtly do not, and so on .. It's like every design decision has just been made by a single developer who likes to come up with new and "smarter" ways to do things.
Jedi72 8y ago
> "As a concept, containers never really lived up to their potential. A Docker container image is a lightweight, standalone, executable package of software that includes everything needed to run an application. As long as that application runs in linux." This is how Erlang releases have worked for 20 years http://erlang.org/doc/design_principles/release_handling.html http://erlang.org/doc/design_principles/release_handling.htm...
inetknght 8y ago
My fellow employed engineers have been moving towards Docker for over a year; slow progress because there's so many problems that Docker introduces whose solutions are convoluted at best. That's on top of the security auditing nightmare if you ever decide to use stuff from Docker Hub.
stephenr 8y ago
> Those cute, cuddly characters I never really got how a shipping container on a whale makes for a good logo, unless it's just meant to be reflective of the apparent reliability: > the "whale" your container is riding on may just fucking dive underwater for hours at a time, but hey containers are designed to be in the ocean.. Wait, they're not?
johnmarcus 8y ago
Wtf are you talking about? I've been running docker in production for 3 years (30+ microservice platform) with litterallly 0 of those problems. Caching works, hence why you need to clean your disk space. And multi stage builds are awesome for many circumstances. It sounds like you don't really understand how to use docker all that well.
- cdoxsey 8y ago
  You've never had docker do anything weird in production? It didn't happen often, but we definitely had problems. We're running kubernetes clusters with several hundred nodes. Caching is very crude. When building its based on lines in the dockerfile which means adding a dependency means redownloading everything. You also can't mount a directory for builds. Multi stage builds are very limited in what they can do and often aren't powerful enough to implement efficient builds. You end up either having 20 minute builds or complex make files to work around the inefficient default workflow. FWIW an intelligent caching mechanism should not require manual cleanup. Thankfully kubernetes does this for us in production... but the crazy 1 GB images you end up with for a moderately complex python app make it hard. (especially when people use :latest and then there are 12 versions of the app laying around)
  - kinghajj 8y ago
    On our CI server, I setup an apt-cache container, and have a 'base' image derived from bitnami/minideb that sets the container's apt proxy to apt-cache. There's also Squid for HTTP caching, Archiva for Java artifacts, and Devpi for Python ones. So, sure, changes will require re-downloads, but they're pretty fast since everything's local. For getting multi-stage builds to work nicely, I just use a Makefile, defining separate targets for 'builder' images and 'runtime ones', and COPY artifacts from the former into the latter to form the final application images.
Mister_Snuggles 8y ago
> As a concept, containers never really lived up to their potential. "A Docker container image is a lightweight, standalone, executable package of software that includes everything needed to run an application." As long as that application runs in linux. I like the concept of containers as a lightweight alternative to virtualization. I even use containers (FreeBSD Jails, specifically) for some particular use-cases where I need stronger isolation than separate UIDs, but don't need a full-blown VM. I don't mind this use-case at all. I really dislike the reality of containers which seem to be "Our deployment procedure and dependencies are so insane so there is no hope of packaging this as an RPM or DEB, so here's an entire userland for you". As an example of papering over a crazy deployment/dependencies nightmare: I remember seeing a project[0] which used four Docker containers to apply some machine learning based automation to Hue lights. Two of the containers are basically infrastructure pieces (RabbitMQ and Cassandra), one container was dedicated to the machine learning piece and one acts to tie the other three together. I have no idea why this project needs to run four separate operating system instances to do this job. If I were building it I'd do it as one application, so no need for RabbitMQ, and I'd use an ORM to let the user choose which database is most suited. I'd have an SQLite database as a reasonable default. Maybe I'd have an option to publish stuff to some kind of message queue so that it could be consumed by other systems. Don't get me wrong, I love the idea behind this project and want to try it out, but it really feels like the author went container-crazy because they could and didn't stop to think about whether they should. [0] https://github.com/tlackemann/hubert https://github.com/tlackemann/hubert

deleted 8y ago

[deleted]

notamy 8y ago

> You’d be surprised by how many companies using Kubernetes gave up using docker a long time ago. What are they using instead? cri-o? rkt?

cdoxsey 8y ago
At Datadog we are using containerd. We've heard of other companies using cri-o. There's also a lot of mixed deployments of container vs native out there. It seems Kubernetes is popular but not many companies with a large number of servers are willing to bet the farm on it, so they may only run a subset of their services with it (stateless, or test environments) You can use Docker, just be ready to account for the instability. With proper detection and strategies to evict bad nodes you can build a reliable platform out of it. (though for stateful things you may end up with a real mess on your hands) Weve seen issues with cointainerd too, but at least so far it seems more stable. Basically every piece of docker is being replaced. The runtime (containerd, crio), the tools to build containers (google has several), the server to host images (ecr, gcr, etc)... it's weird to call it docker when none of the components actually use docker anymore. The gui and installer is nice for local development, so I guess it has that going for it.
- ibn_ibid 8y ago
  Do you have any good pointers on using containerd and building with it? I'm using Docker with the runc runtime but it still isn't the best experience.
- hardwaresofton 8y ago
  +1 for containerd (+ cri-containerd, which is included by default now), it's served really wonderfully (meaning I didn't have to touch it much) as the base for k8s machines I've boostrapped recently -- I don't even install Docker anymore (as in literally docker the daemon is not installed or running, just runc + containerd). The replacement of docker is a good thing IMO (though probably not a good thing for Docker the company) -- it's one of the main benefits of the kubernetes hypetrain, the development of C*I (Container <something> Interface) has been great for the ecosystem. I personally find docker's CLI way more ergonomic however.
- 7ewis 8y ago
  How do Dockerfiles work if you aren't using Docker? Is there another way to define what the container should look like?

politelemon 8y ago

> Please save yourself a lot of heartache and just use containerd. Part of the appeal of Docker is the ease with which developers new to the concept of containers can pick it up. I cannot say the same for containerd. Try a search for containerd tutorials and compare what you find with docker tutorials. > People don't realize just how easy all this stuff will be to replace. Think carefully about why that is. If it were easy, it would have been understood. So, no - understanding the replaceability of the container components is not easy. That is why people have not realized it. The issue with many docker's competitors is it appeals to docker experts. The abstraction and terminology itself is not straightforward. A lot of hyperbole in your comment is unnecessary and/or undeserved. Of course I realize that this is HN where k8s, rkt and containerd are used largely by the visiting audience. But don't forget the dark matter developers.

majewsky 8y ago
> Try a search for containerd tutorials and compare what you find with docker tutorials. containerd is just a building block. You'll actually be looking for a Kubernetes tutorial, which the internet is sprawling with.
- oblio 8y ago
  Yeah, but how's the dev experience for Kubernetes, even with all those tutorial? How easy is it to install on Windows, for example?
  - rutthenut 8y ago
    If you have Docker for Windows, it includes K8S too, out of the box
  - tapoxi 8y ago
    choco install minikube choco install kubernetes-cli minikube start (You'll need Hyper-V, however.)
- raesene9 8y ago

blablabla123 8y ago

I wouldn't go so far to call it a mess but there is some truth to it. Naively, one might expect the following use cases: - test-drive even the most complex deployments on your laptop - spin up all the cool admin tools you ever wanted without wasting months of precious life time - reproducible setups - everything is always up to date But as you say, caching is at best in a mediocre state. If you indeed want to run heavy-weight servers (like ELK), be ready to have a Terabyte space just for images unless you want to GC all the time when tweaking the configuration. Also the laptop thing doesn't work unless you either have a very beefy machine or just work with very lightweight software. In reality Docker can be used for these use cases: - Test-driving some heavy weight or complex server software - CI to some degree I'm starting to think good old Unix tools in combination with automation tools like Chef, Ansible, ... are the way to go - or even just plain .deb files...

IceDane 8y ago

Wow I'm so glad that I'm not the only one with this opinion. Every single time I've had to use docker I've had to slog through inane design decisions which seem to make everything harder than it needs to be. Everything is inconsistent and there are several commands which seem to essentially do the same thing but subtly do not, and so on .. It's like every design decision has just been made by a single developer who likes to come up with new and "smarter" ways to do things.

Jedi72 8y ago

> "As a concept, containers never really lived up to their potential. A Docker container image is a lightweight, standalone, executable package of software that includes everything needed to run an application. As long as that application runs in linux." This is how Erlang releases have worked for 20 years http://erlang.org/doc/design_principles/release_handling.html http://erlang.org/doc/design_principles/release_handling.htm...

inetknght 8y ago

My fellow employed engineers have been moving towards Docker for over a year; slow progress because there's so many problems that Docker introduces whose solutions are convoluted at best. That's on top of the security auditing nightmare if you ever decide to use stuff from Docker Hub.

stephenr 8y ago

> Those cute, cuddly characters I never really got how a shipping container on a whale makes for a good logo, unless it's just meant to be reflective of the apparent reliability: > the "whale" your container is riding on may just fucking dive underwater for hours at a time, but hey containers are designed to be in the ocean.. Wait, they're not?

johnmarcus 8y ago

Wtf are you talking about? I've been running docker in production for 3 years (30+ microservice platform) with litterallly 0 of those problems. Caching works, hence why you need to clean your disk space. And multi stage builds are awesome for many circumstances. It sounds like you don't really understand how to use docker all that well.

cdoxsey 8y ago
You've never had docker do anything weird in production? It didn't happen often, but we definitely had problems. We're running kubernetes clusters with several hundred nodes. Caching is very crude. When building its based on lines in the dockerfile which means adding a dependency means redownloading everything. You also can't mount a directory for builds. Multi stage builds are very limited in what they can do and often aren't powerful enough to implement efficient builds. You end up either having 20 minute builds or complex make files to work around the inefficient default workflow. FWIW an intelligent caching mechanism should not require manual cleanup. Thankfully kubernetes does this for us in production... but the crazy 1 GB images you end up with for a moderately complex python app make it hard. (especially when people use :latest and then there are 12 versions of the app laying around)
- kinghajj 8y ago
  On our CI server, I setup an apt-cache container, and have a 'base' image derived from bitnami/minideb that sets the container's apt proxy to apt-cache. There's also Squid for HTTP caching, Archiva for Java artifacts, and Devpi for Python ones. So, sure, changes will require re-downloads, but they're pretty fast since everything's local. For getting multi-stage builds to work nicely, I just use a Makefile, defining separate targets for 'builder' images and 'runtime ones', and COPY artifacts from the former into the latter to form the final application images.

Mister_Snuggles 8y ago

> As a concept, containers never really lived up to their potential. "A Docker container image is a lightweight, standalone, executable package of software that includes everything needed to run an application." As long as that application runs in linux. I like the concept of containers as a lightweight alternative to virtualization. I even use containers (FreeBSD Jails, specifically) for some particular use-cases where I need stronger isolation than separate UIDs, but don't need a full-blown VM. I don't mind this use-case at all. I really dislike the reality of containers which seem to be "Our deployment procedure and dependencies are so insane so there is no hope of packaging this as an RPM or DEB, so here's an entire userland for you". As an example of papering over a crazy deployment/dependencies nightmare: I remember seeing a project[0] which used four Docker containers to apply some machine learning based automation to Hue lights. Two of the containers are basically infrastructure pieces (RabbitMQ and Cassandra), one container was dedicated to the machine learning piece and one acts to tie the other three together. I have no idea why this project needs to run four separate operating system instances to do this job. If I were building it I'd do it as one application, so no need for RabbitMQ, and I'd use an ORM to let the user choose which database is most suited. I'd have an SQLite database as a reasonable default. Maybe I'd have an option to publish stuff to some kind of message queue so that it could be consumed by other systems. Don't get me wrong, I love the idea behind this project and want to try it out, but it really feels like the author went container-crazy because they could and didn't stop to think about whether they should. [0] https://github.com/tlackemann/hubert https://github.com/tlackemann/hubert