5 ms·

Millions of transactions per second is not the point of using docker. Docker is about running multiple things on the same server. If you have 1 server and you

by ozim 6y ago

Millions of transactions per second is not the point of using docker. Docker is about running multiple things on the same server. If you have 1 server and you have to run 1 app on it, you don't need a Docker container. Now if you have physical server and you want to use it fully you want to run ALL kinds of apps on it. What is hard is that running multiple apps, you run into issues with dependencies. Ideally you run each app on different server or separate OS then you don't have that problem, but that is costly and if one app sits idle you waste money you spent on the server. I don't know why people think about it the other way that you get 1 app and run it distributed is "peak containers". Peak containers is about having common infra so you make the most of your server because you don't care what app is running in there, your server does not need to have any special config for your special app. You don't get bogged down that you cannot install anything else on that server because of dependency hell. Well from the application point of view bonus is that if you dockerize your app you don't care about the servers it is running on. You just ship it and it will run on someones server with other apps that you don't know about and don't care about and their dependencies will not break your app.

capableweb 6y ago
> Docker is about running multiple things on the same server Or about running a single thing on multiple servers. Or providing a temporary runtime with language/lib/software X preinstalled. Or to facilitate easier testing. In the end, many people use containers for many different reasons. There isn't a single "This is what docker is for", although people tend to shove containers into places that might not be 100% suitable for containers.
- sdfhbdf 6y ago
  > There isn't a single "This is what docker is for" 100% this. Choosing the right tool for the job, that's what distinguishes developers from engineers in my opinion. There is place for almost every tool be it leftpad npm package or running go on bare metal. Most sensationalist articles, "Why we don't use [popular tool]" just seem to have usecase that doesn't fit this particular tool.
StreamBright 6y ago
> Now if you have physical server and you want to use it fully you want to run ALL kinds of apps on it. There are so many options other than Docker as of today. These provide very different tradeoffs. Somehow we got to the point that people think that there are only one solution (Docker) to run multiple applications on a single server. Some other options: - single binary packaging - LXC / LXD - Firecracker - chroot
IshKebab 6y ago
> What is hard is that running multiple apps, you run into issues with dependencies. That's exactly the point they were making in this article. Also even with 1 server and 1 app people still use Docker because the pain of setting up a Linux machine with all of the dependencies for your average Linux software is so great, and Docker simplifies that. The single static binary is such a great feature of Go. I wish more languages would copy it.
- meepmorp 6y ago
  Can you compile single binaries for go that has C dependencies? But, yeah, I like the "all-in-one" executable thing and wish it was more prevalent.
  - IshKebab 6y ago
    Sure. They're more difficult to cross-compile though.
KaiserPro 6y ago
> Now if you have physical server and you want to use it fully you want to run ALL kinds of apps on it. What is hard is that running multiple apps, you run into issues with dependencies. Ideally you run each app on different server or separate OS then you don't have that problem, but that is costly and if one app sits idle you waste money you spent on the server. I mean yeah, it was sortof a problem on late 90s x86 kit. But its not been a real problem for a good 20 years. virtualisation solved that issue. Modern processors (well not even modern ones, any xeon from the last 10 years) will have virtualisation optimization in them. Docker allows you to not be as disciplined when it comes to your build environment. This means that you can let developers ship any old shit. This will give you a short term productivity boost, and longterm tech debt. > your server does not need to have any special config for your special app yeahna, thats not really true. You still need to manage secrets (docker has no real security bits built in, barring cgroups) you still have to manage volumes, and you still have to manage connectivity. Not to mention resource discovery. The biggest lie that K8s and docker told us is that you didn't need a coherent plan for your infra. To be efficient, you need to make your apps work on the same dependencies, same platform, and re-use as much as possible.
chriswarbo 6y ago
I use Nix for building projects and managing my laptops (NixOS at home, nix-darwin on macOS at work). This solves the 'dependency hell' in a much cleaner, more efficient and IMHO simpler way than docker. For those who don't know, Nix doesn't install anything system-wide. Everything gets prefixed with a hash of its dependencies (e.g. /nix/store/cfh2830k2p1gg10j40x27rmlmpwqy7p4-git-2.23.1/bin/git). Since nothing is available system-wide, program A can only use program B by specifying its whole path (including the hash); this affects the hash of program A, and so on for anything that refers to it. This is a Merkle tree, giving the provenance of every dependency, all the way down to the 'bootstrap' tools (e.g. a prebuilt GCC that was used to build the GCC that was used to build the tar that was used to extract the bash source ....). Here's a snippet of the dependencies used to build the above git command: $ nix-store -q --requisites $(nix-store --deriver /nix/store/cfh2830k2p1gg10j40x27rmlmpwqy7p4-git-2.23.1) /nix/store/01n3wxxw29wj2pkjqimmmjzv7pihzmd7-which-2.21.tar.gz.drv /nix/store/064jmylcq7h6fa5asg0rna9awcqz3765-0001-x86-Properly-merge-GNU_PROPERTY_X86_ISA_1_USED.patch /nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh /nix/store/na8n4ndbmkc500jm7argsprzj94p27h4-libev-4.27.tar.gz.drv /nix/store/33sl3bqjcqzrdd9clgaad3ljlwyl1pkb-patch-shebangs.sh /nix/store/81ikflgpwzgjk8b5vmvg9gaw9mbkc86k-compress-man-pages.sh /nix/store/9ny6szla9dg61jv8q22qbnqsz37465n0-multiple-outputs.sh /nix/store/a92kz10cwkpa91k5239inl3fd61zp5dh-move-lib64.sh /nix/store/5g9j02amqihyg9b15m9bds9iqkjclcgr-bootstrap-tools.tar.xz.drv /nix/store/b7irlwi2wjlx5aj1dghx4c8k3ax6m56q-busybox.drv /nix/store/c0sr4qdy8halrdrh5dpm7hj05c6hyssa-unpack-bootstrap-tools.sh /nix/store/dccfayc70jsvzwq023smvqv04c57bsib-bootstrap-tools.drv ... Here's a snippet of the above git command's runtime dependencies (i.e. everything it references which, if it were our app, we'd need to include in its container): $ nix-store -q --requisites /nix/store/cfh2830k2p1gg10j40x27rmlmpwqy7p4-git-2.23.1 /nix/store/5gga7b7aavb4vcnia44mzd6m9vrx46gq-perl5.30.0-LWP-MediaTypes-6.04 /nix/store/9n0n6dg310azjmnyfxq6bsn0jsz9cvk8-perl5.30.0-URI-1.76 /nix/store/9zmmc8akdb2jlkapqf9p26jn84xvzbjr-perl5.30.0-HTTP-Date-6.02 /nix/store/ds92hvb2jx8yfh2j3wqy1fqr07654nkr-perl5.30.0-Encode-Locale-1.05 /nix/store/l43m8p3bkqar54krnj3jalbim7m64lid-perl5.30.0-IO-HTML-1.001 /nix/store/zbd643lb2a8iqgp1m36r1adzpqpwhr7s-perl5.30.0-HTTP-Message-6.18 /nix/store/0hkq3crf7357z5zywx4i9dkddi0qvb8k-perl5.30.0-HTTP-Daemon-6.01 /nix/store/5ka41zhii1bjss3f60rzd2npz9mxj060-glibc-2.27 /nix/store/7fvwr8la2k701hrx2w5xnvjr5kkc7ysv-gcc-8.3.0-lib /nix/store/clp0hgzq3pwx03lr8apg92nmswr4izvz-bash-4.4-p23 /nix/store/hgdh73wvrncfnmdmg1damcfrviv9cgp7-gnum4-1.4.18 /nix/store/ig8wzhnxxxrspcyb85wxqdbbn2q53088-bison-3.4.2 /nix/store/11z0140njah4dbrngg5bcx99nyw391kw-gettext-0.19.8.1 /nix/store/1ycvwhslagg7dn8rpc3simyfkirgjbp5-perl5.30.0-Net-HTTP-6.19 /nix/store/n3ja6nl5i91x5sn232f5fyhmx7lxcmj1-ncurses-6.1-20190112 /nix/store/47w0y1vavaxja3mm9gr5dmbgxrjgpw21-libedit-20190324-3.1 /nix/store/84jxhr8l3plkd6z2x4v941za9kvmv88g-zlib-1.2.11 /nix/store/dfgm23z5mvp7i3085dlq4rn0as41jp1p-openssl-1.1.1d ... All of these builds are performed in a sandbox, with no access to the network or global filesystem (e.g. no $HOME, /usr/bin, etc.), every file's timestamps are set to 1970-01-01T00:00:01 and the /nix/store filesystem is read-only. The Guix project takes this a bit further, with its `guix pack` command turning paths like this into a runnable docker image (amongst other options). Compare this to the way docker is used: we download and run some large binary blob ('image'), containing who-knows-what. These images are usually bloated with all sorts os unneeded cruft (due to the way docker builds images 'from the inside'); often a whole OS, package management tools, or at the very least a shell like busybox, all of which makes scaling harder and increases our attack surface. The build steps for those images are often insecure and unreproducible too, e.g. running commands like `apt-get install -y foo` or `pip install bar`, whose behaviour changes over time dependending on the contents of third-party servers. I really want to start using containers for their security and deployment advantages, but so much of their tooling and approach seems completely backwards to me :( (Note that it's possible to build docker images using Nix, but this requires running commands like 'docker load' which I can't get working on my Mac, and deploying to AWS ECS seems to require 'image registries' rather than just pointing to a file on S3. Urgh.)
koheripbal 6y ago
Why not just run VMs? If you have a few VM images you can deploy - isn't it the same benefit for less work?
- ozim 6y ago
  Docker has tooling for building images and is lighter than VM. Devs can make their docker containers quite easily. With VM you have to update operating system and other stuff. With Docker as a Developer you are concerned only about your direct dependencies. I run my systems in VMs and it is quite a lot of work to update all unrelated stuff. Well related stuff is also a lot of work but that is price I am willing to pay for.

by ozim 6y ago

capableweb 6y ago

> Docker is about running multiple things on the same server Or about running a single thing on multiple servers. Or providing a temporary runtime with language/lib/software X preinstalled. Or to facilitate easier testing. In the end, many people use containers for many different reasons. There isn't a single "This is what docker is for", although people tend to shove containers into places that might not be 100% suitable for containers.

sdfhbdf 6y ago
> There isn't a single "This is what docker is for" 100% this. Choosing the right tool for the job, that's what distinguishes developers from engineers in my opinion. There is place for almost every tool be it leftpad npm package or running go on bare metal. Most sensationalist articles, "Why we don't use [popular tool]" just seem to have usecase that doesn't fit this particular tool.

StreamBright 6y ago

> Now if you have physical server and you want to use it fully you want to run ALL kinds of apps on it. There are so many options other than Docker as of today. These provide very different tradeoffs. Somehow we got to the point that people think that there are only one solution (Docker) to run multiple applications on a single server. Some other options: - single binary packaging - LXC / LXD - Firecracker - chroot

IshKebab 6y ago

> What is hard is that running multiple apps, you run into issues with dependencies. That's exactly the point they were making in this article. Also even with 1 server and 1 app people still use Docker because the pain of setting up a Linux machine with all of the dependencies for your average Linux software is so great, and Docker simplifies that. The single static binary is such a great feature of Go. I wish more languages would copy it.

meepmorp 6y ago
Can you compile single binaries for go that has C dependencies? But, yeah, I like the "all-in-one" executable thing and wish it was more prevalent.
- IshKebab 6y ago
  Sure. They're more difficult to cross-compile though.

KaiserPro 6y ago

> Now if you have physical server and you want to use it fully you want to run ALL kinds of apps on it. What is hard is that running multiple apps, you run into issues with dependencies. Ideally you run each app on different server or separate OS then you don't have that problem, but that is costly and if one app sits idle you waste money you spent on the server. I mean yeah, it was sortof a problem on late 90s x86 kit. But its not been a real problem for a good 20 years. virtualisation solved that issue. Modern processors (well not even modern ones, any xeon from the last 10 years) will have virtualisation optimization in them. Docker allows you to not be as disciplined when it comes to your build environment. This means that you can let developers ship any old shit. This will give you a short term productivity boost, and longterm tech debt. > your server does not need to have any special config for your special app yeahna, thats not really true. You still need to manage secrets (docker has no real security bits built in, barring cgroups) you still have to manage volumes, and you still have to manage connectivity. Not to mention resource discovery. The biggest lie that K8s and docker told us is that you didn't need a coherent plan for your infra. To be efficient, you need to make your apps work on the same dependencies, same platform, and re-use as much as possible.

chriswarbo 6y ago

I use Nix for building projects and managing my laptops (NixOS at home, nix-darwin on macOS at work). This solves the 'dependency hell' in a much cleaner, more efficient and IMHO simpler way than docker. For those who don't know, Nix doesn't install anything system-wide. Everything gets prefixed with a hash of its dependencies (e.g. /nix/store/cfh2830k2p1gg10j40x27rmlmpwqy7p4-git-2.23.1/bin/git). Since nothing is available system-wide, program A can only use program B by specifying its whole path (including the hash); this affects the hash of program A, and so on for anything that refers to it. This is a Merkle tree, giving the provenance of every dependency, all the way down to the 'bootstrap' tools (e.g. a prebuilt GCC that was used to build the GCC that was used to build the tar that was used to extract the bash source ....). Here's a snippet of the dependencies used to build the above git command: $ nix-store -q --requisites $(nix-store --deriver /nix/store/cfh2830k2p1gg10j40x27rmlmpwqy7p4-git-2.23.1) /nix/store/01n3wxxw29wj2pkjqimmmjzv7pihzmd7-which-2.21.tar.gz.drv /nix/store/064jmylcq7h6fa5asg0rna9awcqz3765-0001-x86-Properly-merge-GNU_PROPERTY_X86_ISA_1_USED.patch /nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh /nix/store/na8n4ndbmkc500jm7argsprzj94p27h4-libev-4.27.tar.gz.drv /nix/store/33sl3bqjcqzrdd9clgaad3ljlwyl1pkb-patch-shebangs.sh /nix/store/81ikflgpwzgjk8b5vmvg9gaw9mbkc86k-compress-man-pages.sh /nix/store/9ny6szla9dg61jv8q22qbnqsz37465n0-multiple-outputs.sh /nix/store/a92kz10cwkpa91k5239inl3fd61zp5dh-move-lib64.sh /nix/store/5g9j02amqihyg9b15m9bds9iqkjclcgr-bootstrap-tools.tar.xz.drv /nix/store/b7irlwi2wjlx5aj1dghx4c8k3ax6m56q-busybox.drv /nix/store/c0sr4qdy8halrdrh5dpm7hj05c6hyssa-unpack-bootstrap-tools.sh /nix/store/dccfayc70jsvzwq023smvqv04c57bsib-bootstrap-tools.drv ... Here's a snippet of the above git command's runtime dependencies (i.e. everything it references which, if it were our app, we'd need to include in its container): $ nix-store -q --requisites /nix/store/cfh2830k2p1gg10j40x27rmlmpwqy7p4-git-2.23.1 /nix/store/5gga7b7aavb4vcnia44mzd6m9vrx46gq-perl5.30.0-LWP-MediaTypes-6.04 /nix/store/9n0n6dg310azjmnyfxq6bsn0jsz9cvk8-perl5.30.0-URI-1.76 /nix/store/9zmmc8akdb2jlkapqf9p26jn84xvzbjr-perl5.30.0-HTTP-Date-6.02 /nix/store/ds92hvb2jx8yfh2j3wqy1fqr07654nkr-perl5.30.0-Encode-Locale-1.05 /nix/store/l43m8p3bkqar54krnj3jalbim7m64lid-perl5.30.0-IO-HTML-1.001 /nix/store/zbd643lb2a8iqgp1m36r1adzpqpwhr7s-perl5.30.0-HTTP-Message-6.18 /nix/store/0hkq3crf7357z5zywx4i9dkddi0qvb8k-perl5.30.0-HTTP-Daemon-6.01 /nix/store/5ka41zhii1bjss3f60rzd2npz9mxj060-glibc-2.27 /nix/store/7fvwr8la2k701hrx2w5xnvjr5kkc7ysv-gcc-8.3.0-lib /nix/store/clp0hgzq3pwx03lr8apg92nmswr4izvz-bash-4.4-p23 /nix/store/hgdh73wvrncfnmdmg1damcfrviv9cgp7-gnum4-1.4.18 /nix/store/ig8wzhnxxxrspcyb85wxqdbbn2q53088-bison-3.4.2 /nix/store/11z0140njah4dbrngg5bcx99nyw391kw-gettext-0.19.8.1 /nix/store/1ycvwhslagg7dn8rpc3simyfkirgjbp5-perl5.30.0-Net-HTTP-6.19 /nix/store/n3ja6nl5i91x5sn232f5fyhmx7lxcmj1-ncurses-6.1-20190112 /nix/store/47w0y1vavaxja3mm9gr5dmbgxrjgpw21-libedit-20190324-3.1 /nix/store/84jxhr8l3plkd6z2x4v941za9kvmv88g-zlib-1.2.11 /nix/store/dfgm23z5mvp7i3085dlq4rn0as41jp1p-openssl-1.1.1d ... All of these builds are performed in a sandbox, with no access to the network or global filesystem (e.g. no $HOME, /usr/bin, etc.), every file's timestamps are set to 1970-01-01T00:00:01 and the /nix/store filesystem is read-only. The Guix project takes this a bit further, with its `guix pack` command turning paths like this into a runnable docker image (amongst other options). Compare this to the way docker is used: we download and run some large binary blob ('image'), containing who-knows-what. These images are usually bloated with all sorts os unneeded cruft (due to the way docker builds images 'from the inside'); often a whole OS, package management tools, or at the very least a shell like busybox, all of which makes scaling harder and increases our attack surface. The build steps for those images are often insecure and unreproducible too, e.g. running commands like `apt-get install -y foo` or `pip install bar`, whose behaviour changes over time dependending on the contents of third-party servers. I really want to start using containers for their security and deployment advantages, but so much of their tooling and approach seems completely backwards to me :( (Note that it's possible to build docker images using Nix, but this requires running commands like 'docker load' which I can't get working on my Mac, and deploying to AWS ECS seems to require 'image registries' rather than just pointing to a file on S3. Urgh.)

koheripbal 6y ago

Why not just run VMs? If you have a few VM images you can deploy - isn't it the same benefit for less work?

ozim 6y ago
Docker has tooling for building images and is lighter than VM. Devs can make their docker containers quite easily. With VM you have to update operating system and other stuff. With Docker as a Developer you are concerned only about your direct dependencies. I run my systems in VMs and it is quite a lot of work to update all unrelated stuff. Well related stuff is also a lot of work but that is price I am willing to pay for.