6 ms·

Everyone says containers help resource utilization but I think there killer raison d'etre is that they are a common static binary packaging mechanism. I can shi

Everyone says containers help resource utilization but I think there killer raison d'etre is that they are a common static binary packaging mechanism. I can ship Java, Go, Python, or whatever and the download and run mechanism is all abstracted away.

dreamcompiler 10y ago
Does this mean we're admitting defeat with shared libraries and we're going back to static libraries again?
- mook 10y ago
  Mostly, yes. Notice that Go and Rust (two of the newer languages popular at least on HN) also feature static compilation by default. Turns out that shared libraries are awesome, until the libraries can't provide a consistently backwards compatible ABI.
  - zuzun 10y ago
    Go has no versioning, in Rust everything is version 0.1... then one day you update that serialization library from 0.1.4 to 0.1.5 and all hell breaks loose because you didn't notice they changed their data format internally and now your new process can't communicate with the old ones and your integration test missed that because it was running all tests with the new version on your machine. This makes you implement the policy "Only rebuild and ship the full stack" and there you are, scp'ing 1GB of binaries to your server because libleftpad just got updated.
    - michaelmrose 10y ago
      Except outside of Javascript nobody on earth makes libleftpad
    - michaelmrose 10y ago
      Except outside of Javascript nobody on earth makes libleftpad and whose binaries are 1gb?
      - qznc 10y ago
        In D it is part of the standard library. ;) https://dlang.org/library/std/range/pad_left.html https://dlang.org/library/std/range/pad_left.html
- DaiPlusPlus 10y ago
  Static libraries can't be replaced/updated post-deployment - you need to rebuild, whereas, shared libraries in a container can be - which is useful if you're working with dependencies that are updated regularly (in a non-breaking fashion) or proprietary binary blobs.
  - amenghra 10y ago
    It would be nice to have a third option when building binaries: some kind of tar/jar/zip archive with all the dependencies inside. It would give the pros of static and shared libraries without everything else containers imply. The OS could the be smart enough to only load identical libraries once.
    - Longhanks 10y ago
      That's basically what OS X does with bundles.
    - ori_b 10y ago
      That's equivalent to static linking, but with extra runtime overhead. You can already efficiently ship updates to binaries with something like bsdiff or cougrette, so the only reason to bundle shared libraries in an archive is for LGPL license compliance, or for poorly thought out code that wants to dlopen() itself.
      - amenghra 10y ago
        Upgrading a library that has been statically linked isn't as nice as a shared lib + afaik the OS doesn't reuse memory for static libs.
    - dkersten 10y ago
      jars already support this.
      - amenghra 10y ago
        yes, I think we should have the same capabilities in a language agnostic way. Signed jars are a little painful to use (you can't easily bundle them), but that's a minor issue.
- cobookman 10y ago
  Disk space is cheap. And we've got multi CPU core servers. So now we have the issue that you have lots of applications running on the same server, and how do we make sure the right version of some shared lib is on there. And that we won't break another program by updating it. Containers solve that. No more worrying if that java 8 upgrade will break some old application. So now every application stack is a static application.
  - arianvanp 10y ago
    Isn't just about disk space though. It also allows you to quickly make API compatible vulnerability fixes without a rebuild of your application.
    - sagichmal 10y ago
      This isn't a virtue. Containers solve problems in automated continuous-deployment environments where rebuilding and deploying your fleet of cattle is one click away. In the best case, no single container is alive for more than O(hours). Static linking solves way more operational problems than the loss of dynamic linking introduces, security or otherwise.
      - eeZah7Ux 10y ago
        > rebuilding and deploying your fleet ...this applies only software developed and run internally, which is a small fraction of all the software running in the world.
      - josteink 10y ago
        > This isn't a virtue. Containers solve problems in automated continuous-deployment environments where rebuilding and deploying your fleet of cattle is one click away. This has literally zero to do with containers and everything to do with an automated deployment pipeline. As a quick FYI: Those are not unique to containers.
      - _jal 10y ago
        I agree that moving towards static linking, on balance, seems like a a reasonable tradeoff at this point, but it is hardly as cut and dried as a lot of people seem to think. As one very minor point, it turns vulnerability tracking in to an accounting exercise, which sounds like a good idea until you take a look at the dexterity with which most engineering firms manage their AWS accounts. (Sure, just get better at it and it won't be a problem. That advice works with everything else, right?) One's choice of deployment tools may slap a bandaid on some things, but that is not the same thing as solving a problem; that is automated bandaid application. And odd pronouncements like any given container shouldn't be long lived are... odd. I guess if all you do is serve CRUD queries with them, that's probably OK. As a final point, I feel like the container advocates are selling an engineer's view of how ops should work. As with most things, neutral to good ideas end up wrapped up with a lot of rookie mistakes, not to mention typical engineer arrogance[1]. Just the same thing you get anywhere amateurs lecture the pros, but the current hype train surrounding docker is enough to let it actually cause problems[2]. My takeaway is still the same as it was when the noise started. Docker has the potential to be a nice bundling of Linux capabilities as an evolution of a very old idea that solves some real problems in some situations, and I look forward to it growing up. In the mean time, I'm bored with this engineering fad; can we get on with the next one already? [1] One very simple example, because I know someone will ask: Kubernetes logging is a stupid mess that doesn't play well with... well, anything. And to be fair, ops engineers are no better with the arrogance. [2] Problems like there being not even a single clearly production-ready host platform out of the box. Centos? Not yet. Ubuntu? Best of the bunch, but still hacky and buggy. CoreOS? I thought one of the points was a unified platform for dev and prod.
wmf 10y ago
Or you could have done the same thing years earlier with AMIs?
- tannhaeuser 10y ago
  But AMIs are full VM images as opposed to container images, aren't they?
  - wmf 10y ago
    Most Docker images also contain a full OS.
    - cookiecaper 10y ago
      Yes, everyone overlooks this and talks about how Docker containers are "app-only images" or something. They're not app-only. They may be using a thin OS like Alpine, but there's still a completely independent userspace layer. The only thing imported from the host is the kernel. If you made VM images in the same way, they'd also be 200M. The benefit of "containers" is that you don't need to siphon off a dedicated section of RAM to the application.
ori_b 10y ago
This is why something like 25% of containers in the docker registry ship with known vulnerabilities.

dreamcompiler 10y ago

Does this mean we're admitting defeat with shared libraries and we're going back to static libraries again?

mook 10y ago
Mostly, yes. Notice that Go and Rust (two of the newer languages popular at least on HN) also feature static compilation by default. Turns out that shared libraries are awesome, until the libraries can't provide a consistently backwards compatible ABI.
- zuzun 10y ago
  Go has no versioning, in Rust everything is version 0.1... then one day you update that serialization library from 0.1.4 to 0.1.5 and all hell breaks loose because you didn't notice they changed their data format internally and now your new process can't communicate with the old ones and your integration test missed that because it was running all tests with the new version on your machine. This makes you implement the policy "Only rebuild and ship the full stack" and there you are, scp'ing 1GB of binaries to your server because libleftpad just got updated.
  - michaelmrose 10y ago
    Except outside of Javascript nobody on earth makes libleftpad
  - michaelmrose 10y ago
    Except outside of Javascript nobody on earth makes libleftpad and whose binaries are 1gb?
    - qznc 10y ago
      In D it is part of the standard library. ;) https://dlang.org/library/std/range/pad_left.html https://dlang.org/library/std/range/pad_left.html
DaiPlusPlus 10y ago
Static libraries can't be replaced/updated post-deployment - you need to rebuild, whereas, shared libraries in a container can be - which is useful if you're working with dependencies that are updated regularly (in a non-breaking fashion) or proprietary binary blobs.
- amenghra 10y ago
  It would be nice to have a third option when building binaries: some kind of tar/jar/zip archive with all the dependencies inside. It would give the pros of static and shared libraries without everything else containers imply. The OS could the be smart enough to only load identical libraries once.
  - Longhanks 10y ago
    That's basically what OS X does with bundles.
  - ori_b 10y ago
    That's equivalent to static linking, but with extra runtime overhead. You can already efficiently ship updates to binaries with something like bsdiff or cougrette, so the only reason to bundle shared libraries in an archive is for LGPL license compliance, or for poorly thought out code that wants to dlopen() itself.
    - amenghra 10y ago
      Upgrading a library that has been statically linked isn't as nice as a shared lib + afaik the OS doesn't reuse memory for static libs.
  - dkersten 10y ago
    jars already support this.
    - amenghra 10y ago
      yes, I think we should have the same capabilities in a language agnostic way. Signed jars are a little painful to use (you can't easily bundle them), but that's a minor issue.
cobookman 10y ago
Disk space is cheap. And we've got multi CPU core servers. So now we have the issue that you have lots of applications running on the same server, and how do we make sure the right version of some shared lib is on there. And that we won't break another program by updating it. Containers solve that. No more worrying if that java 8 upgrade will break some old application. So now every application stack is a static application.
- arianvanp 10y ago
  Isn't just about disk space though. It also allows you to quickly make API compatible vulnerability fixes without a rebuild of your application.
  - sagichmal 10y ago
    This isn't a virtue. Containers solve problems in automated continuous-deployment environments where rebuilding and deploying your fleet of cattle is one click away. In the best case, no single container is alive for more than O(hours). Static linking solves way more operational problems than the loss of dynamic linking introduces, security or otherwise.
    - eeZah7Ux 10y ago
      > rebuilding and deploying your fleet ...this applies only software developed and run internally, which is a small fraction of all the software running in the world.
    - josteink 10y ago
      > This isn't a virtue. Containers solve problems in automated continuous-deployment environments where rebuilding and deploying your fleet of cattle is one click away. This has literally zero to do with containers and everything to do with an automated deployment pipeline. As a quick FYI: Those are not unique to containers.
    - _jal 10y ago
      I agree that moving towards static linking, on balance, seems like a a reasonable tradeoff at this point, but it is hardly as cut and dried as a lot of people seem to think. As one very minor point, it turns vulnerability tracking in to an accounting exercise, which sounds like a good idea until you take a look at the dexterity with which most engineering firms manage their AWS accounts. (Sure, just get better at it and it won't be a problem. That advice works with everything else, right?) One's choice of deployment tools may slap a bandaid on some things, but that is not the same thing as solving a problem; that is automated bandaid application. And odd pronouncements like any given container shouldn't be long lived are... odd. I guess if all you do is serve CRUD queries with them, that's probably OK. As a final point, I feel like the container advocates are selling an engineer's view of how ops should work. As with most things, neutral to good ideas end up wrapped up with a lot of rookie mistakes, not to mention typical engineer arrogance[1]. Just the same thing you get anywhere amateurs lecture the pros, but the current hype train surrounding docker is enough to let it actually cause problems[2]. My takeaway is still the same as it was when the noise started. Docker has the potential to be a nice bundling of Linux capabilities as an evolution of a very old idea that solves some real problems in some situations, and I look forward to it growing up. In the mean time, I'm bored with this engineering fad; can we get on with the next one already? [1] One very simple example, because I know someone will ask: Kubernetes logging is a stupid mess that doesn't play well with... well, anything. And to be fair, ops engineers are no better with the arrogance. [2] Problems like there being not even a single clearly production-ready host platform out of the box. Centos? Not yet. Ubuntu? Best of the bunch, but still hacky and buggy. CoreOS? I thought one of the points was a unified platform for dev and prod.

wmf 10y ago

Or you could have done the same thing years earlier with AMIs?

tannhaeuser 10y ago
But AMIs are full VM images as opposed to container images, aren't they?
- wmf 10y ago
  Most Docker images also contain a full OS.
  - cookiecaper 10y ago
    Yes, everyone overlooks this and talks about how Docker containers are "app-only images" or something. They're not app-only. They may be using a thin OS like Alpine, but there's still a completely independent userspace layer. The only thing imported from the host is the kernel. If you made VM images in the same way, they'd also be 200M. The benefit of "containers" is that you don't need to siphon off a dedicated section of RAM to the application.

ori_b 10y ago

This is why something like 25% of containers in the docker registry ship with known vulnerabilities.