5 ms·

Containers as a technology are nice, but it's easy to fall back into the same traps that make software non-reproducible using containers as well. You can precis

Containers as a technology are nice, but it's easy to fall back into the same traps that make software non-reproducible using containers as well. You can precisely specify all your dependencies, but it often takes a lot of effort to make that happen. I'm a fan of the approach Guix developers are taking for scientific computing, because it makes reproducible software simple enough for people to use without too many headaches: https://hpc.guix.info/blog/2019/10/towards-reproducible-jupyter-notebooks/ https://hpc.guix.info/blog/2019/10/towards-reproducible-jupy...

CamperBob2 7y ago
Isn't the whole idea behind containers to eliminate external dependencies?
- FakeComments 7y ago
  Sure, but how many people just... apt install foo ...without pinning a version of foo?
- ryukafalz 7y ago
  In a sense. You're right that once a container is built, it has few external dependencies. But you need to get those dependencies from somewhere at build-time, and if you're not careful it's easy to do that in a way that makes it extremely difficult to rebuild that container in the future. To use a slightly more concrete example: let's say you're using a library in your container that has a severe bug. This bug results in incorrect computations, so you would like to upgrade to a fixed version. Now let's say that when you built that container initially, you installed packages in the Dockerfile by running e.g. "pip install <package>". The problem is that once this image is built, it's nontrivial to rebuild this image and ensure you're using the same dependencies you were the first time. In a sense, you've lost that information (though you can probably start to figure it out with close inspection of the image). Yes, there are usually ways around this with the language-specific package managers; Node has package-lock.json, Python has Pipfile.lock, etc. But it's not even close to being the default.

CamperBob2 7y ago

Isn't the whole idea behind containers to eliminate external dependencies?

FakeComments 7y ago
Sure, but how many people just... apt install foo ...without pinning a version of foo?
ryukafalz 7y ago
In a sense. You're right that once a container is built, it has few external dependencies. But you need to get those dependencies from somewhere at build-time, and if you're not careful it's easy to do that in a way that makes it extremely difficult to rebuild that container in the future. To use a slightly more concrete example: let's say you're using a library in your container that has a severe bug. This bug results in incorrect computations, so you would like to upgrade to a fixed version. Now let's say that when you built that container initially, you installed packages in the Dockerfile by running e.g. "pip install <package>". The problem is that once this image is built, it's nontrivial to rebuild this image and ensure you're using the same dependencies you were the first time. In a sense, you've lost that information (though you can probably start to figure it out with close inspection of the image). Yes, there are usually ways around this with the language-specific package managers; Node has package-lock.json, Python has Pipfile.lock, etc. But it's not even close to being the default.