4 ms·

If I understand correctly, Dockerfile, and image layers, encode that path, making it retrace-able, yes?

by ttymck 3y ago

dsr_ 3y ago
If I tell you that there's a remote code execution in libfoobar-1.03 through 1.15, how long does it take you to verify where libfoobar is installed, and what versions are in use? Remember, nobody ships an image layer of libfoobar, it's a common component, though not as common as openssl. Is there one command, or one script, which can do that? You need that, basically daily. Is there one command to rebuild with the new libfoobar-1.17, and at most one more to test for basic functionality? You need that, too.
- Spivak 3y ago
  I mean you're not gonna like the answer but in real life when herding cats the answer is setting up an image scanner and renovate, and calling it a day. It's not like OS images are any better in this respect. I have bit my teeth long enough on software that depends on the base OS and not being able to infra upgrades. Bifurcating responsibility into app/platform is a breath of fresh air by comparison.
- alexchantavy 3y ago
  Yup this is a hard problem. Shameless plug to my blogpost on how we built something to do this: https://eng.lyft.com/vulnerability-management-at-lyft-enforcing-the-cascade-part-1-234d1561b994 https://eng.lyft.com/vulnerability-management-at-lyft-enforc...
- bandrami 3y ago
  Same problem with flatpak. How many versions of openssl are on my laptop? I have no idea!
- salawat 3y ago
  ....I can tell you which ones are used by the current linker on the system in what order. ldconfig -p | grep libfoobar If you go and spread the linkers all over hither and yon and make it nigh impossible to get results out of them, or don't bother to keep track of where you put things... Welp. Can't help ya there. Shoulda been tracking that all along. Oh, excuse me... That'll only work for dynamically built things in the abscence of statically linked stuff or clever LD_PRELOAD shenanigans. Hope ya don't do that. Fact is. You're really never going to get away from keeping track of all the moving pieces. You're just shuffling complexity and layers of abstraction around.
teraflop 3y ago
Not in general, no. Docker image layers are just snapshots of the filesystem changes that result from build commands. But there is nothing that guarantees that the effects of those commands are reproducible. For example, it's incredibly common for Dockerfiles to contain commands like: RUN apt-get update && apt-get install -y foobar which means when you build the container, you get whatever version of foobar happens to currently be in the repository that your base image points to. So you can easily end up in a situation where rebuilding an image gives you different behavior, even though nothing in your Dockerfile or build context changed.
- hinkley 3y ago
  Even the Docker support team is confused by this. Most of why I stopped engaging is that they would reject a feature because it lead to dockerfiles they said were 'not reproducible', but were just as reproducible as many of the core features. Which is to say, not in the slightest. Every time you do something in a Dockerfile that involves pulling something from the network, you don't have reproducibility. Unless you pull something using a cryptographic hash (which I've never actually seen anyone do), the network can give you a different answer every time. The answer might not even be idempotent (calling it once changes the result of calling it again). My argument was the same as yours. Virtually every dockerfile has an 'update' call as the second or third layer, which means none of the repeatable layers afterward (like useradd or mkdir) are actually repeatable. You're building a castle on the sand. Docker images are reproducible. Docker builds are not. And that's okay, as long as everyone accepts that to be true. And if Docker contributors can't accept that then we are all truly fucked, until something else comes along that retreads the ideas.
  - xvector 3y ago
    Sounds like Nix is what you're looking for.
    - evilduck 3y ago
      Which can conveniently produce Docker images.
deleted 3y ago
[deleted]

If I understand correctly, Dockerfile, and image layers, encode that path, making it retrace-able, yes?

by ttymck 3y ago

If I understand correctly, Dockerfile, and image layers, encode that path, making it retrace-able, yes?

dsr_ 3y ago

If I tell you that there's a remote code execution in libfoobar-1.03 through 1.15, how long does it take you to verify where libfoobar is installed, and what versions are in use? Remember, nobody ships an image layer of libfoobar, it's a common component, though not as common as openssl. Is there one command, or one script, which can do that? You need that, basically daily. Is there one command to rebuild with the new libfoobar-1.17, and at most one more to test for basic functionality? You need that, too.

Spivak 3y ago
I mean you're not gonna like the answer but in real life when herding cats the answer is setting up an image scanner and renovate, and calling it a day. It's not like OS images are any better in this respect. I have bit my teeth long enough on software that depends on the base OS and not being able to infra upgrades. Bifurcating responsibility into app/platform is a breath of fresh air by comparison.
alexchantavy 3y ago
Yup this is a hard problem. Shameless plug to my blogpost on how we built something to do this: https://eng.lyft.com/vulnerability-management-at-lyft-enforcing-the-cascade-part-1-234d1561b994 https://eng.lyft.com/vulnerability-management-at-lyft-enforc...
bandrami 3y ago
Same problem with flatpak. How many versions of openssl are on my laptop? I have no idea!
salawat 3y ago
....I can tell you which ones are used by the current linker on the system in what order. ldconfig -p | grep libfoobar If you go and spread the linkers all over hither and yon and make it nigh impossible to get results out of them, or don't bother to keep track of where you put things... Welp. Can't help ya there. Shoulda been tracking that all along. Oh, excuse me... That'll only work for dynamically built things in the abscence of statically linked stuff or clever LD_PRELOAD shenanigans. Hope ya don't do that. Fact is. You're really never going to get away from keeping track of all the moving pieces. You're just shuffling complexity and layers of abstraction around.

teraflop 3y ago

Not in general, no. Docker image layers are just snapshots of the filesystem changes that result from build commands. But there is nothing that guarantees that the effects of those commands are reproducible. For example, it's incredibly common for Dockerfiles to contain commands like: RUN apt-get update && apt-get install -y foobar which means when you build the container, you get whatever version of foobar happens to currently be in the repository that your base image points to. So you can easily end up in a situation where rebuilding an image gives you different behavior, even though nothing in your Dockerfile or build context changed.

hinkley 3y ago
Even the Docker support team is confused by this. Most of why I stopped engaging is that they would reject a feature because it lead to dockerfiles they said were 'not reproducible', but were just as reproducible as many of the core features. Which is to say, not in the slightest. Every time you do something in a Dockerfile that involves pulling something from the network, you don't have reproducibility. Unless you pull something using a cryptographic hash (which I've never actually seen anyone do), the network can give you a different answer every time. The answer might not even be idempotent (calling it once changes the result of calling it again). My argument was the same as yours. Virtually every dockerfile has an 'update' call as the second or third layer, which means none of the repeatable layers afterward (like useradd or mkdir) are actually repeatable. You're building a castle on the sand. Docker images are reproducible. Docker builds are not. And that's okay, as long as everyone accepts that to be true. And if Docker contributors can't accept that then we are all truly fucked, until something else comes along that retreads the ideas.
- xvector 3y ago
  Sounds like Nix is what you're looking for.
  - evilduck 3y ago
    Which can conveniently produce Docker images.

deleted 3y ago

[deleted]