9 ms·

Yes, huge poisoning target enhanced by the fact images/tags are not immutable, you really have no idea what you are fetching straight from dockerhub, one pull o

Yes, huge poisoning target enhanced by the fact images/tags are not immutable, you really have no idea what you are fetching straight from dockerhub, one pull of the same image/tag may be different to the next pull. Most people blindly fetch without verifying regardless with multiple images of varying quality for software packages.

elcomet 8y ago
How could one verify ?
- bayareanative 8y ago
  You can't. Not without end-to-end integrity with nonrepudiation. Checksums aren't anywhere near enough. But that's Docker.. security optional and run random, untrusted code from the internet.
  - jhasse 8y ago
    > Checksums aren't anywhere near enough. Why not?
    - toxik 8y ago
      A checksum’s typical use is to detect transmission errors. A cryptographically secure signature is what’s needed.
      - jhasse 8y ago
        If I hard-code the checksum, then the base image can't be tampered with at least.
        Robin_Message 8y ago
        You can tamper with data protected by checksums: they are not designed to be irreversible, just fast to calculate and good at detecting errors, not deliberate manipulations. Use proper cryptography and don't roll your own!
        jhasse 8y ago
        Wouldn't that mean you need to find a collision?
        MattPalmer1086 8y ago
        The issue is, how do you verify the checksum you are using is valid. If you obtain the checksum from the same place you get the image, then an attacker can simply calculate a new checksum for the malicious image and publish it too. I guess if you were really sure you had obtained a checksum prior to the service compromise, then that would give reasonable assurance the image was not tampered with.
        ithkuil 8y ago
        Checksums/fingerprints can help mitigating the problem of _changing_ images people already use. As you correctly point out they don't solve the problem of authenticated distribution. Assuming you have fetched a given image and captured its sha in a config file in your version control (e.g. a kubetnetes manifest), then whenever you deploy a container you are sure that you're not affected by exploits happening _after_ you saved the fingerprint.
        richardknop 8y ago
        You create the docker image on your local computer, create checksum and write it down / remember it. Then you just use this checksum when downloading the image from other computers to check it's the same one. This only works for images created and uploaded by you of course, for images created by other people it does not work.
        orisho 8y ago
        I think his point is that for some checksums it could be trivial (and for some, tools already exist). Checksums aren't designed for this, while on the other hand secure hashing is. As a result, authors of hashing algorithms often attempt to mathematically prove their strength and resistance to a collision.
      - gtsteve 8y ago
        It uses SHA-256 right? My understanding is that there isn't yet a workable collision attack on the SHA-2 family. Regardless, I think it's certainly an excellent hardening step.
        blattimwind 8y ago
        Infosec in 2019: The server I download code from telling me the hash of said code is "certainly an excellent hardening step".
        ithkuil 8y ago
        Can't the hash be verified by the client too?
        subway 8y ago
        Sure, but who tells the client what the correct hash is?
        krferriter 8y ago
        If you're the entity who created the image you can retain the original hash and verify it against the downloaded copies. But that kind of defeats the purpose of being able to download docker images across distributed hosts. They'd really need to be signatures attached to the images, not just hashes.
        gtsteve 8y ago
        Well I was approaching it from the point of view that you verify the image is correct and then you guarantee you'll always use that image and not some other version given that tags are mutable.
        darkarmani 8y ago
        Why do you need a collision? If you control the build, you control the sha-256 hash. But you can't sign it with a key that you don't have.
  - cyphar 8y ago
    And Docker has a signing system but it's only enabled for the official-library builds! So all user images are completely unsigned despite all of the discussions of how secure the notary project might be. And even if you hosted your own distribution and notary (like we now do for openSUSE and SUSE), you can't force Docker to check the signatures of all images from that server! Only docker.io/library/* has enforced image signing and the only other option is to globally enforce image signing which means "docker build" will result in unusable images out-of-the-box. If you look at something like OBS (the Open Build Service that openSUSE and SUSE use to provide packages as well as host user repos), the signing story is far better (and OBS was written a very long time ago). All packages are signed without exception, and each project gets it's own key that is managed and rotated by OBS. zypper will scream very loudly if you try to install a package that is unsigned or the key for a repo changes without the corresponding rollover setup. And keys are associated with projects so a valid rpm from a different project will also produce a warning and resolution prompt. That's how the Docker Hub should've been designed from the start. (Disclaimer: I work for SUSE, on containers funnily enough.)
  - JohnHaugeland 8y ago
    that's most modern programming languages too
  - bauerm97 8y ago
    The company I work for, Sylabs, is taking what I think to be a pretty great approach to solving this problem. Essentially we've introduced a container image format where the actual runtime filesystem can be cryptographically signed (you can read about that here: https://www.sylabs.io/2018/03/sif-containing-your-containers/ https://www.sylabs.io/2018/03/sif-containing-your-containers...). The Singularity container runtime we develop treats this concept of "end-to-end integrity" as a core philosophy. Our docker hub analogue, the container Library, is working to make cryptographic signing one of the fundamentals of a container workflow. We're also actively working on container image encryption, which I think will bump container integrity up a few notches.
rossmohax 8y ago
tag are not immutable, but images (manifests) are. Much like git commit vs branches/tags. That is why best practice is to resolve docker image tag into "@sha256:..." digest and pull that, instead of tag. It guarantees that image you are pulling stays byte to byte the same.

elcomet 8y ago

How could one verify ?

bayareanative 8y ago
You can't. Not without end-to-end integrity with nonrepudiation. Checksums aren't anywhere near enough. But that's Docker.. security optional and run random, untrusted code from the internet.
- jhasse 8y ago
  > Checksums aren't anywhere near enough. Why not?
  - toxik 8y ago
    A checksum’s typical use is to detect transmission errors. A cryptographically secure signature is what’s needed.
    - jhasse 8y ago
      If I hard-code the checksum, then the base image can't be tampered with at least.
      - Robin_Message 8y ago
        You can tamper with data protected by checksums: they are not designed to be irreversible, just fast to calculate and good at detecting errors, not deliberate manipulations. Use proper cryptography and don't roll your own!
        jhasse 8y ago
        Wouldn't that mean you need to find a collision?
        MattPalmer1086 8y ago
        The issue is, how do you verify the checksum you are using is valid. If you obtain the checksum from the same place you get the image, then an attacker can simply calculate a new checksum for the malicious image and publish it too. I guess if you were really sure you had obtained a checksum prior to the service compromise, then that would give reasonable assurance the image was not tampered with.
        ithkuil 8y ago
        Checksums/fingerprints can help mitigating the problem of _changing_ images people already use. As you correctly point out they don't solve the problem of authenticated distribution. Assuming you have fetched a given image and captured its sha in a config file in your version control (e.g. a kubetnetes manifest), then whenever you deploy a container you are sure that you're not affected by exploits happening _after_ you saved the fingerprint.
        richardknop 8y ago
        You create the docker image on your local computer, create checksum and write it down / remember it. Then you just use this checksum when downloading the image from other computers to check it's the same one. This only works for images created and uploaded by you of course, for images created by other people it does not work.
        orisho 8y ago
        I think his point is that for some checksums it could be trivial (and for some, tools already exist). Checksums aren't designed for this, while on the other hand secure hashing is. As a result, authors of hashing algorithms often attempt to mathematically prove their strength and resistance to a collision.
    - gtsteve 8y ago
      It uses SHA-256 right? My understanding is that there isn't yet a workable collision attack on the SHA-2 family. Regardless, I think it's certainly an excellent hardening step.
      - blattimwind 8y ago
        Infosec in 2019: The server I download code from telling me the hash of said code is "certainly an excellent hardening step".
        ithkuil 8y ago
        Can't the hash be verified by the client too?
        subway 8y ago
        Sure, but who tells the client what the correct hash is?
        krferriter 8y ago
        If you're the entity who created the image you can retain the original hash and verify it against the downloaded copies. But that kind of defeats the purpose of being able to download docker images across distributed hosts. They'd really need to be signatures attached to the images, not just hashes.
        gtsteve 8y ago
        Well I was approaching it from the point of view that you verify the image is correct and then you guarantee you'll always use that image and not some other version given that tags are mutable.
      - darkarmani 8y ago
        Why do you need a collision? If you control the build, you control the sha-256 hash. But you can't sign it with a key that you don't have.
- cyphar 8y ago
  And Docker has a signing system but it's only enabled for the official-library builds! So all user images are completely unsigned despite all of the discussions of how secure the notary project might be. And even if you hosted your own distribution and notary (like we now do for openSUSE and SUSE), you can't force Docker to check the signatures of all images from that server! Only docker.io/library/* has enforced image signing and the only other option is to globally enforce image signing which means "docker build" will result in unusable images out-of-the-box. If you look at something like OBS (the Open Build Service that openSUSE and SUSE use to provide packages as well as host user repos), the signing story is far better (and OBS was written a very long time ago). All packages are signed without exception, and each project gets it's own key that is managed and rotated by OBS. zypper will scream very loudly if you try to install a package that is unsigned or the key for a repo changes without the corresponding rollover setup. And keys are associated with projects so a valid rpm from a different project will also produce a warning and resolution prompt. That's how the Docker Hub should've been designed from the start. (Disclaimer: I work for SUSE, on containers funnily enough.)
- JohnHaugeland 8y ago
  that's most modern programming languages too
- bauerm97 8y ago
  The company I work for, Sylabs, is taking what I think to be a pretty great approach to solving this problem. Essentially we've introduced a container image format where the actual runtime filesystem can be cryptographically signed (you can read about that here: https://www.sylabs.io/2018/03/sif-containing-your-containers/ https://www.sylabs.io/2018/03/sif-containing-your-containers...). The Singularity container runtime we develop treats this concept of "end-to-end integrity" as a core philosophy. Our docker hub analogue, the container Library, is working to make cryptographic signing one of the fundamentals of a container workflow. We're also actively working on container image encryption, which I think will bump container integrity up a few notches.

rossmohax 8y ago

tag are not immutable, but images (manifests) are. Much like git commit vs branches/tags. That is why best practice is to resolve docker image tag into "@sha256:..." digest and pull that, instead of tag. It guarantees that image you are pulling stays byte to byte the same.