4 ms·

A solution I am working on for these sorts of software supply chain attacks uses transparency logs. Take a look here (warning, alpha): https://github.com/transp

A solution I am working on for these sorts of software supply chain attacks uses transparency logs. Take a look here (warning, alpha): https://github.com/transparencylog/btget https://github.com/transparencylog/btget Essentially the binary transparency log acts like a notary service. It appends the cryptographic digest of a URL to an append only log. The append only log cannot be rewound without detection by clients. And clients verify the contents they receive against the entry in the log. Two nice properties for the issues outlined in this post: 1. Hosting providers (or third parties) can add this sort of log without needing developers to do things like manage key material or 2FA tokens 2. It is complementary to good upload authentication or signing systems In fact, Go is using transparency logs for source libraries. I think similar systems should be used for binaries in all other language ecosystems as well. I would love to see a world where major source code distributions start running transparency log servers as part of their own file host integrity protections. Imagine maven, github, npm, etc all running these sorts of services for all uploads and using them by default in their clients. Users would have additional confidence that the v0.4.3 package they downloaded to their system two weeks ago is the same one their colleague got or the CI/CD system got.

ithkuil 6y ago
Some time ago I made a fun hack where I abuse the Go transparency log to store immutable url->fingerprint mappings of arbitrary urls: https://github.com/mkmik/getsum https://github.com/mkmik/getsum Example: $ getsum https://github.com/bazelbuild/bazel/releases/download/3.4.1/bazel-3.4.1-installer-linux-x86_64.sh 9808adad931ac652e8ff5022a74507c532250c2091d21d6aebc7064573669cc5 now, if somebody ever replaces that file on github, getsum will fail with an error. (I was too lazy to make getsum actually download the file if the checksum matches, but that should be trivial; for now just do "getsum $url && wget $url"). This works by creating synthetic Go modules that embed "facts" about URLs inside generated Go code, which gets then downloaded by getsum and parsed using the stdlib "go/parser" package. Since all Go modules are recorded in the Go "sumdb" public transparency log, once this module has been generated it cannot ever change. (Getsum uses a constant "version" for the synthetic packages, thus facts are immutable). I'm confident this works because it piggy-backs on the stable production transparent logs infrastructure of a high-profile project managed by a high-profile company. On the other hand the Go transparency logs admins could ban my "getsum.pub" domain if the generated traffic bothers them. This was for me just a quick way to show to some colleagues the potentials of transparency logs etc, and of course to have fun.
- dependenttypes 6y ago
  > (I was too lazy to make getsum actually download the file if the checksum matches, but that should be trivial; for now just do "getsum $url && wget $url"). Please don't. Instead check the hash of the local file. A malicious server could serve a different file for getsum and for wget (bonus point: you won't have to download it twice)
  - ithkuil 6y ago
    yes that's indeed a problem that needs to be dealt with. That said, technically the getsum client doesn't download the file; it only fetches "$url.sha256" if present or a "SHA256SUMS" in the same directory and compares that with the one stored in the transparency log. It follows a common pattern on release sites (such as major projects on github). In other words getsum only ensures that the checksum file itself is immutable, and the checksum file then is used to ensure that bigger file also hasn't changed. The reason for that lies in the fact that getsum has a server side component (hosted on getsum.pub) that serves the Go modules fetched by the Go sumdb. Thus, the correct instructions are: $ (echo -n $(getsum https://github.com/bazelbuild/bazel/releases/download/3.4.1/bazel-3.4.1-installer-linux-x86_64.sh); echo " bazel-3.4.1-installer-linux-x86_64.sh") > bazel-3.4.1-installer-linux-x86_64.sh.sha256 && wget https://github.com/bazelbuild/bazel/releases/download/3.4.1/bazel-3.4.1-installer-linux-x86_64.sh && sha256sum -c bazel-3.4.1-installer-linux-x86_64.sh.sha256 Big file downloaders tend to be more complicated than expected; progress bars, resuming downloads etc etc. Perhaps getsum instead of downloading the file it could verify the file? E.g.: $ wget https://github.com/bazelbuild/bazel/releases/download/3.4.1/bazel-3.4.1-installer-linux-x86_64.sh && getsum -c bazel-3.4.1-installer-linux-x86_64.sh https://github.com/bazelbuild/bazel/releases/download/3.4.1/bazel-3.4.1-installer-linux-x86_64.sh EDIT: just implemented the "-c" flag described above

ithkuil 6y ago

Some time ago I made a fun hack where I abuse the Go transparency log to store immutable url->fingerprint mappings of arbitrary urls: https://github.com/mkmik/getsum https://github.com/mkmik/getsum Example: $ getsum https://github.com/bazelbuild/bazel/releases/download/3.4.1/bazel-3.4.1-installer-linux-x86_64.sh 9808adad931ac652e8ff5022a74507c532250c2091d21d6aebc7064573669cc5 now, if somebody ever replaces that file on github, getsum will fail with an error. (I was too lazy to make getsum actually download the file if the checksum matches, but that should be trivial; for now just do "getsum $url && wget $url"). This works by creating synthetic Go modules that embed "facts" about URLs inside generated Go code, which gets then downloaded by getsum and parsed using the stdlib "go/parser" package. Since all Go modules are recorded in the Go "sumdb" public transparency log, once this module has been generated it cannot ever change. (Getsum uses a constant "version" for the synthetic packages, thus facts are immutable). I'm confident this works because it piggy-backs on the stable production transparent logs infrastructure of a high-profile project managed by a high-profile company. On the other hand the Go transparency logs admins could ban my "getsum.pub" domain if the generated traffic bothers them. This was for me just a quick way to show to some colleagues the potentials of transparency logs etc, and of course to have fun.

dependenttypes 6y ago
> (I was too lazy to make getsum actually download the file if the checksum matches, but that should be trivial; for now just do "getsum $url && wget $url"). Please don't. Instead check the hash of the local file. A malicious server could serve a different file for getsum and for wget (bonus point: you won't have to download it twice)
- ithkuil 6y ago
  yes that's indeed a problem that needs to be dealt with. That said, technically the getsum client doesn't download the file; it only fetches "$url.sha256" if present or a "SHA256SUMS" in the same directory and compares that with the one stored in the transparency log. It follows a common pattern on release sites (such as major projects on github). In other words getsum only ensures that the checksum file itself is immutable, and the checksum file then is used to ensure that bigger file also hasn't changed. The reason for that lies in the fact that getsum has a server side component (hosted on getsum.pub) that serves the Go modules fetched by the Go sumdb. Thus, the correct instructions are: $ (echo -n $(getsum https://github.com/bazelbuild/bazel/releases/download/3.4.1/bazel-3.4.1-installer-linux-x86_64.sh); echo " bazel-3.4.1-installer-linux-x86_64.sh") > bazel-3.4.1-installer-linux-x86_64.sh.sha256 && wget https://github.com/bazelbuild/bazel/releases/download/3.4.1/bazel-3.4.1-installer-linux-x86_64.sh && sha256sum -c bazel-3.4.1-installer-linux-x86_64.sh.sha256 Big file downloaders tend to be more complicated than expected; progress bars, resuming downloads etc etc. Perhaps getsum instead of downloading the file it could verify the file? E.g.: $ wget https://github.com/bazelbuild/bazel/releases/download/3.4.1/bazel-3.4.1-installer-linux-x86_64.sh && getsum -c bazel-3.4.1-installer-linux-x86_64.sh https://github.com/bazelbuild/bazel/releases/download/3.4.1/bazel-3.4.1-installer-linux-x86_64.sh EDIT: just implemented the "-c" flag described above