4 ms·

Why invest so much time and money in a feature that prevents such a small percentage of data breaches that it's not even categorized on the 2024 Verizon Data Br

by cyrnel 2y ago

Why invest so much time and money in a feature that prevents such a small percentage of data breaches that it's not even categorized on the 2024 Verizon Data Breach Investigations Report? The vast majority of breaches are caused by credential theft, phishing, and exploiting vulnerabilities. It doesn't matter that you can cryptographically verify that a package came from a given commit if that commit has accidentally-vulnerable code, or someone just gets phished.

some_furry 2y ago
Because Verizon's report, while a good read, isn't the end-all-be-all of threat intelligence. https://www.wired.com/story/notpetya-cyberattack-ukraine-russia-code-crashed-the-world/ https://www.wired.com/story/notpetya-cyberattack-ukraine-rus... https://krebsonsecurity.com/2020/12/u-s-treasury-commerce-depts-hacked-through-solarwinds-compromise/ https://krebsonsecurity.com/2020/12/u-s-treasury-commerce-de... Software supply chain attacks are rare, but when they happen, they're usually high-impact ordeals.
- MattPalmer1086 2y ago
  Correct, software supply chain attacks may be rarer than a simple SQL injection attack, but they are much harder to detect and defend against, and can have very high impact. Where I work (highly regulated financial utility), we would absolutely love to have signed attestations for the provenance of python packages. I also have personal experience of NotPetya; I produced the forensic analysis of how it compromised the whole of Maersk. Took less than 30 minutes to take out their entire global IT infrastructure, with a cost of about $400 million.
  - pastage 2y ago
    You need to know where these attestations were made, you need a TPM telling you that this was done on a patched untouched server, in an patched untouched VM, on a signed attested code base, with deploy scripts unchanged by anyone. We are not there as long as Jenkins and most build infrastructure is basically Remote exploit as a service it is also going to take a long time. While Petya like in Ukraine is harder the more attestations you add there is always a hole, especially when all of your infrastructure needs to run things from some proprietary vendor.
    - MattPalmer1086 2y ago
      Well, those would be nice too. But I'm happy to not let the perfect be the enemy of the good.
    - michaelt 2y ago
      The proposal is an attestation from Github, owned by Microsoft. They could instead use a cloud-based trusted execution environment on Azure, and get an attestation from some combination of Intel and Microsoft. Or they could use a TPM and get an attestation from Infineon that the Microsoft-signed shim loaded a kernel blessed by a Microsoft-approved Linux vendor such as Canonical. Seems to me that just trades one corporate attestation for another.
      - amiga386 2y ago
        And that attestation does not affirm that the device's /usr/bin/python3 was squeaky-clean and didn't Ken-Thompson the package build process. It just affirms which machine. The trust that the machine itself is not compromised is implicit; "oh, yes, we trust Microsoft not to mess up its GitHub Actions worker VMs" What would work better is if _everybody_ could run the same CI action to produce a release tarball, and it produced identical builds, and everyone doing this could publish their findings - we can federate a heterogenous network of verifiers, rather than trust a single source's attestation
tptacek 2y ago
The 2024 DBIR, for whatever it's worth, repeatedly mentions software supply chain attacks.
- cyrnel 2y ago
  They consider accidental vulnerabilities to be "supply chain attacks". They make no mention of build/packaging system attacks except for SolarWinds.
gklitz 2y ago
They already went through requirering 2FA for the most popular packages: https://blog.pypi.org/posts/2023-05-25-securing-pypi-with-2fa/ https://blog.pypi.org/posts/2023-05-25-securing-pypi-with-2f... This is just another step in increasing security. And of cause that is something you want to preferably do prior to breaches not only as a reaction.
- the_mitsuhiko 2y ago
  Because publishing goes with GitHub actions for the most part for attestations the attack vector is getting access to GitHub which might be easier at this point.
twothreeone 2y ago
Probably because they got a government contract under which they receive funding for 3-5 FTEs over 24-36 months in return for quarterly reports - and a tool like this makes the DARPA PM happy. They're one of those "Cyber Defense Contractors"..
- woodruffw 2y ago
  This has nothing to do with DARPA. As the linked post in the comments says, this work was funded by Google. Source: I wrote the contract for it.
  - twothreeone 2y ago
    Thanks, I appreciate the honesty.
darkamaul 2y ago
The fact that a security measure doesn't solve all or even most breaches doesn't mean it's not worth implementing. Supply chain attacks may be a smaller percentage of breaches, but they can have massive impact when they do occur (see SolarWinds). Security is all about layers - each measure raises the bar incrementally.
Cthulhu_ 2y ago
Why not? You're presenting a false dichotomy, the time spent on this security does not take away time spent on the other ones you mentioned, and ultimately all security measures should be taken.
- cyrnel 2y ago
  Security is about risk reduction; like creating an ordered list of measures that reduce the most risk for the lowest cost. Working on stuff outside of the top 99%+ of effective solutions deserves to be questioned. As an alternative, they could have been investing time in a form of capabilities-based security for packages like Deno has where not every pip package can spawn info stealer processes at install time or runtime. That would address both the build compromise attacks and actually-common attacks like vulnerability exploitation.
itsgrimetime 2y ago
> Why invest so much time and money in a feature that prevents such a small percentage of data breaches ... Because it's a tractable problem that these devs can solve - and just because they're working on this doesn't meant they (or others) aren't also working on the other things. > It doesn't matter that you can cryptographically verify that a package came from a given commit if ... Sure, but just because it doesn't solve every single problem doesn't mean it's not worthwhile
tzlander 2y ago
This method favors big corporations and provides further lock-in. Python only does what Microsoft/Instagram etc. demand. So you get suit-compatible catch phrases like "SBOM" (notice how free software has been deliberately degraded to "materials" in that acronym!). The corporations want to control open source, milk it, feed it to their LLMs, plagiarize it and so forth. And they pay enough "open" source developers who sell out the valuable parts that are usually written by other people. As you say, it's partly security theater because of the other attack vectors that are especially relevant in an organization that has no stringent procedures, no open discussion culture or commitment to correctness like e.g. FreeBSD.
rfoo 2y ago
> It doesn't matter that you can cryptographically verify that a package came from a given commit if that commit has accidentally-vulnerable code, or someone just gets phished. If that commit has accidentally-vulnerable code, or someone just gets phished and attacker added some malicious code to the repository with his creds, it is visible. However, if the supply chain was secretly compromised and the VCS repo was always clean, only the release contains malware, then good luck finding it out. We've all witnessed this earlier this year, in the xz accident, while the (encrypted) malicious code was presented in the source code repo as part of test data, the code to load and decrypt it only ever existed in release tarballs.

by cyrnel 2y ago

some_furry 2y ago

Because Verizon's report, while a good read, isn't the end-all-be-all of threat intelligence. https://www.wired.com/story/notpetya-cyberattack-ukraine-russia-code-crashed-the-world/ https://www.wired.com/story/notpetya-cyberattack-ukraine-rus... https://krebsonsecurity.com/2020/12/u-s-treasury-commerce-depts-hacked-through-solarwinds-compromise/ https://krebsonsecurity.com/2020/12/u-s-treasury-commerce-de... Software supply chain attacks are rare, but when they happen, they're usually high-impact ordeals.

MattPalmer1086 2y ago
Correct, software supply chain attacks may be rarer than a simple SQL injection attack, but they are much harder to detect and defend against, and can have very high impact. Where I work (highly regulated financial utility), we would absolutely love to have signed attestations for the provenance of python packages. I also have personal experience of NotPetya; I produced the forensic analysis of how it compromised the whole of Maersk. Took less than 30 minutes to take out their entire global IT infrastructure, with a cost of about $400 million.
- pastage 2y ago
  You need to know where these attestations were made, you need a TPM telling you that this was done on a patched untouched server, in an patched untouched VM, on a signed attested code base, with deploy scripts unchanged by anyone. We are not there as long as Jenkins and most build infrastructure is basically Remote exploit as a service it is also going to take a long time. While Petya like in Ukraine is harder the more attestations you add there is always a hole, especially when all of your infrastructure needs to run things from some proprietary vendor.
  - MattPalmer1086 2y ago
    Well, those would be nice too. But I'm happy to not let the perfect be the enemy of the good.
  - michaelt 2y ago
    The proposal is an attestation from Github, owned by Microsoft. They could instead use a cloud-based trusted execution environment on Azure, and get an attestation from some combination of Intel and Microsoft. Or they could use a TPM and get an attestation from Infineon that the Microsoft-signed shim loaded a kernel blessed by a Microsoft-approved Linux vendor such as Canonical. Seems to me that just trades one corporate attestation for another.
    - amiga386 2y ago
      And that attestation does not affirm that the device's /usr/bin/python3 was squeaky-clean and didn't Ken-Thompson the package build process. It just affirms which machine. The trust that the machine itself is not compromised is implicit; "oh, yes, we trust Microsoft not to mess up its GitHub Actions worker VMs" What would work better is if _everybody_ could run the same CI action to produce a release tarball, and it produced identical builds, and everyone doing this could publish their findings - we can federate a heterogenous network of verifiers, rather than trust a single source's attestation

tptacek 2y ago

The 2024 DBIR, for whatever it's worth, repeatedly mentions software supply chain attacks.

cyrnel 2y ago
They consider accidental vulnerabilities to be "supply chain attacks". They make no mention of build/packaging system attacks except for SolarWinds.

gklitz 2y ago

They already went through requirering 2FA for the most popular packages: https://blog.pypi.org/posts/2023-05-25-securing-pypi-with-2fa/ https://blog.pypi.org/posts/2023-05-25-securing-pypi-with-2f... This is just another step in increasing security. And of cause that is something you want to preferably do prior to breaches not only as a reaction.

the_mitsuhiko 2y ago
Because publishing goes with GitHub actions for the most part for attestations the attack vector is getting access to GitHub which might be easier at this point.

twothreeone 2y ago

Probably because they got a government contract under which they receive funding for 3-5 FTEs over 24-36 months in return for quarterly reports - and a tool like this makes the DARPA PM happy. They're one of those "Cyber Defense Contractors"..

woodruffw 2y ago
This has nothing to do with DARPA. As the linked post in the comments says, this work was funded by Google. Source: I wrote the contract for it.
- twothreeone 2y ago
  Thanks, I appreciate the honesty.

darkamaul 2y ago

The fact that a security measure doesn't solve all or even most breaches doesn't mean it's not worth implementing. Supply chain attacks may be a smaller percentage of breaches, but they can have massive impact when they do occur (see SolarWinds). Security is all about layers - each measure raises the bar incrementally.

Cthulhu_ 2y ago

Why not? You're presenting a false dichotomy, the time spent on this security does not take away time spent on the other ones you mentioned, and ultimately all security measures should be taken.

cyrnel 2y ago
Security is about risk reduction; like creating an ordered list of measures that reduce the most risk for the lowest cost. Working on stuff outside of the top 99%+ of effective solutions deserves to be questioned. As an alternative, they could have been investing time in a form of capabilities-based security for packages like Deno has where not every pip package can spawn info stealer processes at install time or runtime. That would address both the build compromise attacks and actually-common attacks like vulnerability exploitation.

itsgrimetime 2y ago

> Why invest so much time and money in a feature that prevents such a small percentage of data breaches ... Because it's a tractable problem that these devs can solve - and just because they're working on this doesn't meant they (or others) aren't also working on the other things. > It doesn't matter that you can cryptographically verify that a package came from a given commit if ... Sure, but just because it doesn't solve every single problem doesn't mean it's not worthwhile

tzlander 2y ago

This method favors big corporations and provides further lock-in. Python only does what Microsoft/Instagram etc. demand. So you get suit-compatible catch phrases like "SBOM" (notice how free software has been deliberately degraded to "materials" in that acronym!). The corporations want to control open source, milk it, feed it to their LLMs, plagiarize it and so forth. And they pay enough "open" source developers who sell out the valuable parts that are usually written by other people. As you say, it's partly security theater because of the other attack vectors that are especially relevant in an organization that has no stringent procedures, no open discussion culture or commitment to correctness like e.g. FreeBSD.

rfoo 2y ago

> It doesn't matter that you can cryptographically verify that a package came from a given commit if that commit has accidentally-vulnerable code, or someone just gets phished. If that commit has accidentally-vulnerable code, or someone just gets phished and attacker added some malicious code to the repository with his creds, it is visible. However, if the supply chain was secretly compromised and the VCS repo was always clean, only the release contains malware, then good luck finding it out. We've all witnessed this earlier this year, in the xz accident, while the (encrypted) malicious code was presented in the source code repo as part of test data, the code to load and decrypt it only ever existed in release tarballs.