6 ms·

The bummer about lots of supply chain work is that it does not address the attacks we see in the wild like xz where malicious code was added at the source, and

The bummer about lots of supply chain work is that it does not address the attacks we see in the wild like xz where malicious code was added at the source, and attested all the way through. There are gains to be had through these approaches, like inventory, but nobody has a good approach to stopping malicious code entering the ecosystem through the front door and attackers find this much easier than tampering with artifacts after the fact.

yencabulator 1y ago
I think a big part of the push is just being able to easily & conclusively answer "are we vulnerable or not" when a new attack is discovered. Exhaustive inventory already is huge.
- tough 1y ago
  i read somewhere go has a great package for this that checks statically typed usage of the vuln specific functions not whole package deps
  - yencabulator 1y ago
    https://pkg.go.dev/golang.org/x/vuln/cmd/govulncheck https://pkg.go.dev/golang.org/x/vuln/cmd/govulncheck
    - tough 1y ago
      ty ty exactly what I was thinking does something like this exist for other languages like rust, python or js?
      - yencabulator 1y ago
        I don't think the Rust ecosystem has that at this time. They're annotating the vulnerabilities with affected functions, but as far as I know nobody's written the static analysis side of it. https://github.com/rustsec/rustsec/issues/21 https://github.com/rustsec/rustsec/issues/21 Python and JS might be so dynamic that such static analysis just isn't as useful.
      - dwattttt 1y ago
        For Rust, the advisory database cargo-audit uses (https://github.com/RustSec/advisory-db/ https://github.com/RustSec/advisory-db/) does track which functions are affected by a cve (if provided). I'm not sure if the tool uses them though.
XiZhao 1y ago
I run a sw supply chain company (fossa.com) -- agree that there's a lot of low hanging gains like inventory still around. There is a shocking amount of very basic but invisible surface area that leads to downstream attack vectors. From a company's PoV -- I think you'd have to just assume all 3rd party code is popped and install some kind of control step given that assumption. I like the idea of reviewing all 3rd party code as if its your own which is now possible with some scalable code review tools.
- nyrikki 1y ago
  Those projects seem to devolve into a boil the ocean style projects and tend to be viewed as intractable and thus ignorable. In the days everything was http I use to set a proxy variable and have the proxy save all downloaded assets to compair later, today I would probably blacklist the public CAs and do an intercept, just for the data of what is grabbing what. Fedramp was defunded and is moving forward with a GOA style agile model. If you have the resources I would highly encourage you to participate in conversations. The timelines are tight and they are trying to move fast, so look into their GitHub discussions and see if you can move it forward. There is a chance to make real changes but they need feedback now. https://github.com/FedRAMP https://github.com/FedRAMP
- beardedwizard 1y ago
  +1, I think you have to assume owned as well and start defending from there. Companies like edera are betting on that, but sandbox isn't panacea, you really need some way to know expected behavior.
kuruczgy 1y ago
Actually this is not quite true, in the xz hack part of the malicious code was in generated files only present in the release tarball. When I personally package stuff using Nix, I go out of my way to build everything from source as much as possible. E.g. if some repo contains checked in generated files, I prefer to delete and regenerate them. It's nice that Nix makes adding extra build steps like this easy. I think most of the time the motivation for having generated files in repos (or release tarballs) is the limitations of various build systems.
- throwawayqqq11 1y ago
  Your preference to compile your backdoors does not really fix the problem of malicious code supply. I have this vague idea to fingerprint the relevant AST down to all syscalls and store it in a lock file to have a better chance of detection. But this isnt a true fix either.
  - kuruczgy 1y ago
    Yes you are right, what I am proposing is not a solution by itself, it's just a way to be reasonably confident that _if you audit the code_, that's going to be the actual logic running on your computer. (I don't get the value of your AST checksumming idea over just checksumming the source text, which is what almost all distro packages do. I think the number of changes that change the code but not the AST are negligible. If the code (and AST) is changed, you have to audit the diff no matter what.) The more interesting question that does not have a single good answer is how to do the auditing. In almost all cases right now the only metric you have is "how much you trust upstream", in very few cases is actually reading through all the code and changes viable. I like to look at how upstream does their auditing of changes, e.g. how they do code review and how clean is their VCS history (so that _if_ you discover something fishy in the code, there is a clean audit trail of where that piece of code came from).
    - vasco 1y ago
      > it's just a way to be reasonably confident that _if you audit the code_ Why do we pretend this is easy many times in conversation about dependencies? It's as if security bugs in dependencies were calling out at us, like a house inspector looking at a huge hole on the floor of the house. But it's not like that at all, most people would inspect 99.9% of CVEs and read the vulnerable code and accept it. As did the reviewers in the open-source project, who know that codebase much more than someone who's adding a dependency because they want to do X faster. And they missed it or the CVE wouldn't be there, but somehow a random dev looking at it for the first time will find it? In fact, if to use dependencies I have to read and understand the code and validate it, the number of dependencies I'd use would go to zero. And many things I would be locked out of doing, because I'm too dumb to understand them, so I can't audit the code, which means I'm definitely too dumb to replicate the library myself. Asking people to audit the code in hopes of finding a security bug is a big crapshoot. The industry needs better tools.
TZubiri 1y ago
When you have so many dependencies that you need to create complex systems to manage and "secure" the dependencies. The problem is that you have too many dependencies, you are relying on too many volunteer work, and you are demanding too many features while paying for too little. The professional solution is to PAY for your Operating System, and rely on them to secure it. Whether it be to Microsoft or to Red Hat. You KNOW it's the right thing to do, and this is overintellectualizing your need to have a gratis operating system and charge non-gratis prices to your clients in turn.
- deleted 1y ago
  [deleted]
- lmm 1y ago
  How does that solve the problem? Both Microsoft and IBM/Red Hat have shipped backdoored code in the past and will no doubt do so again. At most you might be able to sue them for a refund of what you paid them, at which point you're no better off than if you'd used a free system from the start.
- justinrubek 1y ago
  I couldn't disagree more with everything you've said.

yencabulator 1y ago

I think a big part of the push is just being able to easily & conclusively answer "are we vulnerable or not" when a new attack is discovered. Exhaustive inventory already is huge.

tough 1y ago
i read somewhere go has a great package for this that checks statically typed usage of the vuln specific functions not whole package deps
- yencabulator 1y ago
  https://pkg.go.dev/golang.org/x/vuln/cmd/govulncheck https://pkg.go.dev/golang.org/x/vuln/cmd/govulncheck
  - tough 1y ago
    ty ty exactly what I was thinking does something like this exist for other languages like rust, python or js?
    - yencabulator 1y ago
      I don't think the Rust ecosystem has that at this time. They're annotating the vulnerabilities with affected functions, but as far as I know nobody's written the static analysis side of it. https://github.com/rustsec/rustsec/issues/21 https://github.com/rustsec/rustsec/issues/21 Python and JS might be so dynamic that such static analysis just isn't as useful.
    - dwattttt 1y ago
      For Rust, the advisory database cargo-audit uses (https://github.com/RustSec/advisory-db/ https://github.com/RustSec/advisory-db/) does track which functions are affected by a cve (if provided). I'm not sure if the tool uses them though.

XiZhao 1y ago

I run a sw supply chain company (fossa.com) -- agree that there's a lot of low hanging gains like inventory still around. There is a shocking amount of very basic but invisible surface area that leads to downstream attack vectors. From a company's PoV -- I think you'd have to just assume all 3rd party code is popped and install some kind of control step given that assumption. I like the idea of reviewing all 3rd party code as if its your own which is now possible with some scalable code review tools.

nyrikki 1y ago
Those projects seem to devolve into a boil the ocean style projects and tend to be viewed as intractable and thus ignorable. In the days everything was http I use to set a proxy variable and have the proxy save all downloaded assets to compair later, today I would probably blacklist the public CAs and do an intercept, just for the data of what is grabbing what. Fedramp was defunded and is moving forward with a GOA style agile model. If you have the resources I would highly encourage you to participate in conversations. The timelines are tight and they are trying to move fast, so look into their GitHub discussions and see if you can move it forward. There is a chance to make real changes but they need feedback now. https://github.com/FedRAMP https://github.com/FedRAMP
beardedwizard 1y ago
+1, I think you have to assume owned as well and start defending from there. Companies like edera are betting on that, but sandbox isn't panacea, you really need some way to know expected behavior.

kuruczgy 1y ago

Actually this is not quite true, in the xz hack part of the malicious code was in generated files only present in the release tarball. When I personally package stuff using Nix, I go out of my way to build everything from source as much as possible. E.g. if some repo contains checked in generated files, I prefer to delete and regenerate them. It's nice that Nix makes adding extra build steps like this easy. I think most of the time the motivation for having generated files in repos (or release tarballs) is the limitations of various build systems.

throwawayqqq11 1y ago
Your preference to compile your backdoors does not really fix the problem of malicious code supply. I have this vague idea to fingerprint the relevant AST down to all syscalls and store it in a lock file to have a better chance of detection. But this isnt a true fix either.
- kuruczgy 1y ago
  Yes you are right, what I am proposing is not a solution by itself, it's just a way to be reasonably confident that _if you audit the code_, that's going to be the actual logic running on your computer. (I don't get the value of your AST checksumming idea over just checksumming the source text, which is what almost all distro packages do. I think the number of changes that change the code but not the AST are negligible. If the code (and AST) is changed, you have to audit the diff no matter what.) The more interesting question that does not have a single good answer is how to do the auditing. In almost all cases right now the only metric you have is "how much you trust upstream", in very few cases is actually reading through all the code and changes viable. I like to look at how upstream does their auditing of changes, e.g. how they do code review and how clean is their VCS history (so that _if_ you discover something fishy in the code, there is a clean audit trail of where that piece of code came from).
  - vasco 1y ago
    > it's just a way to be reasonably confident that _if you audit the code_ Why do we pretend this is easy many times in conversation about dependencies? It's as if security bugs in dependencies were calling out at us, like a house inspector looking at a huge hole on the floor of the house. But it's not like that at all, most people would inspect 99.9% of CVEs and read the vulnerable code and accept it. As did the reviewers in the open-source project, who know that codebase much more than someone who's adding a dependency because they want to do X faster. And they missed it or the CVE wouldn't be there, but somehow a random dev looking at it for the first time will find it? In fact, if to use dependencies I have to read and understand the code and validate it, the number of dependencies I'd use would go to zero. And many things I would be locked out of doing, because I'm too dumb to understand them, so I can't audit the code, which means I'm definitely too dumb to replicate the library myself. Asking people to audit the code in hopes of finding a security bug is a big crapshoot. The industry needs better tools.

TZubiri 1y ago

When you have so many dependencies that you need to create complex systems to manage and "secure" the dependencies. The problem is that you have too many dependencies, you are relying on too many volunteer work, and you are demanding too many features while paying for too little. The professional solution is to PAY for your Operating System, and rely on them to secure it. Whether it be to Microsoft or to Red Hat. You KNOW it's the right thing to do, and this is overintellectualizing your need to have a gratis operating system and charge non-gratis prices to your clients in turn.

deleted 1y ago
[deleted]
lmm 1y ago
How does that solve the problem? Both Microsoft and IBM/Red Hat have shipped backdoored code in the past and will no doubt do so again. At most you might be able to sue them for a refund of what you paid them, at which point you're no better off than if you'd used a free system from the start.
justinrubek 1y ago
I couldn't disagree more with everything you've said.