6 ms·

There are a lot of people who think subdeps should be able to just assert that they’re not vulnerable at all. It’s a popular idea, enough so it might be impleme

There are a lot of people who think subdeps should be able to just assert that they’re not vulnerable at all. It’s a popular idea, enough so it might be implemented in Node. Even though the originator of the idea was receptive to my many objections, the RFC authors were not. Here’s a thought that’s been living in my head rent free ever since: what if we don’t have transitive dependencies at all? What if we just… package them all up together? That sounds absurd, but we are almost all using a build system, we who are all have sourcemaps and dead code elimination baked in. We can standardize this. The package NPM delivers is already different from the published thing. We can define “this is how you reproduce my build”, and distributors can say “this is what is present in this package” with high confidence. Sourcemaps can deduplicate.

vladvasiliu 5y ago
I'm not very familiar with how Node dependencies work, but isn't this just reshuffling the same card deck? If package M is malicious, what difference does it make if it's required directly by your app, or through a sub-dependency chain, like A -> B -> C -> M? If you have an approach that allows you to vet all your direct dependencies, why wouldn't that allow you to vet sub-dependencies as well?
- eyelidlessness 5y ago
  What I’m proposing is specifically in cases where sub-dependencies may have a known vulnerability but that isn’t in any of the call paths of your direct dependency. It’s an alternative to the “audit assertions”[1] proposal, which I find problematic for reasons I discussed there before I bowed out. My idea is that you can be confident you’re not affected by a vulnerability in a dependency (at any depth), if that vulnerability is no longer in the code in the first place. It also reduces the surface area to vet in the first place. It’s highly likely many dependencies will be stripped down considerably, if not outright deduplicated or eliminated. The “npm installs thousands of dependencies” thing is a real problem, but it’s also partly because it’s installing stuff you’ll never actually execute in any way. You can pare down sub-dependencies with confidence, because you already know what code paths are hit by the parent dependency at packaging time. You can’t do that with direct dependencies until you go to package/deploy, because of course you may expand your usage of their APIs during development. 1: https://github.com/npm/rfcs/pull/422 https://github.com/npm/rfcs/pull/422
  - vladvasiliu 5y ago
    This sounds like a good idea to me. My shallow understating of the Node ecosystem makes me think of tree-shaking. Is that something else? I was under the impression that it allowed not shipping pieces of the dependencies which are unused, which should help with limiting the surface area to vet. From my superficial understanding of how this works, couldn't the "build" phase take note which parts of dependencies are loaded, and then compare those with where the known vulnerabilities are? I realize some issues may be bigger or harder to narrow down, but it still looks like it would be a step in the right direction.
    - eyelidlessness 5y ago
      You’re spot on in your understanding of the idea. I think it could go further than the state of the art tree shaking, by adding deduplication of unchanged code between different versions of the same package. This would mostly be a benefit for shrinking installed dependencies, but that helps make the idea of actually vetting/auditing them a lot more plausible too. And it could help make dealing with semver-incompatible versions more flexible (i.e. if the remaining code is identical, tada now it's semver-compatible because it’s the same code!), somewhat mitigating a common audit resolution blocker. All of this of course would need some sort of a manifest detailing which code comes from which location(s) in which package(s), and (at least optionally) a human-friendly file system structure that users can actually audit/vet. If I didn’t already have a backlog of side projects a mile long, I’d probably be working on this already, because there’s a clear need for improvement in this area. But none of it feels outlandishly hard to pull off if anyone wanted to run with the idea.

vladvasiliu 5y ago

I'm not very familiar with how Node dependencies work, but isn't this just reshuffling the same card deck? If package M is malicious, what difference does it make if it's required directly by your app, or through a sub-dependency chain, like A -> B -> C -> M? If you have an approach that allows you to vet all your direct dependencies, why wouldn't that allow you to vet sub-dependencies as well?

eyelidlessness 5y ago
What I’m proposing is specifically in cases where sub-dependencies may have a known vulnerability but that isn’t in any of the call paths of your direct dependency. It’s an alternative to the “audit assertions”[1] proposal, which I find problematic for reasons I discussed there before I bowed out. My idea is that you can be confident you’re not affected by a vulnerability in a dependency (at any depth), if that vulnerability is no longer in the code in the first place. It also reduces the surface area to vet in the first place. It’s highly likely many dependencies will be stripped down considerably, if not outright deduplicated or eliminated. The “npm installs thousands of dependencies” thing is a real problem, but it’s also partly because it’s installing stuff you’ll never actually execute in any way. You can pare down sub-dependencies with confidence, because you already know what code paths are hit by the parent dependency at packaging time. You can’t do that with direct dependencies until you go to package/deploy, because of course you may expand your usage of their APIs during development. 1: https://github.com/npm/rfcs/pull/422 https://github.com/npm/rfcs/pull/422
- vladvasiliu 5y ago
  This sounds like a good idea to me. My shallow understating of the Node ecosystem makes me think of tree-shaking. Is that something else? I was under the impression that it allowed not shipping pieces of the dependencies which are unused, which should help with limiting the surface area to vet. From my superficial understanding of how this works, couldn't the "build" phase take note which parts of dependencies are loaded, and then compare those with where the known vulnerabilities are? I realize some issues may be bigger or harder to narrow down, but it still looks like it would be a step in the right direction.
  - eyelidlessness 5y ago
    You’re spot on in your understanding of the idea. I think it could go further than the state of the art tree shaking, by adding deduplication of unchanged code between different versions of the same package. This would mostly be a benefit for shrinking installed dependencies, but that helps make the idea of actually vetting/auditing them a lot more plausible too. And it could help make dealing with semver-incompatible versions more flexible (i.e. if the remaining code is identical, tada now it's semver-compatible because it’s the same code!), somewhat mitigating a common audit resolution blocker. All of this of course would need some sort of a manifest detailing which code comes from which location(s) in which package(s), and (at least optionally) a human-friendly file system structure that users can actually audit/vet. If I didn’t already have a backlog of side projects a mile long, I’d probably be working on this already, because there’s a clear need for improvement in this area. But none of it feels outlandishly hard to pull off if anyone wanted to run with the idea.