7 ms·

People don't directly import thousands of modules. It's actually a lot closer to your "10 stable dependencies". But those dependencies have dependencies that ha

by vosper 5y ago

People don't directly import thousands of modules. It's actually a lot closer to your "10 stable dependencies". But those dependencies have dependencies that have dependencies. It's a little hard to point the finger at application developers here, IMO.

pdonis 5y ago
> It's a little hard to point the finger at application developers here, IMO. I disagree. Any application developer who seriously thinks that they only have 10 dependencies if they're only importing directly 10 dependencies should not be an application developer in the first place.
- IggleSniggle 5y ago
  You sure about that? Even if you’re writing just for a vetted distribution of an OS, and you write code with zero explicit dependencies, you still have much more than zero dependencies. It’s turtles all the way down. The key is to have an entire ecosystem that you can, to some degree more or less, trust.
  - marcus_holmes 5y ago
    No. we've been shouting warnings for years. There have been dozens, if not hundreds of threads on HN alone warning of supply-chain security threats. At this point if you're not actively auditing your dependencies, and reducing all of them where you can, then you're on the wrong side of history and going down with the Titanic. The frank truth is that including a dependency is, and always has been, giving a random person from the internet commit privileges to prod. The fact that "everyone else did it" doesn't make it less stupid.
    - erulabs 5y ago
      > The frank truth is that including a dependency is, and always has been, giving a random person from the internet commit privileges to prod I mean, no. This is hyperbole at best and just wrong at median. A system of relative trust has worked very well for a very long time - Linus doesn’t have root access to all our systems, even if we don’t have to read every line of code.
      - chha 5y ago
        Linus doesn't have root access to our systems for several reasons. One of them is the fact that we get the actual source code, and not just a compiled blob doing "something". Another is the fact that they have at least some level of reviews wrt who can commit code, although this isn't perfect as the case with the University of Minnesota proved. Npm on the other hand is much, much worse. Anyone can publish anything they want, and they can point to any random source code repository claiming that this is the source. If we look at how often vulnerable packages are discovered in eg. npm, I'd argue that the current level of trust and quality aren't sustainable, partly due to the potentially huge number of direct and transitive dependencies a project may have. Unless you start to review the actual component you have no way to verify this, and unlike the Linux kernel there is no promise that anyone has ever reviewed the package you download. You can of course add free tools such as the OWASP Dependency Check, but these will typically lag a bit behind as they rely on published vulnerabilities. Other tools such as the Sonatype Nexus platform is more proactive, but can be expensive.
        elevader 5y ago
        Maybe this is arguing semantics but unless you run something like Gentoo you will most likely get the linux kernel as a binary blob contained in a package your distribution provides. There isn't really any guarantee that this will actually contain untampered linux kernel sources (and in case of something like RHEL it most likely doesn't because of backports) unless you audit it, which most people won't do (and maybe can't do). So, in princpile at least, this isn't really that much better than the node_modules situation. Security and trust are hard issues and piling on 100s of random js dependencies sure doesn't help but you either build everything yourself or you need to trust somebody at some point.
        ptx 5y ago
        You do need to trust somebody (such as your Linux distribution of choice) but with NPM you're trusting thousands of somebodies and your system's security depends directly on all of them being secure and trustworthy.
    - waingake 5y ago
      And should we all start rolling our own crypto now to avoid dependencies? In most cases a stable library is going to be much more secure than a custom implementation of `x`. Everything has trade-offs. What's stupid is dogma.
      - djbusby 5y ago
        I know you're being hyperbolic and I also want to add that for crypto you should just use libsodium. The algos and the code are very good. And lots of very smart folk have given it a lot of review. And its API is very nice.
        IggleSniggle 5y ago
        When you say this, do you mean actual C libsodium? Because surely you don’t mean that I, a js developer, should need to figure out how to wrap this .h file thingy to get it to work in js when there’s SIX third-party libsodium implementations/wrappers/projects sitting right there listed on the libsodium website? /s
    - FranksTV 5y ago
      Or perhaps, the sky is not falling.
    - jimmont 5y ago
      This. And npm isn't the only instance. (Appreciate the voice in the wilderness Marcus.)
  - pdonis 5y ago
    > Even if you’re writing just for a vetted distribution of an OS, and you write code with zero explicit dependencies, you still have much more than zero dependencies. Sure, the entire OS is a dependency. Nothing I said contradicts that. And yes, every application developer should be aware of what they are depending on when they write software for a particular OS. > The key is to have an entire ecosystem that you can, to some degree more or less, trust. You don't necessarily need to trust an entire ecosystem, but yes, every dependency you have is a matter of trust on your part; you are trusting the dependency to work the way you need it to work and not to introduce vulnerabilities that you aren't aware of and can't deal with. Which is why you need to be explicitly aware of every dependency you have, not just the ones you directly import.
    - IggleSniggle 5y ago
      I am actually not sure if this is possible, while also accepting security updates etc from my OS distributor? How do you literally personally vet every line of code that gets run directly AND indirectly by your application, and still have time to write an application? I’m okay with saying, “I trust RHEL to be roughly ok, just understand the model and how to use it, and keep my ear to the ground for the experts in case something comes up.” At the level of npm, I feel roughly the same about React. I don’t trust it quite as much, but I’m also not going to read every code change. I’ll read a CHANGELOG, sure, and spelunk through the code from time to time, but that’s not really the same. I’ll probably check out their direct dependencies the first time, but that’s it. I actually don’t know how you could call yourself an application developer in most ecosystems and know every single dependency you actually have all the way down, soup to nuts. Heck, there are dependencies that I accept so that my code will run on machines that I have no special knowledge of, not just my own familiar architecture. I accept them because I want to work on the details of my application and have it be useful on more than just my own machine. Edit for clarity: I agree with almost everything you’re suggesting as sensible. Just not with your conclusion: that you’re not a “real” application developer if you don’t know all of your dependencies
      - pdonis 5y ago
        > I am actually not sure if this is possible, while also accepting security updates etc from my OS distributor? Accepting the OS as a dependency includes the security updates from the OS, sure. > How do you literally personally vet every line of code Ah, I see, you think "understanding the dependency" requires vetting every line of code. That's not what I meant. What I meant is, if you use library A, and library A depends on libraries B, C, and D, and those libraries in turn depend on libraries E, F, G, H, I, etc. etc., then you don't just need to be aware that you depend on library A, because that's the only one you're directly importing. You need to be aware of all the dependencies, all the way down. You might not personally vet every line of code in every one of them, but you need to be aware that you're using them and you need to be aware of how trustworthy they are, so you can judge whether it's really worth having them and exposing your application to the risks of using them. > I’ll probably check out their direct dependencies the first time, but that’s it. So if they introduce a new dependency, you don't care? You should. That's the kind of thing I'm talking about. Again, you might not go and vet every line of code in the new dependency, but you need to be aware that it's there and how risky it is. > I actually don’t know how you could call yourself an application developer in most ecosystems and know every single dependency you actually have all the way down, soup to nuts. If you're developing using open source code, information about what dependencies a given library has is easily discoverable. If you're developing for a proprietary system, things might be different.
  - deleted 5y ago
    [deleted]
nitwit005 5y ago
I mean, you say that, but the practice of pulling in so many dependencies is fairly recent. It wasn't even possible for most projects before everyone had fast internet.
dpweb 5y ago
Some of the comments in this thread are wild. Huge dependency trees are bad pattern, plain and simple. The problem isn’t only ridiculous amounts of untrusted code, but thousands of new developers of the last 10 years who think this is the way to write reliable code. Never acknowledged the risks of having everyone write your code for you, and overestimate how unique and interesting their apps are. If you must participate in this madness, static analysis tools exist to scan your 10000 dependencies, taking security seriously is the issue.
- deleted 5y ago
  [deleted]
- noway421 5y ago
  Most of those dependencies have well defined, stable api. They use or at least try to follow semver. And you're probably only hitting about 10% of your dependencies on the critical path you're using, meaning that a lot of potentially vulnerable code is never executed. I get the supply chain attacks. I get that you have a tree of untrusted javascript code that you're executing in your app, on install, on build and in runtime. But there's also Snyk and Dependabot which issue you alerts when your dependency tree has published CVEs. We can talk about alert fatigue, but to be honest, I feel more secure with my node_modules folder than I do with my operating system and plethora of DLLs it loads. I don't wanna turn this into a whataboutism argument, but at some point you gotta get to work, write some code and depend on some libraries other people have written.
  - laurent92 5y ago
    > Snyk and Dependabot Wait, I’m not safe using “npm audit”?
  - grey-area 5y ago
    Semver will not save you.
  - ptx 5y ago
    > And you're probably only hitting about 10% of your dependencies on the critical path you're using, meaning that a lot of potentially vulnerable code is never executed. If a dependency has been compromised it doesn't matter if its code is actually used, since it can include a lifecycle script that's executed at install-time, which was apparently the mechanism for the recent ua-parser-js exploit.
grey-area 5y ago
This is the direct result of the culture of tiny dependencies in JS and some other languages, but not all ecosystems are like this. If you choose to use node, this is where you end up, but it was a choice. Many languages have a decent standard library which covers most of the bases, so it’s possible to have a very restricted set of dependencies.
- jschrf 5y ago
  Unfortunately for frontend and the Node ecosystem, it's too late to try and put the toothpaste back in the tube. Hopefully Deno helps with this pain point.

by vosper 5y ago

pdonis 5y ago

> It's a little hard to point the finger at application developers here, IMO. I disagree. Any application developer who seriously thinks that they only have 10 dependencies if they're only importing directly 10 dependencies should not be an application developer in the first place.

IggleSniggle 5y ago
You sure about that? Even if you’re writing just for a vetted distribution of an OS, and you write code with zero explicit dependencies, you still have much more than zero dependencies. It’s turtles all the way down. The key is to have an entire ecosystem that you can, to some degree more or less, trust.
- marcus_holmes 5y ago
  No. we've been shouting warnings for years. There have been dozens, if not hundreds of threads on HN alone warning of supply-chain security threats. At this point if you're not actively auditing your dependencies, and reducing all of them where you can, then you're on the wrong side of history and going down with the Titanic. The frank truth is that including a dependency is, and always has been, giving a random person from the internet commit privileges to prod. The fact that "everyone else did it" doesn't make it less stupid.
  - erulabs 5y ago
    > The frank truth is that including a dependency is, and always has been, giving a random person from the internet commit privileges to prod I mean, no. This is hyperbole at best and just wrong at median. A system of relative trust has worked very well for a very long time - Linus doesn’t have root access to all our systems, even if we don’t have to read every line of code.
    - chha 5y ago
      Linus doesn't have root access to our systems for several reasons. One of them is the fact that we get the actual source code, and not just a compiled blob doing "something". Another is the fact that they have at least some level of reviews wrt who can commit code, although this isn't perfect as the case with the University of Minnesota proved. Npm on the other hand is much, much worse. Anyone can publish anything they want, and they can point to any random source code repository claiming that this is the source. If we look at how often vulnerable packages are discovered in eg. npm, I'd argue that the current level of trust and quality aren't sustainable, partly due to the potentially huge number of direct and transitive dependencies a project may have. Unless you start to review the actual component you have no way to verify this, and unlike the Linux kernel there is no promise that anyone has ever reviewed the package you download. You can of course add free tools such as the OWASP Dependency Check, but these will typically lag a bit behind as they rely on published vulnerabilities. Other tools such as the Sonatype Nexus platform is more proactive, but can be expensive.
      - elevader 5y ago
        Maybe this is arguing semantics but unless you run something like Gentoo you will most likely get the linux kernel as a binary blob contained in a package your distribution provides. There isn't really any guarantee that this will actually contain untampered linux kernel sources (and in case of something like RHEL it most likely doesn't because of backports) unless you audit it, which most people won't do (and maybe can't do). So, in princpile at least, this isn't really that much better than the node_modules situation. Security and trust are hard issues and piling on 100s of random js dependencies sure doesn't help but you either build everything yourself or you need to trust somebody at some point.
        ptx 5y ago
        You do need to trust somebody (such as your Linux distribution of choice) but with NPM you're trusting thousands of somebodies and your system's security depends directly on all of them being secure and trustworthy.
  - waingake 5y ago
    And should we all start rolling our own crypto now to avoid dependencies? In most cases a stable library is going to be much more secure than a custom implementation of `x`. Everything has trade-offs. What's stupid is dogma.
    - djbusby 5y ago
      I know you're being hyperbolic and I also want to add that for crypto you should just use libsodium. The algos and the code are very good. And lots of very smart folk have given it a lot of review. And its API is very nice.
      - IggleSniggle 5y ago
        When you say this, do you mean actual C libsodium? Because surely you don’t mean that I, a js developer, should need to figure out how to wrap this .h file thingy to get it to work in js when there’s SIX third-party libsodium implementations/wrappers/projects sitting right there listed on the libsodium website? /s
  - FranksTV 5y ago
    Or perhaps, the sky is not falling.
  - jimmont 5y ago
    This. And npm isn't the only instance. (Appreciate the voice in the wilderness Marcus.)
- pdonis 5y ago
  > Even if you’re writing just for a vetted distribution of an OS, and you write code with zero explicit dependencies, you still have much more than zero dependencies. Sure, the entire OS is a dependency. Nothing I said contradicts that. And yes, every application developer should be aware of what they are depending on when they write software for a particular OS. > The key is to have an entire ecosystem that you can, to some degree more or less, trust. You don't necessarily need to trust an entire ecosystem, but yes, every dependency you have is a matter of trust on your part; you are trusting the dependency to work the way you need it to work and not to introduce vulnerabilities that you aren't aware of and can't deal with. Which is why you need to be explicitly aware of every dependency you have, not just the ones you directly import.
  - IggleSniggle 5y ago
    I am actually not sure if this is possible, while also accepting security updates etc from my OS distributor? How do you literally personally vet every line of code that gets run directly AND indirectly by your application, and still have time to write an application? I’m okay with saying, “I trust RHEL to be roughly ok, just understand the model and how to use it, and keep my ear to the ground for the experts in case something comes up.” At the level of npm, I feel roughly the same about React. I don’t trust it quite as much, but I’m also not going to read every code change. I’ll read a CHANGELOG, sure, and spelunk through the code from time to time, but that’s not really the same. I’ll probably check out their direct dependencies the first time, but that’s it. I actually don’t know how you could call yourself an application developer in most ecosystems and know every single dependency you actually have all the way down, soup to nuts. Heck, there are dependencies that I accept so that my code will run on machines that I have no special knowledge of, not just my own familiar architecture. I accept them because I want to work on the details of my application and have it be useful on more than just my own machine. Edit for clarity: I agree with almost everything you’re suggesting as sensible. Just not with your conclusion: that you’re not a “real” application developer if you don’t know all of your dependencies
    - pdonis 5y ago
      > I am actually not sure if this is possible, while also accepting security updates etc from my OS distributor? Accepting the OS as a dependency includes the security updates from the OS, sure. > How do you literally personally vet every line of code Ah, I see, you think "understanding the dependency" requires vetting every line of code. That's not what I meant. What I meant is, if you use library A, and library A depends on libraries B, C, and D, and those libraries in turn depend on libraries E, F, G, H, I, etc. etc., then you don't just need to be aware that you depend on library A, because that's the only one you're directly importing. You need to be aware of all the dependencies, all the way down. You might not personally vet every line of code in every one of them, but you need to be aware that you're using them and you need to be aware of how trustworthy they are, so you can judge whether it's really worth having them and exposing your application to the risks of using them. > I’ll probably check out their direct dependencies the first time, but that’s it. So if they introduce a new dependency, you don't care? You should. That's the kind of thing I'm talking about. Again, you might not go and vet every line of code in the new dependency, but you need to be aware that it's there and how risky it is. > I actually don’t know how you could call yourself an application developer in most ecosystems and know every single dependency you actually have all the way down, soup to nuts. If you're developing using open source code, information about what dependencies a given library has is easily discoverable. If you're developing for a proprietary system, things might be different.
- deleted 5y ago
  [deleted]

nitwit005 5y ago

I mean, you say that, but the practice of pulling in so many dependencies is fairly recent. It wasn't even possible for most projects before everyone had fast internet.

dpweb 5y ago

Some of the comments in this thread are wild. Huge dependency trees are bad pattern, plain and simple. The problem isn’t only ridiculous amounts of untrusted code, but thousands of new developers of the last 10 years who think this is the way to write reliable code. Never acknowledged the risks of having everyone write your code for you, and overestimate how unique and interesting their apps are. If you must participate in this madness, static analysis tools exist to scan your 10000 dependencies, taking security seriously is the issue.

deleted 5y ago
[deleted]
noway421 5y ago
Most of those dependencies have well defined, stable api. They use or at least try to follow semver. And you're probably only hitting about 10% of your dependencies on the critical path you're using, meaning that a lot of potentially vulnerable code is never executed. I get the supply chain attacks. I get that you have a tree of untrusted javascript code that you're executing in your app, on install, on build and in runtime. But there's also Snyk and Dependabot which issue you alerts when your dependency tree has published CVEs. We can talk about alert fatigue, but to be honest, I feel more secure with my node_modules folder than I do with my operating system and plethora of DLLs it loads. I don't wanna turn this into a whataboutism argument, but at some point you gotta get to work, write some code and depend on some libraries other people have written.
- laurent92 5y ago
  > Snyk and Dependabot Wait, I’m not safe using “npm audit”?
- grey-area 5y ago
  Semver will not save you.
- ptx 5y ago
  > And you're probably only hitting about 10% of your dependencies on the critical path you're using, meaning that a lot of potentially vulnerable code is never executed. If a dependency has been compromised it doesn't matter if its code is actually used, since it can include a lifecycle script that's executed at install-time, which was apparently the mechanism for the recent ua-parser-js exploit.

grey-area 5y ago

This is the direct result of the culture of tiny dependencies in JS and some other languages, but not all ecosystems are like this. If you choose to use node, this is where you end up, but it was a choice. Many languages have a decent standard library which covers most of the bases, so it’s possible to have a very restricted set of dependencies.

jschrf 5y ago
Unfortunately for frontend and the Node ecosystem, it's too late to try and put the toothpaste back in the tube. Hopefully Deno helps with this pain point.