6 ms·

> Dependencies of your dependencies! Imagine installing a library that depends on a couple dozen libraries in its own right. You may not have time to inspect ev

> Dependencies of your dependencies! Imagine installing a library that depends on a couple dozen libraries in its own right. You may not have time to inspect every single node in the dependency graph. Excuse me, are you, the developer, telling me, a user, that you don't know what you're giving me to run on my computer?

michaelmrose 7y ago
Hi my name is node_modules have you met me? If you invite me to a party I bring my 500 closest friends as my plus one! I just looked at the node_modules folder for a project that runs user configured code when your imap server gets a new mail via imap idle. It's called imapnotify and inside its node_modules dir I see it pulls in 549 js files. The majority are under 100 lines per file dozens are between 1 and 30. Cutting just the last segment so that /foo/bar/baz.js and /bam/baz.js both appear as baz.js and removing dupes reduced the number of unique filenames to 383. Basically every js dev is either telling you or not telling you that they don't know what they are giving you to run on your computer.
- catalogia 7y ago
  There are reasons forums known for being less diplomatic than HN often take delight in heaping scorn and ridicule on js devs. This is one of those reasons.
- JohnFen 7y ago
  > If you invite me to a party I bring my 500 closest friends as my plus one! That sort of thing is a big red flag that perhaps you should reconsider using that package. But even if you do use it, that in no way reduces the fact that it's your responsibility to know what you're shipping.
  - michaelmrose 7y ago
    That is every javascript package ever incidentally. Do you have an alternative suggestion as far as connecting to an imap server and responding to events? Not a library to write such a thing but an actual implementation.
    - JohnFen 7y ago
      No, I don't, but even if there are no such alternate packages, that doesn't mean it's OK to use one that you can't sufficiently understand. If you can't find an acceptable package, then I'd absolutely recommend writing that functionality yourself. connecting to an IMAP server and responding to events aren't complicated tasks, after all.
      - michaelmrose 7y ago
        This is entirely feasible but what percentage of people can write replacements for problematic tools and what tools can they afford to replace time?
        JohnFen 7y ago
        Among software engineers? I would expect (hope?) that the percentage that could do this approaches 100.
        michaelmrose 7y ago
        How much extra time do you have to reimpliment things that work because you disagree with how they are written?
        JohnFen 7y ago
        You misunderstand what I'm saying. This isn't about disagreeing with how anything is written, this is about the responsibility of developers to know what they're shipping. If devs can't do things like be able to tell what dependencies a module/library has, then they shouldn't use that module/library.
root_axis 7y ago
Is that a joke? Surely you aren't suggesting that developers should have full knowledge of all transitive dependencies that compose their application stack.
- bildung 7y ago
  > Is that a joke? Surely you aren't suggesting that developers should have full knowledge of all transitive dependencies that compose their application stack. This will be borderline impossible for a nodejs developer, but a perfectly realistic expectation for e.g. a Django project.
- ivan_gammel 7y ago
  as CTO I know all the dependencies in the application stack of our company and I require developers to have sufficient understanding of their purpose, licensing model etc. There are tools to simplify this task. It’s much harder to do for TypeScript than for Java, mostly because NPM ecosystem is a huge pile of junk, but it’s still important part of developer's job.
  - root_axis 7y ago
    > as CTO I know all the dependencies in the application stack of our company You know all of the transitive dependencies in the application stack of your company? Out of curiosity, what is the ballpark figure on the total number of transitive dependencies in your application?
    - ivan_gammel 7y ago
      Hundreds, mostly because of the JS nonsense. Our Java microservices are lean and strictly controlled.
  - nl 7y ago
    as CTO I know all the dependencies in the application stack of our company I'm a few years out of the Java world these days, but you've vetted every line in every one of those Apache commons projects that gets pulled in? Because that's a lot of reading... Over 10 years ago I went through every single Java RSS parsing library and all their forks (I think there were 5) to fix Xml eXternal Entity (XXE) attacks (eg[1]) and submitted fixes for them all. That was a huge amount of work, because they all used random version of different XML parsers, all of which required external entity processing to be turned off in different ways. None of the authors of the packages were aware of the issue, and as far I could determine at the time I was the only person in the entire Java ecosystem who knew all the different parsers arguments. This was a critical, remotely exploitable vulnerability. I'd never claim to know about it now, and it was only for a few weeks back in 2006 that I really knew it then. So when you say you "know" all the dependencies in your stack, what do you mean exactly? [1] https://rometools.jira.com/browse/ROME-46 https://rometools.jira.com/browse/ROME-46
wpietri 7y ago
I think that ship sailed a long time ago. Right now I'm building an app with Google's mobile toolkit, Flutter. Flutter itself is a few hundred kloc. And on top of that, I'm using a bunch of libraries; I wouldn't be shocked if it were the same amount of code again. Assuming I can properly code review a few hundred lines per hour, reviewing all that will take me a year or so. Which is maybe 5x what it will take me to ship an MVP.
- concerned_user 7y ago
  You are missing the fact that dependency tree is constantly changing as dependencies get bug fixes and security patches, so in the middle of your review you will be forced to re-check. Unless you are willing to freeze whole tree until review is done but then you don't get bug fixes and security updates for period of initial review + review of updated packages.
concerned_user 7y ago
With NPM as an example to fully know every dependency is a highly time-intensive and complicated task due to following: * you can't be sure code in package is same you see on github as npm doesn't do signatures * code already in node_modules is usually minified and obfuscated So only real way to be sure is to make own repository, copy every dependency, check code, build own packages and only use packages from your repo. Repeat process when versions are updated. So to answer your question, yes, 99% of apps, they don't know.
- JohnFen 7y ago
  That's an excellent reason to avoid using apps.

michaelmrose 7y ago

Hi my name is node_modules have you met me? If you invite me to a party I bring my 500 closest friends as my plus one! I just looked at the node_modules folder for a project that runs user configured code when your imap server gets a new mail via imap idle. It's called imapnotify and inside its node_modules dir I see it pulls in 549 js files. The majority are under 100 lines per file dozens are between 1 and 30. Cutting just the last segment so that /foo/bar/baz.js and /bam/baz.js both appear as baz.js and removing dupes reduced the number of unique filenames to 383. Basically every js dev is either telling you or not telling you that they don't know what they are giving you to run on your computer.

catalogia 7y ago
There are reasons forums known for being less diplomatic than HN often take delight in heaping scorn and ridicule on js devs. This is one of those reasons.
JohnFen 7y ago
> If you invite me to a party I bring my 500 closest friends as my plus one! That sort of thing is a big red flag that perhaps you should reconsider using that package. But even if you do use it, that in no way reduces the fact that it's your responsibility to know what you're shipping.
- michaelmrose 7y ago
  That is every javascript package ever incidentally. Do you have an alternative suggestion as far as connecting to an imap server and responding to events? Not a library to write such a thing but an actual implementation.
  - JohnFen 7y ago
    No, I don't, but even if there are no such alternate packages, that doesn't mean it's OK to use one that you can't sufficiently understand. If you can't find an acceptable package, then I'd absolutely recommend writing that functionality yourself. connecting to an IMAP server and responding to events aren't complicated tasks, after all.
    - michaelmrose 7y ago
      This is entirely feasible but what percentage of people can write replacements for problematic tools and what tools can they afford to replace time?
      - JohnFen 7y ago
        Among software engineers? I would expect (hope?) that the percentage that could do this approaches 100.
        michaelmrose 7y ago
        How much extra time do you have to reimpliment things that work because you disagree with how they are written?
        JohnFen 7y ago
        You misunderstand what I'm saying. This isn't about disagreeing with how anything is written, this is about the responsibility of developers to know what they're shipping. If devs can't do things like be able to tell what dependencies a module/library has, then they shouldn't use that module/library.

root_axis 7y ago

Is that a joke? Surely you aren't suggesting that developers should have full knowledge of all transitive dependencies that compose their application stack.

bildung 7y ago
> Is that a joke? Surely you aren't suggesting that developers should have full knowledge of all transitive dependencies that compose their application stack. This will be borderline impossible for a nodejs developer, but a perfectly realistic expectation for e.g. a Django project.
ivan_gammel 7y ago
as CTO I know all the dependencies in the application stack of our company and I require developers to have sufficient understanding of their purpose, licensing model etc. There are tools to simplify this task. It’s much harder to do for TypeScript than for Java, mostly because NPM ecosystem is a huge pile of junk, but it’s still important part of developer's job.
- root_axis 7y ago
  > as CTO I know all the dependencies in the application stack of our company You know all of the transitive dependencies in the application stack of your company? Out of curiosity, what is the ballpark figure on the total number of transitive dependencies in your application?
  - ivan_gammel 7y ago
    Hundreds, mostly because of the JS nonsense. Our Java microservices are lean and strictly controlled.
- nl 7y ago
  as CTO I know all the dependencies in the application stack of our company I'm a few years out of the Java world these days, but you've vetted every line in every one of those Apache commons projects that gets pulled in? Because that's a lot of reading... Over 10 years ago I went through every single Java RSS parsing library and all their forks (I think there were 5) to fix Xml eXternal Entity (XXE) attacks (eg[1]) and submitted fixes for them all. That was a huge amount of work, because they all used random version of different XML parsers, all of which required external entity processing to be turned off in different ways. None of the authors of the packages were aware of the issue, and as far I could determine at the time I was the only person in the entire Java ecosystem who knew all the different parsers arguments. This was a critical, remotely exploitable vulnerability. I'd never claim to know about it now, and it was only for a few weeks back in 2006 that I really knew it then. So when you say you "know" all the dependencies in your stack, what do you mean exactly? [1] https://rometools.jira.com/browse/ROME-46 https://rometools.jira.com/browse/ROME-46

wpietri 7y ago

I think that ship sailed a long time ago. Right now I'm building an app with Google's mobile toolkit, Flutter. Flutter itself is a few hundred kloc. And on top of that, I'm using a bunch of libraries; I wouldn't be shocked if it were the same amount of code again. Assuming I can properly code review a few hundred lines per hour, reviewing all that will take me a year or so. Which is maybe 5x what it will take me to ship an MVP.

concerned_user 7y ago
You are missing the fact that dependency tree is constantly changing as dependencies get bug fixes and security patches, so in the middle of your review you will be forced to re-check. Unless you are willing to freeze whole tree until review is done but then you don't get bug fixes and security updates for period of initial review + review of updated packages.

concerned_user 7y ago

With NPM as an example to fully know every dependency is a highly time-intensive and complicated task due to following: * you can't be sure code in package is same you see on github as npm doesn't do signatures * code already in node_modules is usually minified and obfuscated So only real way to be sure is to make own repository, copy every dependency, check code, build own packages and only use packages from your repo. Repeat process when versions are updated. So to answer your question, yes, 99% of apps, they don't know.

JohnFen 7y ago
That's an excellent reason to avoid using apps.