4 ms·

Is that a joke? Surely you aren't suggesting that developers should have full knowledge of all transitive dependencies that compose their application stack.

bildung 7y ago
> Is that a joke? Surely you aren't suggesting that developers should have full knowledge of all transitive dependencies that compose their application stack. This will be borderline impossible for a nodejs developer, but a perfectly realistic expectation for e.g. a Django project.
ivan_gammel 7y ago
as CTO I know all the dependencies in the application stack of our company and I require developers to have sufficient understanding of their purpose, licensing model etc. There are tools to simplify this task. It’s much harder to do for TypeScript than for Java, mostly because NPM ecosystem is a huge pile of junk, but it’s still important part of developer's job.
- root_axis 7y ago
  > as CTO I know all the dependencies in the application stack of our company You know all of the transitive dependencies in the application stack of your company? Out of curiosity, what is the ballpark figure on the total number of transitive dependencies in your application?
  - ivan_gammel 7y ago
    Hundreds, mostly because of the JS nonsense. Our Java microservices are lean and strictly controlled.
- nl 7y ago
  as CTO I know all the dependencies in the application stack of our company I'm a few years out of the Java world these days, but you've vetted every line in every one of those Apache commons projects that gets pulled in? Because that's a lot of reading... Over 10 years ago I went through every single Java RSS parsing library and all their forks (I think there were 5) to fix Xml eXternal Entity (XXE) attacks (eg[1]) and submitted fixes for them all. That was a huge amount of work, because they all used random version of different XML parsers, all of which required external entity processing to be turned off in different ways. None of the authors of the packages were aware of the issue, and as far I could determine at the time I was the only person in the entire Java ecosystem who knew all the different parsers arguments. This was a critical, remotely exploitable vulnerability. I'd never claim to know about it now, and it was only for a few weeks back in 2006 that I really knew it then. So when you say you "know" all the dependencies in your stack, what do you mean exactly? [1] https://rometools.jira.com/browse/ROME-46 https://rometools.jira.com/browse/ROME-46
  - ivan_gammel 7y ago
    We barely have any of the Apache commons dependencies - in most cases use of them is not justified in modern Java. If there’s anything like that we are ready to support such dependency on source code level, patching and setting up own build pipeline if necessary. Free software cannot be trusted if you are not ready to own it. Commercial software cannot be trusted either, but there are other ways to manage the risks there.
- fookitty 7y ago
  I can't believe you check all the source code, dependencies source code and the rest of the shit show that npm brings to the party, that's very cool, all CTOs should be like you. As a CTO should you have to do this though? I'm not sure.
  - ivan_gammel 7y ago
    We are growing startup and it’s important to have this culture of understanding what and how you build from the very beginning. This responsibility is now transferred to technical leads, but I’m still aware of what’s going on and spend some time on the code reviews.
- imtringued 7y ago
  How much time does it take to read through the millions of lines of code?
  - diggan 7y ago
    Usually you don't write a project and then sit down to read through all dependencies, but it's a process that happens when you do add a dependency. So, the process would be something like: have a problem that a library might solve, find a library that could solve the problem, verify it can solve the problem, look through the implementation, look through it's dependencies, then actually include it in the build. If anything doesn't look nice at each step, rollback and find something else or start writing your own implementation. Do this from beginning of the project and you'll always have a full understanding of the system.
flukus 7y ago
Not full knowledge. But you should know about every single dependency, know what they do and if they're from a trustworthy source. Being able to say with confidence that you aren't installing malware on your customers machines is the least you can do. On the developer side there's more though, you should know what their release cycles are like, how they manage compatibility, how long they support older releases etc. You don't what your app stuck in a 3 month upgrade cycle just because a transitive dependency needed an update. There's a reason dependencies are there own form of technical debt and need to be minimized.
- root_axis 7y ago
  Knowing every transitive dependency in your application stack is totally unrealistic and is pretty much impossible for any modern commercial software outside of specialized/embedded systems. Having knowledge of every dependency explicitly linked into your application is a basic tenet of software development that literally every software professional would endorse as a necessary best practice, but it's not what we're talking about here. I am 100% certain that every developer with a commercial Electron app is aware that they are relying on Electron as a dependency and Github as a trustworthy software vendor.
  - flukus 7y ago
    > outside of specialized/embedded systems. I think you're conflating platforms and dependencies. The OS it runs on is a platform that the the user already trusts, left-pad is a dependency the developers are responsible for. There's some grey area like electron or the JVM but even there electron sticks out as a huge binary blob the user isn't explicitly asked to install.
    - root_axis 7y ago
      I never brought up operating systems in the context of this discussion on dependencies. Electron is not an operating system, so I am not sure what your point is.
- rdevnull 7y ago
ryandrake 7y ago
Honestly, it boggles the mind to think that a responsible developer would just pull in some 3rd party dependency without auditing what it does/calls and what other sub-dependencies it has, and then ship it as their product that their business relies on. Everywhere I’ve ever worked, adopting a dependent library was a HUGE DEAL. You don’t do it lightly. You have to know what’s in it, know what it calls, know what the license is, understand the increased security attack surface, measure how much bigger it makes your binary, measure any performance deltas, and so on. In bigger companies, you need to get all sorts of approvals... Who the heck are these companies where you just hook up the git submodule and wing it?
- watermelon0 7y ago
  Can you even know every transitive dependency in an average Node.js project? I just looked at a simple web service we have, which doesn't use any framework or such, and it's 5 dependencies pull in 75 dependencies altogether. Since most applications nowadays are server side, you would usually already pull 1 GB+ of Docker layers, before you even add application + libraries. Facebook on iOS seems to be around 400 MB of size (first result from quick googling, might be wrong), and average Electron app is 120 MB+, so I don't think the size is a huge concern.
DagAgren 7y ago
I do know just that for apps I ship. I will never in a million year ship anything on, for instance, node.js, for exactly this reason.
josefx 7y ago
For licensing reasons alone, almost got hit by one dependency switching from LGPL 2 to GPL 3 and one project we bought dropped dead once someone went over its dependency tree and noticed the mess of highly incompatible licenses, it would have taken years to get to a point where we could legally publish anything. Not knowing about all of your dependencies is a liability.
JohnFen 7y ago
I think that's not an unreasonable thing to expect at all. If the libraries you're using make that impossible, then you shouldn't use those libraries. You should know what you're shipping. As a developer, I'm honestly and utterly stunned to see other developers arguing otherwise.

bildung 7y ago

> Is that a joke? Surely you aren't suggesting that developers should have full knowledge of all transitive dependencies that compose their application stack. This will be borderline impossible for a nodejs developer, but a perfectly realistic expectation for e.g. a Django project.

ivan_gammel 7y ago

as CTO I know all the dependencies in the application stack of our company and I require developers to have sufficient understanding of their purpose, licensing model etc. There are tools to simplify this task. It’s much harder to do for TypeScript than for Java, mostly because NPM ecosystem is a huge pile of junk, but it’s still important part of developer's job.

root_axis 7y ago
> as CTO I know all the dependencies in the application stack of our company You know all of the transitive dependencies in the application stack of your company? Out of curiosity, what is the ballpark figure on the total number of transitive dependencies in your application?
- ivan_gammel 7y ago
  Hundreds, mostly because of the JS nonsense. Our Java microservices are lean and strictly controlled.
nl 7y ago
as CTO I know all the dependencies in the application stack of our company I'm a few years out of the Java world these days, but you've vetted every line in every one of those Apache commons projects that gets pulled in? Because that's a lot of reading... Over 10 years ago I went through every single Java RSS parsing library and all their forks (I think there were 5) to fix Xml eXternal Entity (XXE) attacks (eg[1]) and submitted fixes for them all. That was a huge amount of work, because they all used random version of different XML parsers, all of which required external entity processing to be turned off in different ways. None of the authors of the packages were aware of the issue, and as far I could determine at the time I was the only person in the entire Java ecosystem who knew all the different parsers arguments. This was a critical, remotely exploitable vulnerability. I'd never claim to know about it now, and it was only for a few weeks back in 2006 that I really knew it then. So when you say you "know" all the dependencies in your stack, what do you mean exactly? [1] https://rometools.jira.com/browse/ROME-46 https://rometools.jira.com/browse/ROME-46
- ivan_gammel 7y ago
  We barely have any of the Apache commons dependencies - in most cases use of them is not justified in modern Java. If there’s anything like that we are ready to support such dependency on source code level, patching and setting up own build pipeline if necessary. Free software cannot be trusted if you are not ready to own it. Commercial software cannot be trusted either, but there are other ways to manage the risks there.
fookitty 7y ago
I can't believe you check all the source code, dependencies source code and the rest of the shit show that npm brings to the party, that's very cool, all CTOs should be like you. As a CTO should you have to do this though? I'm not sure.
- ivan_gammel 7y ago
  We are growing startup and it’s important to have this culture of understanding what and how you build from the very beginning. This responsibility is now transferred to technical leads, but I’m still aware of what’s going on and spend some time on the code reviews.
imtringued 7y ago
How much time does it take to read through the millions of lines of code?
- diggan 7y ago
  Usually you don't write a project and then sit down to read through all dependencies, but it's a process that happens when you do add a dependency. So, the process would be something like: have a problem that a library might solve, find a library that could solve the problem, verify it can solve the problem, look through the implementation, look through it's dependencies, then actually include it in the build. If anything doesn't look nice at each step, rollback and find something else or start writing your own implementation. Do this from beginning of the project and you'll always have a full understanding of the system.

flukus 7y ago

Not full knowledge. But you should know about every single dependency, know what they do and if they're from a trustworthy source. Being able to say with confidence that you aren't installing malware on your customers machines is the least you can do. On the developer side there's more though, you should know what their release cycles are like, how they manage compatibility, how long they support older releases etc. You don't what your app stuck in a 3 month upgrade cycle just because a transitive dependency needed an update. There's a reason dependencies are there own form of technical debt and need to be minimized.

root_axis 7y ago
Knowing every transitive dependency in your application stack is totally unrealistic and is pretty much impossible for any modern commercial software outside of specialized/embedded systems. Having knowledge of every dependency explicitly linked into your application is a basic tenet of software development that literally every software professional would endorse as a necessary best practice, but it's not what we're talking about here. I am 100% certain that every developer with a commercial Electron app is aware that they are relying on Electron as a dependency and Github as a trustworthy software vendor.
- flukus 7y ago
  > outside of specialized/embedded systems. I think you're conflating platforms and dependencies. The OS it runs on is a platform that the the user already trusts, left-pad is a dependency the developers are responsible for. There's some grey area like electron or the JVM but even there electron sticks out as a huge binary blob the user isn't explicitly asked to install.
  - root_axis 7y ago
    I never brought up operating systems in the context of this discussion on dependencies. Electron is not an operating system, so I am not sure what your point is.
rdevnull 7y ago

ryandrake 7y ago

Honestly, it boggles the mind to think that a responsible developer would just pull in some 3rd party dependency without auditing what it does/calls and what other sub-dependencies it has, and then ship it as their product that their business relies on. Everywhere I’ve ever worked, adopting a dependent library was a HUGE DEAL. You don’t do it lightly. You have to know what’s in it, know what it calls, know what the license is, understand the increased security attack surface, measure how much bigger it makes your binary, measure any performance deltas, and so on. In bigger companies, you need to get all sorts of approvals... Who the heck are these companies where you just hook up the git submodule and wing it?

watermelon0 7y ago
Can you even know every transitive dependency in an average Node.js project? I just looked at a simple web service we have, which doesn't use any framework or such, and it's 5 dependencies pull in 75 dependencies altogether. Since most applications nowadays are server side, you would usually already pull 1 GB+ of Docker layers, before you even add application + libraries. Facebook on iOS seems to be around 400 MB of size (first result from quick googling, might be wrong), and average Electron app is 120 MB+, so I don't think the size is a huge concern.

DagAgren 7y ago

I do know just that for apps I ship. I will never in a million year ship anything on, for instance, node.js, for exactly this reason.

josefx 7y ago

For licensing reasons alone, almost got hit by one dependency switching from LGPL 2 to GPL 3 and one project we bought dropped dead once someone went over its dependency tree and noticed the mess of highly incompatible licenses, it would have taken years to get to a point where we could legally publish anything. Not knowing about all of your dependencies is a liability.

JohnFen 7y ago

I think that's not an unreasonable thing to expect at all. If the libraries you're using make that impossible, then you shouldn't use those libraries. You should know what you're shipping. As a developer, I'm honestly and utterly stunned to see other developers arguing otherwise.