3 ms·

you are part of the problem here. Many folks are refusing to even consider the fundamental issue because it is different from the mindset they are used to. Yes

by apalmer 11y ago

you are part of the problem here. Many folks are refusing to even consider the fundamental issue because it is different from the mindset they are used to. Yes this particular episode is related to the unpublishing results... however the underlying issue is not only about unpublishing results. The underlying issue is: number 1, Having thousands of dependencies is a software engineering trade off, their is a upside and a downside of that trade off. Having Thousands of dependencies is fundamentally more 'risky' than have a few. Period. End Of Story. It is ok to accept the trade off based on what you are optimizing for. It is not ok to refuse to accept the fact you are making a tradeoff. number 2, is you should not be running production builds or builds for important components based on dependencies on the naked internet. This is also an engineering tradeoff but is firmly in the camp of what is considered 'bad practice'. This is fundamentally the same exact underlying reason why we have build servers instead of just builds from developer work stations. Developer workstations are not controlled environments, which means the builds are not guaranteed repeatable. A build server that depends on the state of the naked internet is not a controlled environment. It blows up today because of unpublishing, it will blow up tomorrow because of something else.

Wintamute 11y ago
I agree with what you're saying. But I think the "grand experiment" of npm and modern javascript development is to go "all in" on the "lots of small modules" approach. Having many small dependencies is, and always has been, a trade off. The question is how far can we get optimising away from the pitfalls, while still retaining the benefits? I think it's a worthy exercise, since the prize is no less than efficiently distributing coding effort across the globe. Problems like we've seen today are a double edged sword. On the one hand they're troubling, but it'll focus the community on the problems, and hopefully npm will make a few changes too. > A build server that depends on the state of the naked internet Whether npm based or not, don't the vast majority of build servers, for whatever language or platform, rely on the state of the naked internet? Whether that's downloading binaries, compiling from source, or whatever?
- clappski 11y ago
  Your build servers shouldn't really be exposed to the public internet, best practise would be to get all dependencies/binaries of a local network.
gnaritas 11y ago
> This is fundamentally the same exact underlying reason why we have build servers instead of just builds from developer work stations. Developer workstations are not controlled environments, which means the builds are not guaranteed repeatable. A side note, if your build depends on the box it's being built on, your build process is bad. The build from any dev workstation should be identical to any other build. People have build boxes to automate building/deployment and testing and alerting people things broke; they aren't about dependency management. If you're using a build box because of dependency management issues with dev workstations, you're doing it wrong.
1wd 11y ago
Thank you. I find it very surprising that dependencies on the naked internet are so common nowadays. Why isn't it trivial to build my own package repositories where I mirror the known good dependencies that I need?
joepie91_ 11y ago
> Having Thousands of dependencies is fundamentally more 'risky' than have a few. Period. End Of Story. No, this is misrepresenting the issue. The amount of dependencies you have is just as irrelevant as the amount of lines they contain. What actually matters here are the following things: 1. How many different entities am I trusting? 2. How hard is it to audit this code? 3. How easy is it to 'lock in' a dependency version without missing out on updates I need? 4. How much work is it to review updates? There's no significant difference between small and large dependencies for point 1. You can choose how many entities you trust, regardless of whether the functionality you are using is split up into many small packages from the same author, or combined into one big one. There will usually be no difference for point 4, because the amount of updates generally relates to the amount of functionality that is being updated. This means that larger modules means less CHANGELOG files to track down, but smaller modules means less chance of an upgrade affecting something you're not using to begin with. It evens out nicely. However, for points 2 and 3, there is a difference - in favour of small modules. A much smaller, more well-defined surface with less implicit dependencies and coupling. These are the metrics that actually matter. EDIT: To clarify, 'smaller' and 'larger' here refers to complexity, not line count.

by apalmer 11y ago

Wintamute 11y ago

I agree with what you're saying. But I think the "grand experiment" of npm and modern javascript development is to go "all in" on the "lots of small modules" approach. Having many small dependencies is, and always has been, a trade off. The question is how far can we get optimising away from the pitfalls, while still retaining the benefits? I think it's a worthy exercise, since the prize is no less than efficiently distributing coding effort across the globe. Problems like we've seen today are a double edged sword. On the one hand they're troubling, but it'll focus the community on the problems, and hopefully npm will make a few changes too. > A build server that depends on the state of the naked internet Whether npm based or not, don't the vast majority of build servers, for whatever language or platform, rely on the state of the naked internet? Whether that's downloading binaries, compiling from source, or whatever?

clappski 11y ago
Your build servers shouldn't really be exposed to the public internet, best practise would be to get all dependencies/binaries of a local network.

gnaritas 11y ago

> This is fundamentally the same exact underlying reason why we have build servers instead of just builds from developer work stations. Developer workstations are not controlled environments, which means the builds are not guaranteed repeatable. A side note, if your build depends on the box it's being built on, your build process is bad. The build from any dev workstation should be identical to any other build. People have build boxes to automate building/deployment and testing and alerting people things broke; they aren't about dependency management. If you're using a build box because of dependency management issues with dev workstations, you're doing it wrong.

1wd 11y ago

Thank you. I find it very surprising that dependencies on the naked internet are so common nowadays. Why isn't it trivial to build my own package repositories where I mirror the known good dependencies that I need?

joepie91_ 11y ago

> Having Thousands of dependencies is fundamentally more 'risky' than have a few. Period. End Of Story. No, this is misrepresenting the issue. The amount of dependencies you have is just as irrelevant as the amount of lines they contain. What actually matters here are the following things: 1. How many different entities am I trusting? 2. How hard is it to audit this code? 3. How easy is it to 'lock in' a dependency version without missing out on updates I need? 4. How much work is it to review updates? There's no significant difference between small and large dependencies for point 1. You can choose how many entities you trust, regardless of whether the functionality you are using is split up into many small packages from the same author, or combined into one big one. There will usually be no difference for point 4, because the amount of updates generally relates to the amount of functionality that is being updated. This means that larger modules means less CHANGELOG files to track down, but smaller modules means less chance of an upgrade affecting something you're not using to begin with. It evens out nicely. However, for points 2 and 3, there is a difference - in favour of small modules. A much smaller, more well-defined surface with less implicit dependencies and coupling. These are the metrics that actually matter. EDIT: To clarify, 'smaller' and 'larger' here refers to complexity, not line count.