8 ms·

I’m fascinated by the fact that while node has become a new standard in the industry , and the project is receiving lots of supports from all sorts of companies

by asien 7y ago

I’m fascinated by the fact that while node has become a new standard in the industry , and the project is receiving lots of supports from all sorts of companies ( IBM , Microsoft etc...) absolutely no discussion has been opened about how much at risk the JavaScript ecosystem actually is with « npm » and it’s weekly dramas Not a month pass without something going wrong inside of inc, millions of developers are dependant on it but nothing seem to worry people...

TimothyBJacobs 7y ago
There has been. CJ Silverio gave a talk on that topic at JSConf this year: https://www.youtube.com/watch?v=MO8hZlgK5zc https://www.youtube.com/watch?v=MO8hZlgK5zc See also Entropic as a possible alternative to NPM: https://github.com/entropic-dev/entropic https://github.com/entropic-dev/entropic
- pluma 7y ago
  For context: Entropic was started by several former npm Inc core developers including CJ who left the company after the CEO change was announced internally. CJ herself is the former CTO of npm Inc. Basically most of the top developers have left the company, others were laid off. It seems to be only a matter of time until none of the people working on npm a year ago remain.
- tuananh 7y ago
  i doubt entropic will become sth significant. it already lost the momentum it seems.
humtum 7y ago
People care. Developers have limited registry options due to lock-in. Creating effective security processes across a massive ecosystem of open source developers is a difficult problem. Registries can't easily create security practices that fit into a heterogenous pool of oss governance and development models. Especially when implementing more rigorous security has the potential to diminish their network effects and developer productivity. Curious to see how npm, GitHub Package Manager, and others address these issues.
paulddraper 7y ago
The people that would care the most are mostly immune to npm issues. Privately hosted npm repos, checked-in to version control systems, etc. There are plenty of solutions if you don't like npm. However, they carry the traditional costs of managing dependencies; if you want super super super easy above all else, that's what npm does.
- tiborsaas 7y ago
  I don't think that's accurate. Most devs and thus companies treat NPM as a utility. Maybe very large companies would not feel it, but if NPM went down tomorrow there would be utter chaos on the internet.
  - paulddraper 7y ago
    There would be chaos, just not for FAANG.
    - Bartweiss 7y ago
      Someone upthread asked "why is NPM different from PyPi/pip in this?" There are lots of practical answers - PyPi is open source, Python packages aren't so fragmented, and so on. But honestly, a huge part of the difference is that PyPi has sponsors like PyPi and AWS using its baseline implementation. NPM's private repository system means the public system just doesn't have that kind of pressure on it.
      - vageli 7y ago
        I would be deeply surprised if AWS teams use public pypi. Much more reasonable would be to mirror public packages they use internally. What if a minor version change contains a relicensing of the library, for instance?
        Bartweiss 7y ago
        Good point. Presumably they're fixing versions, even companies on public registries should do that to avoid re-licensing issues, but it'd be an unreasonable legal & security risk. I guess my broader thought was that PyPi is a more reliable free offering than NPM because it's not focused on a 'premium' version for the biggest users. But that's different than AWS - presumably they're sponsoring it in a broader "making development accessible is good for AWS" sense.
knightofmars 7y ago
Yes. The Node ecosystem is a huge liability just waiting to happen. Any organization that depends on NPM is making a huge gamble. You can do a lot to mitigate this (private NPM repo, locks) but the reality is that the dependency chains are dangerous. Is someone in an organization going to audit all of those dependencies? Especially under the circumstances where they've been declared without an explicit version (>, >=, <, <=, ~, ^, 1.2.x, *).
- president 7y ago
  As someone who has no insight into the Node/NPM/JS world, how is this different from Python's PyPi, which I would think suffers the same issue?
  - josegonzalez 7y ago
    The number of packages you need to audit for what would otherwise seem to be a trivial feature is exponentially larger in JS world than in Python. A good example of that is the left-pad debacle, wherein a package that left-pads a string was taken down, causing other packages - notably React - to fail to be installed because of either direct or transitive dependencies. In the Python world, it is indeed likely that unpublishing requests will cause issues, but the number of dependencies you'd need to audit/vendor is _much_ smaller for a typical python app than it is for a typical nodejs app, so your "attack surface" is also comparatively much smaller.
  - takeda 7y ago
    I think the problem is that NodeJS has a weird culture where a separate package is created for every little thing, then tons of other packages start depending on it. I don't live in NodeJS world but even I heard about package Left-Pad, that all it does is padding string from the left side. The author decided to pull it out from the repo rendering tons of other packages nonoperational[1]. [1] https://www.theregister.co.uk/2016/03/23/npm_left_pad_chaos/ https://www.theregister.co.uk/2016/03/23/npm_left_pad_chaos/
    - deergomoo 7y ago
cjblomqvist 7y ago
Honest question, is it that much better in other communities? In particular, it's there anything inherent to npm that's problematic or is it just that a huge community with a Unix mindset (small packages that does one thing well) is problematic?
- jonny_eh 7y ago
  Any community dependent on a particular repo is at risk. I'd argue that RubyGems is just as risky as NPM, for example. The problem is that there's no good fix or alternative, it's hard to avoid a single point of failure.
- pluma 7y ago
  No other package manager is run by a VC backed startup. That's inherently problematic and means the registry is in the hands of a company that could be killed off or sold at any moment because it needs to make massive profits (without having any way to truly make a profit in the first place) to continue to exist.
- kace91 7y ago
  Lack of a large enough standard library is a big differenciating factor, as it makes you very dependent on third party libraries. Even if you avoid it by creating your own utils, chances are that the creators of the large packages you use (like a database manager or a rest framework) will depend directly or indirectly of those third party tools.
- qbaqbaqba 7y ago
  It was a problem in otherwise very successful perl's CPAN. I don't recall any "dramas", but because of a huge number of dependencies installing Catalyst had a low change of going right the first time. But! Because of the test everything culture and CPAN testers effort broken modules were very rare. So after you managed to set up your system and had a long walk while tests would run it was guaranteed to work. More or less.
r3trohack3r 7y ago
There is actually a lot of discussion about this. There is also a lot of work going into different approaches to solve this. For example Frea. It's a conservative approach for incrementally federating the package registry and you can start using it today. It's ready for all your traffic. website: https://freajs.com https://freajs.com how it works: https://docs.google.com/presentation/d/16pxrYfpxxKRzhpMM0zZVlF5jo4f1bF9zOuceF9I-0is/edit https://docs.google.com/presentation/d/16pxrYfpxxKRzhpMM0zZV...
- paulddraper 7y ago
  Why was this downvoted?

by asien 7y ago

TimothyBJacobs 7y ago

There has been. CJ Silverio gave a talk on that topic at JSConf this year: https://www.youtube.com/watch?v=MO8hZlgK5zc https://www.youtube.com/watch?v=MO8hZlgK5zc See also Entropic as a possible alternative to NPM: https://github.com/entropic-dev/entropic https://github.com/entropic-dev/entropic

pluma 7y ago
For context: Entropic was started by several former npm Inc core developers including CJ who left the company after the CEO change was announced internally. CJ herself is the former CTO of npm Inc. Basically most of the top developers have left the company, others were laid off. It seems to be only a matter of time until none of the people working on npm a year ago remain.
tuananh 7y ago
i doubt entropic will become sth significant. it already lost the momentum it seems.

humtum 7y ago

People care. Developers have limited registry options due to lock-in. Creating effective security processes across a massive ecosystem of open source developers is a difficult problem. Registries can't easily create security practices that fit into a heterogenous pool of oss governance and development models. Especially when implementing more rigorous security has the potential to diminish their network effects and developer productivity. Curious to see how npm, GitHub Package Manager, and others address these issues.

paulddraper 7y ago

The people that would care the most are mostly immune to npm issues. Privately hosted npm repos, checked-in to version control systems, etc. There are plenty of solutions if you don't like npm. However, they carry the traditional costs of managing dependencies; if you want super super super easy above all else, that's what npm does.

tiborsaas 7y ago
I don't think that's accurate. Most devs and thus companies treat NPM as a utility. Maybe very large companies would not feel it, but if NPM went down tomorrow there would be utter chaos on the internet.
- paulddraper 7y ago
  There would be chaos, just not for FAANG.
  - Bartweiss 7y ago
    Someone upthread asked "why is NPM different from PyPi/pip in this?" There are lots of practical answers - PyPi is open source, Python packages aren't so fragmented, and so on. But honestly, a huge part of the difference is that PyPi has sponsors like PyPi and AWS using its baseline implementation. NPM's private repository system means the public system just doesn't have that kind of pressure on it.
    - vageli 7y ago
      I would be deeply surprised if AWS teams use public pypi. Much more reasonable would be to mirror public packages they use internally. What if a minor version change contains a relicensing of the library, for instance?
      - Bartweiss 7y ago
        Good point. Presumably they're fixing versions, even companies on public registries should do that to avoid re-licensing issues, but it'd be an unreasonable legal & security risk. I guess my broader thought was that PyPi is a more reliable free offering than NPM because it's not focused on a 'premium' version for the biggest users. But that's different than AWS - presumably they're sponsoring it in a broader "making development accessible is good for AWS" sense.

knightofmars 7y ago

Yes. The Node ecosystem is a huge liability just waiting to happen. Any organization that depends on NPM is making a huge gamble. You can do a lot to mitigate this (private NPM repo, locks) but the reality is that the dependency chains are dangerous. Is someone in an organization going to audit all of those dependencies? Especially under the circumstances where they've been declared without an explicit version (>, >=, <, <=, ~, ^, 1.2.x, *).

president 7y ago
As someone who has no insight into the Node/NPM/JS world, how is this different from Python's PyPi, which I would think suffers the same issue?
- josegonzalez 7y ago
  The number of packages you need to audit for what would otherwise seem to be a trivial feature is exponentially larger in JS world than in Python. A good example of that is the left-pad debacle, wherein a package that left-pads a string was taken down, causing other packages - notably React - to fail to be installed because of either direct or transitive dependencies. In the Python world, it is indeed likely that unpublishing requests will cause issues, but the number of dependencies you'd need to audit/vendor is _much_ smaller for a typical python app than it is for a typical nodejs app, so your "attack surface" is also comparatively much smaller.
- takeda 7y ago
  I think the problem is that NodeJS has a weird culture where a separate package is created for every little thing, then tons of other packages start depending on it. I don't live in NodeJS world but even I heard about package Left-Pad, that all it does is padding string from the left side. The author decided to pull it out from the repo rendering tons of other packages nonoperational[1]. [1] https://www.theregister.co.uk/2016/03/23/npm_left_pad_chaos/ https://www.theregister.co.uk/2016/03/23/npm_left_pad_chaos/
  - deergomoo 7y ago

cjblomqvist 7y ago

Honest question, is it that much better in other communities? In particular, it's there anything inherent to npm that's problematic or is it just that a huge community with a Unix mindset (small packages that does one thing well) is problematic?

jonny_eh 7y ago
Any community dependent on a particular repo is at risk. I'd argue that RubyGems is just as risky as NPM, for example. The problem is that there's no good fix or alternative, it's hard to avoid a single point of failure.
pluma 7y ago
No other package manager is run by a VC backed startup. That's inherently problematic and means the registry is in the hands of a company that could be killed off or sold at any moment because it needs to make massive profits (without having any way to truly make a profit in the first place) to continue to exist.
kace91 7y ago
Lack of a large enough standard library is a big differenciating factor, as it makes you very dependent on third party libraries. Even if you avoid it by creating your own utils, chances are that the creators of the large packages you use (like a database manager or a rest framework) will depend directly or indirectly of those third party tools.
qbaqbaqba 7y ago
It was a problem in otherwise very successful perl's CPAN. I don't recall any "dramas", but because of a huge number of dependencies installing Catalyst had a low change of going right the first time. But! Because of the test everything culture and CPAN testers effort broken modules were very rare. So after you managed to set up your system and had a long walk while tests would run it was guaranteed to work. More or less.

r3trohack3r 7y ago

There is actually a lot of discussion about this. There is also a lot of work going into different approaches to solve this. For example Frea. It's a conservative approach for incrementally federating the package registry and you can start using it today. It's ready for all your traffic. website: https://freajs.com https://freajs.com how it works: https://docs.google.com/presentation/d/16pxrYfpxxKRzhpMM0zZVlF5jo4f1bF9zOuceF9I-0is/edit https://docs.google.com/presentation/d/16pxrYfpxxKRzhpMM0zZV...

paulddraper 7y ago
Why was this downvoted?