5 ms·

> Super broken, everything related to npm's package lock stuff is broken by design. I've been saying it for years now and it seems people still cling to blindly

> Super broken, everything related to npm's package lock stuff is broken by design. I've been saying it for years now and it seems people still cling to blindly trusting what corporations say. Because this isn't true. Just because you're experience this effect (which blows), doesn't mean the tool and related tooling are somehow broken. These Regex issues should be fixed, libraries should update to safe versions, things should advance and any incentive we have we should use to make this happen.

taurath 5y ago
Read as: tough, that company says you need to do the work so I demand you do the work.
junon 5y ago
> Just because you're experience this effect (which blows), doesn't mean the tool and related tooling are somehow broken. I've been in the node scene since 0.10. That's around 10 years. My packages have billions of downloads annually. My viewpoint here carries the weight of hours of debug time and frustration and confused users of my code, as well as meeting and knowing the npm staff at the time quite personally, and knowing under which circumstances package lock files were implemented. They are broken. > These Regex issues should be fixed They do, pretty much immediately after they're reported. > libraries should update to safe versions I check all the version ranges of dependent libraries when I push a patch with vuln fixes. They get pulled just fine without needing to update every single package. This is what version ranges are for. > things should advance Yes but this is nebulous and vague and aside from the point. > and any incentive we have we should use to make this happen. I don't see where the disagreement is. This is exactly what happens all the time, nothing is the problem here. I don't get your point. --- Package lock files were designed in a few short days and pushed out prematurely without much review by a single Npm employee (at the time) since they promised it for the v5 release. They were on a time crunch because they were trying to keep with Node.js's next major release timeline, which operates independently of npm's (at least, that's how it was conveyed to me). So this change got pushed out, had an absolute mountain of bugs that took ages to fix (e.g. at one point adding a new dependency would delete your entire node_modules folder), and promised added security when in reality they do nothing of the sort. Instead, they cause subtle caching-related bugs, they add an artifact to source control (which is always code smell in my book), crap up diffs/PRs, cause headaches across platforms, and do very little to help... anything, really. They're super, super broken by design. Yet npm tells you you need them ("please commit this to your repository") and refuses to do basic security things without them (npm audit). So why were they added? IIRC it was because the version resolution was a massive strain on npm's servers, so lockfiles removed the need to fetch tons of version information each time you added another dependency. Oh, and don't even begin to whine about them on Twitter (at the time), lest you be yelled at by the implementor for being ignorant or something. It was a shit show. They add absolutely nothing to the industry.
- esprehn 5y ago
  Hope do you make your builds hermetic and reproducible without package locks?
  - jahewson 5y ago
    Ha ha, our builds are not “hermetic and reproducible” with package locks. Why? Caching.
    - tekknik 5y ago
      How does caching effect this? are you’re devs / build processes not doing clean installs?
  - junon 5y ago
    Your builds are not reproducible with anything related to npm. Neither npm nor any bundler that I'm aware of guarantees that. Unless we're not talking about the same reproducibility property. Also I don't know what "hermetic" means in this context but I doubt it's anything that npm solves correctly.
    - wickedsickeune 5y ago
      There is a way, but it's troublesome. Create a docker image with installed node modules. Save it, and from then onwards you have frozen node modules. If you need a new dependency/updated version you need to create a new image and npm i.
      - junon 5y ago
        That's absolutely no different than just installing and not re-installing. Docker adds nothing in this case. Not re-building doesn't make your build reproducible. It just means that you're... not building. If I save the result of a single iteration of an RNG, I can't claim that the RNG always produces the same result because I saved the result somewhere...
raxxorrax 5y ago
I think it helps to inform the developer about possible issues, but I think in most cases depending on the software this is plainly not relevant and can be ignored. I wouldn't classify it has high severity. Also, It might just not be trivial to develop a regex library that cannot be DDOSed or the mechanism that was declared a vulnerability. Might be nice to be able to tag libraries that should be ignored in audits. Perhaps there is such a function, not really a NPM expert. But if your projects has too many of these "high severity" problems, you probably stop doing them. Still, I think the availability of such audits from the package manager is quite neat. As an embedded dev I think these are certainly luxury problems.
- krageon 5y ago
  If it's too hard for you to develop a library hardened against DoS attacks which are pretty trivial to perform perhaps it's a good thing that npm audit flags your library as containing bugs.
  - junon 5y ago
    A bit pretentious to imply you're better than everyone else at writing regular expressions, so much so that you'd never write one that had exponential time/space complexity on large inputs. Or do you just not understand what regex DoS vulnerabilities are? Either way, you come across very foul and condescending in this comment.
    - krageon 5y ago
      You're attacking a strawman, not me.

taurath 5y ago

Read as: tough, that company says you need to do the work so I demand you do the work.

junon 5y ago

> Just because you're experience this effect (which blows), doesn't mean the tool and related tooling are somehow broken. I've been in the node scene since 0.10. That's around 10 years. My packages have billions of downloads annually. My viewpoint here carries the weight of hours of debug time and frustration and confused users of my code, as well as meeting and knowing the npm staff at the time quite personally, and knowing under which circumstances package lock files were implemented. They are broken. > These Regex issues should be fixed They do, pretty much immediately after they're reported. > libraries should update to safe versions I check all the version ranges of dependent libraries when I push a patch with vuln fixes. They get pulled just fine without needing to update every single package. This is what version ranges are for. > things should advance Yes but this is nebulous and vague and aside from the point. > and any incentive we have we should use to make this happen. I don't see where the disagreement is. This is exactly what happens all the time, nothing is the problem here. I don't get your point. --- Package lock files were designed in a few short days and pushed out prematurely without much review by a single Npm employee (at the time) since they promised it for the v5 release. They were on a time crunch because they were trying to keep with Node.js's next major release timeline, which operates independently of npm's (at least, that's how it was conveyed to me). So this change got pushed out, had an absolute mountain of bugs that took ages to fix (e.g. at one point adding a new dependency would delete your entire node_modules folder), and promised added security when in reality they do nothing of the sort. Instead, they cause subtle caching-related bugs, they add an artifact to source control (which is always code smell in my book), crap up diffs/PRs, cause headaches across platforms, and do very little to help... anything, really. They're super, super broken by design. Yet npm tells you you need them ("please commit this to your repository") and refuses to do basic security things without them (npm audit). So why were they added? IIRC it was because the version resolution was a massive strain on npm's servers, so lockfiles removed the need to fetch tons of version information each time you added another dependency. Oh, and don't even begin to whine about them on Twitter (at the time), lest you be yelled at by the implementor for being ignorant or something. It was a shit show. They add absolutely nothing to the industry.

esprehn 5y ago
Hope do you make your builds hermetic and reproducible without package locks?
- jahewson 5y ago
  Ha ha, our builds are not “hermetic and reproducible” with package locks. Why? Caching.
  - tekknik 5y ago
    How does caching effect this? are you’re devs / build processes not doing clean installs?
- junon 5y ago
  Your builds are not reproducible with anything related to npm. Neither npm nor any bundler that I'm aware of guarantees that. Unless we're not talking about the same reproducibility property. Also I don't know what "hermetic" means in this context but I doubt it's anything that npm solves correctly.
  - wickedsickeune 5y ago
    There is a way, but it's troublesome. Create a docker image with installed node modules. Save it, and from then onwards you have frozen node modules. If you need a new dependency/updated version you need to create a new image and npm i.
    - junon 5y ago
      That's absolutely no different than just installing and not re-installing. Docker adds nothing in this case. Not re-building doesn't make your build reproducible. It just means that you're... not building. If I save the result of a single iteration of an RNG, I can't claim that the RNG always produces the same result because I saved the result somewhere...

raxxorrax 5y ago

I think it helps to inform the developer about possible issues, but I think in most cases depending on the software this is plainly not relevant and can be ignored. I wouldn't classify it has high severity. Also, It might just not be trivial to develop a regex library that cannot be DDOSed or the mechanism that was declared a vulnerability. Might be nice to be able to tag libraries that should be ignored in audits. Perhaps there is such a function, not really a NPM expert. But if your projects has too many of these "high severity" problems, you probably stop doing them. Still, I think the availability of such audits from the package manager is quite neat. As an embedded dev I think these are certainly luxury problems.

krageon 5y ago
If it's too hard for you to develop a library hardened against DoS attacks which are pretty trivial to perform perhaps it's a good thing that npm audit flags your library as containing bugs.
- junon 5y ago
  A bit pretentious to imply you're better than everyone else at writing regular expressions, so much so that you'd never write one that had exponential time/space complexity on large inputs. Or do you just not understand what regex DoS vulnerabilities are? Either way, you come across very foul and condescending in this comment.
  - krageon 5y ago
    You're attacking a strawman, not me.