5 ms·

What the npm ecosystem really needs is something like the distinction between Ubuntu's "main" and "universe" repositories, so that you have a smaller subset of

What the npm ecosystem really needs is something like the distinction between Ubuntu's "main" and "universe" repositories, so that you have a smaller subset of known-good packages with harmonized transitive dependencies, stronger centralized curation, a lot more direct scrutiny, tighter control over versioning, and some party that is responsible for addressing vulnerabilities and ensuring appropriate maintainership. If you could rely on that for core functionality and only needed to go outside of it for the long tail of more specialized things, it would be a lot cleaner and safer than what we do today. Linux distros have many, many years of experience curating packages at scale. It is crazy to me that the node community just totally disregards all of the best practices learned by the Linux distros in favor of practices that are lazy and unambiguously dangerous.

bostik 7y ago
Intel and Nokia were trying to add repository trust levels to rpm back in 2010/2011. To this day, if you can add a new repository, it is fully trusted and can provide trojaned updates to any core package. Bump the version number to something higher than what the core distro would use and you can persist for quite a long time. I remember pointing this out at a Nokia project (would have been no later than early 2008) as an attack vector via apt repositories. We wanted to prevent a rogue or compromised repository from being able to provide an update to, say, glibc or libstdc++. Never got to work on that - Elopcalypse took it all down.
- hnarn 7y ago
  > To this day, if you can add a new repository, it is fully trusted and can provide trojaned updates to any core package. Invent idiot-proof security and someone will invent a better idiot. Even if you have a "trusted=1" flag for repos, you can be sure people will set external repos to be just as trusted as the core ones without a second thought as soon as they stop them from doing what they want to do (whatever that is, or however in/sane it is). It's maybe not the most elegant option, but for security-conscious sysadmins you could already today disable all repos ad hoc and run your "system wide update" only from certain whitelisted repos, while still allowing specific one-off package installs from third party repos. By adding a third party repo you have already said you trust that repo to install software, probably as root, on your machine. It doesn't get much worse than that from a security standpoint anyway right, will "trust flags" really make much of a difference to the wider security issue? Sorry if I'm making too many assumptions here or thinking out loud.
  - marcosdumay 7y ago
    Take a look at Debian's package pining. It's not a security feature, but a convenience one for forbidding repositories from updating the packages from the ones you trust more (with correctness). Yes, a binary flag would be useless. What the GP wants also does not add any security, but it's an important part of a stable system that language-specific repositories have been overlooking for a while.
  - bostik 7y ago
    > Even if you have a "trusted=1" flag for repos, you can be sure people will set external repos to be just as trusted as the core ones That was the thing - you couldn't. I don't remember which way the level hierarchy went, but the idea was that rpm could not be executed directly even by root (LSM prevented that), and all package installation logic was confined to Zypper/libzypp. Packages and repos could declare their security level. Trying to pull in an upgrade to a LEVEL(core) package from a LEVEL(media) repository would fail. Zypper simply would not allow to install a package from a lower-security repository over an installed package that had come from a higher-security origin. Manually added LEVEL(core) repos would have been ignored. There were a few layers of applied crypto, package signatures and immutable keyrings involved too. I had a prototype Zypper/libzypp with package-level signature verification almost ready just before my vacation and planned to polish it to an RFC after coming back. Best laid plans... During those two weeks the entirety of Nokia's linux development had been axed, maemo killed and thousands of good embedded systems engineers were suddenly looking for new jobs. Funny thing, though. Zypper's source code was almost pleasant to work with after having seen the Lovecraftian horrors that made up libapt.
- pferde 7y ago
  Debian-based distros can use package pinning to this effect, but since it is an opt-in feature for advanced users, it is not a full solution. I'm mentioning it here just for sake of completeness.
blobs 7y ago
Totally second this. I see novice web developers writing in Typescript in an effort to be more type save and at the same time using hundreds of npm packages that are often written by amateurs that make basic mistakes. Just take an average startups web project node_modules directory, what's inside there? Hundreds and hundreds of packages of which most are dependencies of other packages. Anyone could have written it! Novice devs swear by using Typescript, but at the same time using hundreds of black boxes that can easily contain stuff way more damaging that a string applied to a number.. Remember left-pad? That was an easy one to fix, but still caused damage at large scale. What about a vulnerability in a larger and more complex package, owned by some bad party?
- jannes 7y ago
  For me the biggest problem is webpack. Do you know an alternative bundler with less dependencies and typescript support? I already use webpack without the webpack-cli package in most of my projects in order to avoid the extra dependencies. I'm currently investigating rollup because it only has 3 dependencies (2 of which are @types)
  - rmilejczz 7y ago
    I recommend rollup for libraries and parcel for web apps.
    - jannes 7y ago
      Parcel has 57 dependencies... (not even counting transient dependencies) That sort of defeats the purpose of switching from webpack. And for libraries: Why do you even need a bundler? Can you not distribute the library as multiple files?
      - rmilejczz 7y ago
        Yeah I made some incorrect assumptions about the nature of parcel, foolish mistake on my part. For your other question it depends on your library. If you’re only targeting ESM or CJS sure, if you need a UMD or want separate outputs for separate targets a bundler like rollup can help prevent a lot of pointless boilerplate.
pjc50 7y ago
> It is crazy to me that the node community just totally disregards all of the best practices learned by the Linux distros in favor of practices that are lazy and unambiguously dangerous. "Move fast and break things." The whole Javascript ecosystem depends on high churn rapid adoption of "new" technologies. Stopping to check things would slow this down. Higher-QA technologies get outcompeted by lower-QA technologies.
- zbentley 7y ago
  > Higher-QA technologies get outcompeted by lower-QA technologies. While both are still relatively strong, it seems that JS's recent (~8y) surge in popularity over established mainstays like Java may stands counter to that point.
  - pjc50 7y ago
    JS has a unique position in being the only supported language in browsers after Java Applets, Flash, and ActiveX were all killed off (mostly for valid reasons). Given that the frontend has to be written in Javascript, it's tempting to use it for the backend as well.
  - xkcd-sucks 7y ago
    The software industry is where quality goes to die, if anything the surge in popularity proves the point
lachenmayer 7y ago
In the React Native ecosystem, the Expo project https://expo.io https://expo.io is doing a great job at providing this kind of centralized curation, with monthly release cycles, constistent APIs and dealing with the version hell that makes "raw" React Native such a pain to work with.
dsfyu404ed 7y ago
>Linux distros have many, many years of experience curating packages at scale. It is crazy to me that the node community just totally disregards all of the best practices learned by the Linux distros in favor of practices that are lazy and unambiguously dangerous. "Screw your wheel, we can invent invent our own" is not exactly a surprising strategy when it comes from the language/development community that is mostly building framework after framework that gets abandoned in a year or two.

bostik 7y ago

Intel and Nokia were trying to add repository trust levels to rpm back in 2010/2011. To this day, if you can add a new repository, it is fully trusted and can provide trojaned updates to any core package. Bump the version number to something higher than what the core distro would use and you can persist for quite a long time. I remember pointing this out at a Nokia project (would have been no later than early 2008) as an attack vector via apt repositories. We wanted to prevent a rogue or compromised repository from being able to provide an update to, say, glibc or libstdc++. Never got to work on that - Elopcalypse took it all down.

hnarn 7y ago
> To this day, if you can add a new repository, it is fully trusted and can provide trojaned updates to any core package. Invent idiot-proof security and someone will invent a better idiot. Even if you have a "trusted=1" flag for repos, you can be sure people will set external repos to be just as trusted as the core ones without a second thought as soon as they stop them from doing what they want to do (whatever that is, or however in/sane it is). It's maybe not the most elegant option, but for security-conscious sysadmins you could already today disable all repos ad hoc and run your "system wide update" only from certain whitelisted repos, while still allowing specific one-off package installs from third party repos. By adding a third party repo you have already said you trust that repo to install software, probably as root, on your machine. It doesn't get much worse than that from a security standpoint anyway right, will "trust flags" really make much of a difference to the wider security issue? Sorry if I'm making too many assumptions here or thinking out loud.
- marcosdumay 7y ago
  Take a look at Debian's package pining. It's not a security feature, but a convenience one for forbidding repositories from updating the packages from the ones you trust more (with correctness). Yes, a binary flag would be useless. What the GP wants also does not add any security, but it's an important part of a stable system that language-specific repositories have been overlooking for a while.
- bostik 7y ago
  > Even if you have a "trusted=1" flag for repos, you can be sure people will set external repos to be just as trusted as the core ones That was the thing - you couldn't. I don't remember which way the level hierarchy went, but the idea was that rpm could not be executed directly even by root (LSM prevented that), and all package installation logic was confined to Zypper/libzypp. Packages and repos could declare their security level. Trying to pull in an upgrade to a LEVEL(core) package from a LEVEL(media) repository would fail. Zypper simply would not allow to install a package from a lower-security repository over an installed package that had come from a higher-security origin. Manually added LEVEL(core) repos would have been ignored. There were a few layers of applied crypto, package signatures and immutable keyrings involved too. I had a prototype Zypper/libzypp with package-level signature verification almost ready just before my vacation and planned to polish it to an RFC after coming back. Best laid plans... During those two weeks the entirety of Nokia's linux development had been axed, maemo killed and thousands of good embedded systems engineers were suddenly looking for new jobs. Funny thing, though. Zypper's source code was almost pleasant to work with after having seen the Lovecraftian horrors that made up libapt.
pferde 7y ago
Debian-based distros can use package pinning to this effect, but since it is an opt-in feature for advanced users, it is not a full solution. I'm mentioning it here just for sake of completeness.

blobs 7y ago

Totally second this. I see novice web developers writing in Typescript in an effort to be more type save and at the same time using hundreds of npm packages that are often written by amateurs that make basic mistakes. Just take an average startups web project node_modules directory, what's inside there? Hundreds and hundreds of packages of which most are dependencies of other packages. Anyone could have written it! Novice devs swear by using Typescript, but at the same time using hundreds of black boxes that can easily contain stuff way more damaging that a string applied to a number.. Remember left-pad? That was an easy one to fix, but still caused damage at large scale. What about a vulnerability in a larger and more complex package, owned by some bad party?

jannes 7y ago
For me the biggest problem is webpack. Do you know an alternative bundler with less dependencies and typescript support? I already use webpack without the webpack-cli package in most of my projects in order to avoid the extra dependencies. I'm currently investigating rollup because it only has 3 dependencies (2 of which are @types)
- rmilejczz 7y ago
  I recommend rollup for libraries and parcel for web apps.
  - jannes 7y ago
    Parcel has 57 dependencies... (not even counting transient dependencies) That sort of defeats the purpose of switching from webpack. And for libraries: Why do you even need a bundler? Can you not distribute the library as multiple files?
    - rmilejczz 7y ago
      Yeah I made some incorrect assumptions about the nature of parcel, foolish mistake on my part. For your other question it depends on your library. If you’re only targeting ESM or CJS sure, if you need a UMD or want separate outputs for separate targets a bundler like rollup can help prevent a lot of pointless boilerplate.

pjc50 7y ago

> It is crazy to me that the node community just totally disregards all of the best practices learned by the Linux distros in favor of practices that are lazy and unambiguously dangerous. "Move fast and break things." The whole Javascript ecosystem depends on high churn rapid adoption of "new" technologies. Stopping to check things would slow this down. Higher-QA technologies get outcompeted by lower-QA technologies.

zbentley 7y ago
> Higher-QA technologies get outcompeted by lower-QA technologies. While both are still relatively strong, it seems that JS's recent (~8y) surge in popularity over established mainstays like Java may stands counter to that point.
- pjc50 7y ago
  JS has a unique position in being the only supported language in browsers after Java Applets, Flash, and ActiveX were all killed off (mostly for valid reasons). Given that the frontend has to be written in Javascript, it's tempting to use it for the backend as well.
- xkcd-sucks 7y ago
  The software industry is where quality goes to die, if anything the surge in popularity proves the point

lachenmayer 7y ago

In the React Native ecosystem, the Expo project https://expo.io https://expo.io is doing a great job at providing this kind of centralized curation, with monthly release cycles, constistent APIs and dealing with the version hell that makes "raw" React Native such a pain to work with.

dsfyu404ed 7y ago

>Linux distros have many, many years of experience curating packages at scale. It is crazy to me that the node community just totally disregards all of the best practices learned by the Linux distros in favor of practices that are lazy and unambiguously dangerous. "Screw your wheel, we can invent invent our own" is not exactly a surprising strategy when it comes from the language/development community that is mostly building framework after framework that gets abandoned in a year or two.