10 ms·

So we need gpg signed packages :> And... all packages should be namespaced under the author who published them. And... I kind of want to say "once it's publis

So we need gpg signed packages :> And... all packages should be namespaced under the author who published them. And... I kind of want to say "once it's published, it's forever".

andrewfong 11y ago
GPG isn't strictly necessary if you trust NPM's authentication system (of course, that's a big "if" for many folks). Publish forever (barring requests to remove for legal compliance or whatever) is a good idea. Or at the very least, it should be a default option. And if you install a dependency that isn't "publish forever", you should get a warning.
cookrn 11y ago
What if such a system was implemented using IPFS[0] (or similar) for storage? [0] https://github.com/ipfs/ipfs https://github.com/ipfs/ipfs
- smaddox 11y ago
  I'm surprised all package managers don't use an IPFS-like system that uses immutable state with mutable labels and namespaces. Now that IPFS exists, and provides distributed hosting, it's even easier.
  - nikolay 11y ago
    No signatures, no (at least!) an IPFS mirror as a backup option - how can one trust NPM or the likes?!
  - kylecordes 11y ago
    NPM doesn't even have immutable versions. Many would love to see this improved.
    - jakub_g 11y ago
      What you mean by that? It used to be possible to republish a version (it broke our build when a dep was republished with a breaking change, that's how I learnt about it) but this was fixed some 2-3 years ago IIRC
  - viraptor 11y ago
    As much as I agree, IPFS is still very much under construction and I don't think any known package managers got started after IPFS was reliable. You can experiment with ipfs-backed git remotes though. That's already possible.
    - jplur 11y ago
      Yes the IPFS implementation might change but not the content multihash addressing. Linking to data with those addresses is the generic 'package management' that solves all these problems (references to mutable data at mutable origins, circular dependencies, data caching, namespace conflicts). The specifics of resolving links will hopefully be something we don't think about much. I've played around with ipfs.js for resolving links into eval'd js at runtime and imagine a npm replacement would be pretty trivial. The IPFS peer to peer swarm seems stable to me but you could also dump all your hash-named files into a s3 bucket or something as a fallback repo.
    - noffle 11y ago
      gx is a generic package manager on top of IPFS that uses git-style hooks for adding per-language support. It's already being used to manage dependencies on the go-ipfs project: https://github.com/whyrusleeping/gx https://github.com/whyrusleeping/gx Bonus: there's also a IPFS git remote implementation! https://github.com/cryptix/git-remote-ipfs https://github.com/cryptix/git-remote-ipfs
- Ericson2314 11y ago
  Somewhat related, I just coincidentally stumbled upon https://github.com/alexanderGugel/ied https://github.com/alexanderGugel/ied . "Upcomming Nix-inspired features", to paraphrase their README, could well prevent this debacle. (And btw, We Nix users very much do hope to start using IPFS :).)
- gritzko 11y ago
  Git has crypto for a reason. Every package manager must have it too.
- ArneBab 11y ago
  There’s already apt-get over Freenet: http://127.0.0.1:8888/freenet:USK@oRy7ltZLJM-w-kcOBdiZS1pAA8P-BxZ3BPiiqkmfk0E,6a1KFG6S-Bwp6E-MplW52iH~Y3La6GigQVQDeMjI6rg,AQACAAE/deb.mempo.org/64/#install_apt_freenet http://127.0.0.1:8888/freenet:USK@oRy7ltZLJM-w-kcOBdiZS1pAA8...
- niutech 11y ago
  There is already gx package manager: https://github.com/whyrusleeping/gx https://github.com/whyrusleeping/gx
callmevlad 11y ago
Looks like the npm team will not be removing the ability to unpublish packages - see reply by core committer "othiym23" on https://github.com/npm/npm/pull/12017 https://github.com/npm/npm/pull/12017
- voltagex_ 11y ago
  https://github.com/npm/npm/pull/12017#issuecomment-200131039 https://github.com/npm/npm/pull/12017#issuecomment-200131039 There are a lot of problems with NuGet, but they got this right. I do wish there was a way to mark a package as deprecated, though. For ages, there was an unofficial JQuery package that was years out of date.
  - kemitchell 11y ago
    `npm help deprecate`
    - voltagex_ 11y ago
      Yep, unfortunately the same does not exist for NuGet.
- techdragon 11y ago
  Read the whole thread. Rather concerned by the final one. "Locking temporarily" to get away from the discussion? That feels sort of like the online discussion equivalent of sticking your fingers in your ears and going "la la la I'm not listening". I don't expect someone in their position to be unable to ignore a conversation and "take a break" but I would expect them to be capable of doing so without resorting to "suppressing" the ongoing group discussion.
  - kbenson 11y ago
    Or it's a "we're discussing internally, and would rather not deal with the shit-show that Github issues becomes once the issue becomes politicized and rampant misinformation and misguided activism take over."[1] There will be plenty of time for people to froth at the mouth and complain that they chose one way or the other once they've made a clear decision, which as of the locking the thread to collaborators, they have not (the current thinking has been outlined, but they said they are thinking about it). 1: See the recent systemd efivarfs issue at https://github.com/systemd/systemd/issues/2402 https://github.com/systemd/systemd/issues/2402 and associated HN discussions, which was solved through a kernel fix. Pitchforks abound.
profmonocle 11y ago
> And... I kind of want to say "once it's published, it's forever". This is effectively the norm with more traditional, curated package managers. Say I release a piece of open source software, and some Linux distro adds it to their package manager. Under a typical open source license, I have no legal right to ask them to stop distributing it. They can just say "sorry, you licensed this code to us under X license and we're distributing it under those terms. Removing it would break our users' systems, so we won't do it." The difference is that NPM is self-service - publishers add packages themselves, and NPM has chosen to also provide a self-service option to remove packages. I honestly wouldn't have a problem with them removing that option, and only allowing packages to be removed by contacting support with a good reason. (Accidental private info disclosure, copyright violation, severe security bug, etc.)
- echelon 11y ago
  I honestly wouldn't have a problem with them removing that option, and only allowing packages to be removed by contacting support with a good reason. (Accidental private info disclosure, copyright violation, severe security bug, etc.) Even Rust's Cargo won't allow you to revoke secrets [1]. I think this is the correct policy. [1] http://doc.crates.io/crates-io.html#cargo-yank http://doc.crates.io/crates-io.html#cargo-yank
  - drewgross 11y ago
    Aside from secrets there is also sensitive data. If someone accidentally uploads some personal information, they need a way to remove it if, say, they receive a court order ordering them to remove it.
    - Joky 11y ago
      I'm not sure how the court could force you to do something you can't possibly do...
      - jzwinck 11y ago
dantiberian 11y ago
This is what happens with Clojars. It is possible to remove packages, but it requires a manual email to the admins, along with an explanation, e.g. published internal lib accidentally. This prevents scenarios like this, but also cases where people want to 'clean up' things they no longer need, even though others are depending on them.
amadeusw 11y ago
That's a good idea, but what if Kik lawyers come knocking on your door saying that you're breaking the law and this package cannot stay there forever, or for any moment longer?
- deleted 11y ago
  [deleted]
edem 11y ago
Well it shouldn't be hard...oh wait. This is javascript. Good luck then. :)
deleted 11y ago
[deleted]
coroutines 11y ago
I think I'd just want to add that namepsacing by author doesn't entirely fix the problem. For the fewer instances where there is a collision, we still have this issue with lawyers asserting trademarks. "Would the real Slim Shady please stand up?" We want multiple 'kik's and multiple Shady's simultaneously. So record the gpg sig of the author in package.json, and filter the name + semver against just their published modules when updating. Depending on how unique you need to be: npm install <module> --save npm install <author>/<module> --save npm install <gpg>/<module> --save On a side note, npm-search really sucks. It lacks a lot of fine-grained filtering. I'd love to be able to search by tags, or exclude results with an incompatible license, or even prioritize results by specified dependencies. npm-search needs love.

So we need gpg signed packages :> And... all packages should be namespaced under the author who published them. And... I kind of want to say "once it's published, it's forever".

andrewfong 11y ago

GPG isn't strictly necessary if you trust NPM's authentication system (of course, that's a big "if" for many folks). Publish forever (barring requests to remove for legal compliance or whatever) is a good idea. Or at the very least, it should be a default option. And if you install a dependency that isn't "publish forever", you should get a warning.

cookrn 11y ago

What if such a system was implemented using IPFS[0] (or similar) for storage? [0] https://github.com/ipfs/ipfs https://github.com/ipfs/ipfs

smaddox 11y ago
I'm surprised all package managers don't use an IPFS-like system that uses immutable state with mutable labels and namespaces. Now that IPFS exists, and provides distributed hosting, it's even easier.
- nikolay 11y ago
  No signatures, no (at least!) an IPFS mirror as a backup option - how can one trust NPM or the likes?!
- kylecordes 11y ago
  NPM doesn't even have immutable versions. Many would love to see this improved.
  - jakub_g 11y ago
    What you mean by that? It used to be possible to republish a version (it broke our build when a dep was republished with a breaking change, that's how I learnt about it) but this was fixed some 2-3 years ago IIRC
- viraptor 11y ago
  As much as I agree, IPFS is still very much under construction and I don't think any known package managers got started after IPFS was reliable. You can experiment with ipfs-backed git remotes though. That's already possible.
  - jplur 11y ago
    Yes the IPFS implementation might change but not the content multihash addressing. Linking to data with those addresses is the generic 'package management' that solves all these problems (references to mutable data at mutable origins, circular dependencies, data caching, namespace conflicts). The specifics of resolving links will hopefully be something we don't think about much. I've played around with ipfs.js for resolving links into eval'd js at runtime and imagine a npm replacement would be pretty trivial. The IPFS peer to peer swarm seems stable to me but you could also dump all your hash-named files into a s3 bucket or something as a fallback repo.
  - noffle 11y ago
    gx is a generic package manager on top of IPFS that uses git-style hooks for adding per-language support. It's already being used to manage dependencies on the go-ipfs project: https://github.com/whyrusleeping/gx https://github.com/whyrusleeping/gx Bonus: there's also a IPFS git remote implementation! https://github.com/cryptix/git-remote-ipfs https://github.com/cryptix/git-remote-ipfs
Ericson2314 11y ago
Somewhat related, I just coincidentally stumbled upon https://github.com/alexanderGugel/ied https://github.com/alexanderGugel/ied . "Upcomming Nix-inspired features", to paraphrase their README, could well prevent this debacle. (And btw, We Nix users very much do hope to start using IPFS :).)
gritzko 11y ago
Git has crypto for a reason. Every package manager must have it too.
ArneBab 11y ago
There’s already apt-get over Freenet: http://127.0.0.1:8888/freenet:USK@oRy7ltZLJM-w-kcOBdiZS1pAA8P-BxZ3BPiiqkmfk0E,6a1KFG6S-Bwp6E-MplW52iH~Y3La6GigQVQDeMjI6rg,AQACAAE/deb.mempo.org/64/#install_apt_freenet http://127.0.0.1:8888/freenet:USK@oRy7ltZLJM-w-kcOBdiZS1pAA8...
niutech 11y ago
There is already gx package manager: https://github.com/whyrusleeping/gx https://github.com/whyrusleeping/gx

callmevlad 11y ago

Looks like the npm team will not be removing the ability to unpublish packages - see reply by core committer "othiym23" on https://github.com/npm/npm/pull/12017 https://github.com/npm/npm/pull/12017

voltagex_ 11y ago
https://github.com/npm/npm/pull/12017#issuecomment-200131039 https://github.com/npm/npm/pull/12017#issuecomment-200131039 There are a lot of problems with NuGet, but they got this right. I do wish there was a way to mark a package as deprecated, though. For ages, there was an unofficial JQuery package that was years out of date.
- kemitchell 11y ago
  `npm help deprecate`
  - voltagex_ 11y ago
    Yep, unfortunately the same does not exist for NuGet.
techdragon 11y ago
Read the whole thread. Rather concerned by the final one. "Locking temporarily" to get away from the discussion? That feels sort of like the online discussion equivalent of sticking your fingers in your ears and going "la la la I'm not listening". I don't expect someone in their position to be unable to ignore a conversation and "take a break" but I would expect them to be capable of doing so without resorting to "suppressing" the ongoing group discussion.
- kbenson 11y ago
  Or it's a "we're discussing internally, and would rather not deal with the shit-show that Github issues becomes once the issue becomes politicized and rampant misinformation and misguided activism take over."[1] There will be plenty of time for people to froth at the mouth and complain that they chose one way or the other once they've made a clear decision, which as of the locking the thread to collaborators, they have not (the current thinking has been outlined, but they said they are thinking about it). 1: See the recent systemd efivarfs issue at https://github.com/systemd/systemd/issues/2402 https://github.com/systemd/systemd/issues/2402 and associated HN discussions, which was solved through a kernel fix. Pitchforks abound.

profmonocle 11y ago

> And... I kind of want to say "once it's published, it's forever". This is effectively the norm with more traditional, curated package managers. Say I release a piece of open source software, and some Linux distro adds it to their package manager. Under a typical open source license, I have no legal right to ask them to stop distributing it. They can just say "sorry, you licensed this code to us under X license and we're distributing it under those terms. Removing it would break our users' systems, so we won't do it." The difference is that NPM is self-service - publishers add packages themselves, and NPM has chosen to also provide a self-service option to remove packages. I honestly wouldn't have a problem with them removing that option, and only allowing packages to be removed by contacting support with a good reason. (Accidental private info disclosure, copyright violation, severe security bug, etc.)

echelon 11y ago
I honestly wouldn't have a problem with them removing that option, and only allowing packages to be removed by contacting support with a good reason. (Accidental private info disclosure, copyright violation, severe security bug, etc.) Even Rust's Cargo won't allow you to revoke secrets [1]. I think this is the correct policy. [1] http://doc.crates.io/crates-io.html#cargo-yank http://doc.crates.io/crates-io.html#cargo-yank
- drewgross 11y ago
  Aside from secrets there is also sensitive data. If someone accidentally uploads some personal information, they need a way to remove it if, say, they receive a court order ordering them to remove it.
  - Joky 11y ago
    I'm not sure how the court could force you to do something you can't possibly do...
    - jzwinck 11y ago

dantiberian 11y ago

This is what happens with Clojars. It is possible to remove packages, but it requires a manual email to the admins, along with an explanation, e.g. published internal lib accidentally. This prevents scenarios like this, but also cases where people want to 'clean up' things they no longer need, even though others are depending on them.

amadeusw 11y ago

That's a good idea, but what if Kik lawyers come knocking on your door saying that you're breaking the law and this package cannot stay there forever, or for any moment longer?

deleted 11y ago
[deleted]

edem 11y ago

Well it shouldn't be hard...oh wait. This is javascript. Good luck then. :)

deleted 11y ago

[deleted]

coroutines 11y ago

I think I'd just want to add that namepsacing by author doesn't entirely fix the problem. For the fewer instances where there is a collision, we still have this issue with lawyers asserting trademarks. "Would the real Slim Shady please stand up?" We want multiple 'kik's and multiple Shady's simultaneously. So record the gpg sig of the author in package.json, and filter the name + semver against just their published modules when updating. Depending on how unique you need to be: npm install <module> --save npm install <author>/<module> --save npm install <gpg>/<module> --save On a side note, npm-search really sucks. It lacks a lot of fine-grained filtering. I'd love to be able to search by tags, or exclude results with an incompatible license, or even prioritize results by specified dependencies. npm-search needs love.