17 ms·

Show HN: ied – an alternative package manager for Node

86 points by blubbi2 11y ago

crabasa 11y ago
> The easiest way to install ied is using npm Bootstrapping ftw.
- blubbi2 11y ago
  The initial set of dependencies is being installed via npm, the it installs its own dependencies via ied if told so: https://github.com/alexanderGugel/ied#installation https://github.com/alexanderGugel/ied#installation This is a "cool" feature during development, since it's a nice proof of concept. Originally I checked in the node_modules directory, but then reddit was shitting on me as usual (yes, you shouldn't check in node_modules in an actual app, but this is PACKAGE MANAGER!). As far as I know, npm has also its own dependencies check in + a ton of packages as tarballs for tests, so I might do that later.
  - dkns 11y ago
    Dude, don't listen to reddit when it comes to managing your own open source project.
    - jayflux 11y ago
      reddit gets mentioned a lot in these parts. Is there a subreddit i don't know off?
    - mikekchar 11y ago
      I was just thinking about that yesterday. I write my own projects so that I don't have to dance the political dance and make my colleagues happy. My own projects are for exploring my own ideas. Advice is always appreciated, but if someone wants to dictate how I write code in my own projects, they better damn well pay me (a lot). To the OP: Don't let people bully you. Many people have strong ideas and will want you to do things their way. You aren't going to make everyone happy, though. Somebody will be pissed off no matter what you do (if you are popular enough -- normally people won't pay any attention to you ;-) ). "Because that's what I want to do" is a completely valid reason for any decision on your own project.
macmac 11y ago
Unfortunate naming ie https://en.wikipedia.org/wiki/Improvised_explosive_device https://en.wikipedia.org/wiki/Improvised_explosive_device
- blubbi2 11y ago
  Damn. Didn't think of that. In fact I literally just renamed it: https://github.com/alexanderGugel/ied/commit/84628b3c871c85d7424d9d4a3813b19d02721bc6 https://github.com/alexanderGugel/ied/commit/84628b3c871c85d... Originally it was called mpm, but I figured that would have been pretty confusing, but it looks like the new name isn't necessarily better. I'm pretty terrible at naming. Any suggestions are more than welcome!
  - diggan 11y ago
    I think it's fine as it is. Could have chosen a worse name for sure! And, managing dependencies is like dealing with explosives anyways...
    - JoshTriplett 11y ago
      > And, managing dependencies is like dealing with explosives anyways... Only funny for anyone who hasn't been affected by one, or had friends or family who were. Still better to avoid names with negative connotations (and search for them first to check). > Any suggestions are more than welcome! A few ideas: bpm - Better Package Manager edge - the thing that connects nodes jpm - Javascript Package Manager ppm - Peer Package Manager fpm - Functional Package Manager ayp - All Your Packages nnm - New Node Manager
      - inopinatus 11y ago
        yanm
        xixixao 11y ago
        yapm
blubbi2 11y ago
Hey everyone! I made this. I'm happy to answer any questions, but please bear in mind that this is a WIP. There is still a lot of work to be done, although feature parity with npm is not the goal. Upcoming features are: * Nix-like rollbacks * built in registry server * discovery + installation via BitTorrent DHT Would love to get any feedback!
- diggan 11y ago
  For the discovery and installation, you might be interested in looking into using IPFS for it. Kind of gives you all the features you want out-of-the-box. Also, another guy is working on mirroring the npm registry in IPFS, might be interesting for you: https://github.com/diasdavid/registry-mirror https://github.com/diasdavid/registry-mirror
  - blubbi2 11y ago
    This looks very interesting! Also related: https://github.com/dominictarr/npmd https://github.com/dominictarr/npmd
  - daviddias 11y ago
    Hi Alexander, this is really great! I've started `registry-mirror` to demonstrate how a Content-Addressed file structure and P2P discovery, can bring a lot of speed improvements, specially when the bottleneck is low bandwidth/latency to the backbone, by connecting to more local peers that have the content that we are looking for. Right now, the goal with `registry-mirror` is to have a very large IPFS node in the network with the entire npm and that keeps replicating it, while end user machines only download the modules they need (and if they agree, provide them to the network as well). Each end user will be able to get the latest state of the registry, through a IPNS hash, a mutable pointer, that changes each time the mirror is updated. The nix package manager model layers perfectly on top of IPFS' MerkleDAG (https://github.com/ipfs/specs/tree/master/merkledag https://github.com/ipfs/specs/tree/master/merkledag), it can be a very awesome transport for ied. An example of a package manager that uses IPFS to distribute the packages is GX https://github.com/whyrusleeping/gx https://github.com/whyrusleeping/gx - Still a WIP. If this is interesting to you, join us at IRC Freenode #IPFS, it would be great to bounce more ideas! :)
drinchev 11y ago
That seems so cool. One question is how compatible is with npm? Can I use it as a drop-in replacement ( talking about the CLI tool )?
- blubbi2 11y ago
  Mostly, yes, but there are some features that are still missing: `ied publish` and `ied version` is coming next week. I'm also thinking of adding scoped modules, but I'm not sure about that yet. You can also configure a private npm registry to be used: https://github.com/alexanderGugel/ied/blob/master/lib/config.js#L9 https://github.com/alexanderGugel/ied/blob/master/lib/config...
gfosco 11y ago
I never thought to myself, "wow, npm is slow." Is that really the core problem this is solving?
- broodbucket 11y ago
  Have you used any other package manager? I'm not completely sure if it's just Node's dependency hell, but npm feels so incredibly slow compared to Cargo or bundler or something.
  - mbell 11y ago
    Never profiled it but I would guess that is largely due to the npm's main problem: It doesn't locked dependency versions. As a result it probably builds the dependency graph a lot more than it should.
    - mikekchar 11y ago
      For me this is the main thing I'd like to see fixed. Not only does it build more than it need to, it can result in broken builds. When my dependency doesn't update, I don't expect its dependencies to update (especially when I don't have control over setting the versions of the dependencies of dependencies). This has bitten me more than once.
      - mbell 11y ago
        It really baffles me that most dependency management systems get this wrong. You'd think it would be consideration #1.
- maaaats 11y ago
  I work on a fairly large frontend project where everything is modularized. So we have maybe 50+ dependencies that are our own. These constantly change. I detest changing branches and having to run npm to get the correct version of stuff. Takes minutes before it even starts installing packages.
- 11y ago
nadocrew 11y ago
You say you are creating a more performant NPM. Is it impossible to fix the current NPM? Why not submit fixes back to NPM? Why create a new project?
- glenjamin 11y ago
  Works is ongoing, but npm is pretty complicated at this point https://github.com/npm/npm/issues/10380 https://github.com/npm/npm/issues/10380
- blubbi2 11y ago
  Its install process is very different from the one npm uses. E.g. just look into the node_modules directory produced by npm vs the one produced by ied. Also the way it wires up dependencies (using symlinks) has been called "non humane design" by npm... so I thought it would be easier to just start a new project... it's not a lot of code/logic needed there actually... :)
  - nailer 11y ago
    npm v3 already uses the 'flat by default' design, so the node_modules folders aren't that different.
    - blubbi2 11y ago
      They are extremely different. I made a quick comparison here: https://gist.github.com/alexanderGugel/a10ed5655d366875a280 https://gist.github.com/alexanderGugel/a10ed5655d366875a280 Basically ied uses symlinks in order to resolve circular dependencies, while ied exploits the fact that require "falls back" in the directory structure.
      - nailer 11y ago
        Understood. Though I believe your comment (which compared ied with itself) should actually read: > Basically ied uses symlinks in order to resolve circular dependencies, while npm exploits the fact that require "falls back" in the directory structure.
jmandzik 11y ago
Cloned a popular project (babel) and on a MacBook Pro: npm install 52.28s user 8.08s system 73% cpu 1:22.41 total ied install 10.22s user 4.36s system 142% cpu 10.230 total Impressive.
- blubbi2 11y ago
  Awesome! Glad it worked that well! Please let me know if you run into any bugs: https://github.com/alexanderGugel/ied https://github.com/alexanderGugel/ied
  - jmandzik 11y ago
    Happy to open an issue, but any plans on supporting git+ssh urls? For work projects, we have some modules installed via git and when I tried to install, I got the same error described here: https://github.com/alexanderGugel/ied/issues/2 https://github.com/alexanderGugel/ied/issues/2 Very promising project!
    - tracker1 11y ago
      +1 here, git+ssh and git+http(s) are essential imho... its' the easiest way to sidestep the need for an npm server for some internal projects/libraries.
      - blubbi2 11y ago
        Agreed. It's definitely on the roadmap. Should be done in a couple of weeks.
xrstf 11y ago
> produces a flat node_modules directory Finally. npm's node_modules makes Node on Windows unbearable from time to time. Shut up and take my money!
- nailer 11y ago
  npm v3 already does this by default.
js_throw_away 11y ago
Guys NPM is not the problem, Javascript is.

86 points by blubbi2 11y ago

crabasa 11y ago

> The easiest way to install ied is using npm Bootstrapping ftw.

blubbi2 11y ago
The initial set of dependencies is being installed via npm, the it installs its own dependencies via ied if told so: https://github.com/alexanderGugel/ied#installation https://github.com/alexanderGugel/ied#installation This is a "cool" feature during development, since it's a nice proof of concept. Originally I checked in the node_modules directory, but then reddit was shitting on me as usual (yes, you shouldn't check in node_modules in an actual app, but this is PACKAGE MANAGER!). As far as I know, npm has also its own dependencies check in + a ton of packages as tarballs for tests, so I might do that later.
- dkns 11y ago
  Dude, don't listen to reddit when it comes to managing your own open source project.
  - jayflux 11y ago
    reddit gets mentioned a lot in these parts. Is there a subreddit i don't know off?
  - mikekchar 11y ago
    I was just thinking about that yesterday. I write my own projects so that I don't have to dance the political dance and make my colleagues happy. My own projects are for exploring my own ideas. Advice is always appreciated, but if someone wants to dictate how I write code in my own projects, they better damn well pay me (a lot). To the OP: Don't let people bully you. Many people have strong ideas and will want you to do things their way. You aren't going to make everyone happy, though. Somebody will be pissed off no matter what you do (if you are popular enough -- normally people won't pay any attention to you ;-) ). "Because that's what I want to do" is a completely valid reason for any decision on your own project.

macmac 11y ago

Unfortunate naming ie https://en.wikipedia.org/wiki/Improvised_explosive_device https://en.wikipedia.org/wiki/Improvised_explosive_device

blubbi2 11y ago
Damn. Didn't think of that. In fact I literally just renamed it: https://github.com/alexanderGugel/ied/commit/84628b3c871c85d7424d9d4a3813b19d02721bc6 https://github.com/alexanderGugel/ied/commit/84628b3c871c85d... Originally it was called mpm, but I figured that would have been pretty confusing, but it looks like the new name isn't necessarily better. I'm pretty terrible at naming. Any suggestions are more than welcome!
- diggan 11y ago
  I think it's fine as it is. Could have chosen a worse name for sure! And, managing dependencies is like dealing with explosives anyways...
  - JoshTriplett 11y ago
    > And, managing dependencies is like dealing with explosives anyways... Only funny for anyone who hasn't been affected by one, or had friends or family who were. Still better to avoid names with negative connotations (and search for them first to check). > Any suggestions are more than welcome! A few ideas: bpm - Better Package Manager edge - the thing that connects nodes jpm - Javascript Package Manager ppm - Peer Package Manager fpm - Functional Package Manager ayp - All Your Packages nnm - New Node Manager
    - inopinatus 11y ago
      yanm
      - xixixao 11y ago
        yapm

blubbi2 11y ago

Hey everyone! I made this. I'm happy to answer any questions, but please bear in mind that this is a WIP. There is still a lot of work to be done, although feature parity with npm is not the goal. Upcoming features are: * Nix-like rollbacks * built in registry server * discovery + installation via BitTorrent DHT Would love to get any feedback!

diggan 11y ago
For the discovery and installation, you might be interested in looking into using IPFS for it. Kind of gives you all the features you want out-of-the-box. Also, another guy is working on mirroring the npm registry in IPFS, might be interesting for you: https://github.com/diasdavid/registry-mirror https://github.com/diasdavid/registry-mirror
- blubbi2 11y ago
  This looks very interesting! Also related: https://github.com/dominictarr/npmd https://github.com/dominictarr/npmd
- daviddias 11y ago
  Hi Alexander, this is really great! I've started `registry-mirror` to demonstrate how a Content-Addressed file structure and P2P discovery, can bring a lot of speed improvements, specially when the bottleneck is low bandwidth/latency to the backbone, by connecting to more local peers that have the content that we are looking for. Right now, the goal with `registry-mirror` is to have a very large IPFS node in the network with the entire npm and that keeps replicating it, while end user machines only download the modules they need (and if they agree, provide them to the network as well). Each end user will be able to get the latest state of the registry, through a IPNS hash, a mutable pointer, that changes each time the mirror is updated. The nix package manager model layers perfectly on top of IPFS' MerkleDAG (https://github.com/ipfs/specs/tree/master/merkledag https://github.com/ipfs/specs/tree/master/merkledag), it can be a very awesome transport for ied. An example of a package manager that uses IPFS to distribute the packages is GX https://github.com/whyrusleeping/gx https://github.com/whyrusleeping/gx - Still a WIP. If this is interesting to you, join us at IRC Freenode #IPFS, it would be great to bounce more ideas! :)

drinchev 11y ago

That seems so cool. One question is how compatible is with npm? Can I use it as a drop-in replacement ( talking about the CLI tool )?

blubbi2 11y ago
Mostly, yes, but there are some features that are still missing: `ied publish` and `ied version` is coming next week. I'm also thinking of adding scoped modules, but I'm not sure about that yet. You can also configure a private npm registry to be used: https://github.com/alexanderGugel/ied/blob/master/lib/config.js#L9 https://github.com/alexanderGugel/ied/blob/master/lib/config...

gfosco 11y ago

I never thought to myself, "wow, npm is slow." Is that really the core problem this is solving?

broodbucket 11y ago
Have you used any other package manager? I'm not completely sure if it's just Node's dependency hell, but npm feels so incredibly slow compared to Cargo or bundler or something.
- mbell 11y ago
  Never profiled it but I would guess that is largely due to the npm's main problem: It doesn't locked dependency versions. As a result it probably builds the dependency graph a lot more than it should.
  - mikekchar 11y ago
    For me this is the main thing I'd like to see fixed. Not only does it build more than it need to, it can result in broken builds. When my dependency doesn't update, I don't expect its dependencies to update (especially when I don't have control over setting the versions of the dependencies of dependencies). This has bitten me more than once.
    - mbell 11y ago
      It really baffles me that most dependency management systems get this wrong. You'd think it would be consideration #1.
maaaats 11y ago
I work on a fairly large frontend project where everything is modularized. So we have maybe 50+ dependencies that are our own. These constantly change. I detest changing branches and having to run npm to get the correct version of stuff. Takes minutes before it even starts installing packages.
11y ago

nadocrew 11y ago

You say you are creating a more performant NPM. Is it impossible to fix the current NPM? Why not submit fixes back to NPM? Why create a new project?

glenjamin 11y ago
Works is ongoing, but npm is pretty complicated at this point https://github.com/npm/npm/issues/10380 https://github.com/npm/npm/issues/10380
blubbi2 11y ago
Its install process is very different from the one npm uses. E.g. just look into the node_modules directory produced by npm vs the one produced by ied. Also the way it wires up dependencies (using symlinks) has been called "non humane design" by npm... so I thought it would be easier to just start a new project... it's not a lot of code/logic needed there actually... :)
- nailer 11y ago
  npm v3 already uses the 'flat by default' design, so the node_modules folders aren't that different.
  - blubbi2 11y ago
    They are extremely different. I made a quick comparison here: https://gist.github.com/alexanderGugel/a10ed5655d366875a280 https://gist.github.com/alexanderGugel/a10ed5655d366875a280 Basically ied uses symlinks in order to resolve circular dependencies, while ied exploits the fact that require "falls back" in the directory structure.
    - nailer 11y ago
      Understood. Though I believe your comment (which compared ied with itself) should actually read: > Basically ied uses symlinks in order to resolve circular dependencies, while npm exploits the fact that require "falls back" in the directory structure.

jmandzik 11y ago

Cloned a popular project (babel) and on a MacBook Pro: npm install 52.28s user 8.08s system 73% cpu 1:22.41 total ied install 10.22s user 4.36s system 142% cpu 10.230 total Impressive.

blubbi2 11y ago
Awesome! Glad it worked that well! Please let me know if you run into any bugs: https://github.com/alexanderGugel/ied https://github.com/alexanderGugel/ied
- jmandzik 11y ago
  Happy to open an issue, but any plans on supporting git+ssh urls? For work projects, we have some modules installed via git and when I tried to install, I got the same error described here: https://github.com/alexanderGugel/ied/issues/2 https://github.com/alexanderGugel/ied/issues/2 Very promising project!
  - tracker1 11y ago
    +1 here, git+ssh and git+http(s) are essential imho... its' the easiest way to sidestep the need for an npm server for some internal projects/libraries.
    - blubbi2 11y ago
      Agreed. It's definitely on the roadmap. Should be done in a couple of weeks.

xrstf 11y ago

> produces a flat node_modules directory Finally. npm's node_modules makes Node on Windows unbearable from time to time. Shut up and take my money!

nailer 11y ago
npm v3 already does this by default.

js_throw_away 11y ago

Guys NPM is not the problem, Javascript is.