4 ms·

Yeah, the install function of npm/yarn/pnpm are all incredbly slow. And also seems to get slower super-linearly with the number of dependencies. I have one proj

Yeah, the install function of npm/yarn/pnpm are all incredbly slow. And also seems to get slower super-linearly with the number of dependencies. I have one project where it can take minutes (on my 2015 MacBook - admittedly it’s quicker on my brand new machine) just to add one be dependency and re-resolve the lock file. If that can solved by a reliable tool I’d definitely switch!

oreilles 4y ago
This is one of if not the most insane thing in web dev at the moment. Git can diff thousand of files between two commit in less time than it takes to render the result on screen. But somehow it can take actual minutes to find out where to place a dependency in a simple tree with npm. God, why ?
- capableweb 4y ago
  > it can take actual minutes to find out where to place a dependency in a simple tree with npm. God, why ? npm is famous for a lot of things & reasons, but none of those are "because it's well engineered". To this day, npm still runs the `preinstall` script after dependencies have actually been downloaded to your disk. It modifies a `yarn.lock` file if you have it on disk when running `npm install`. Lots of things like these, so that the install is slow, is hardly surprising.
  - hoten 4y ago
    Since when would npm install modify a yarn lock file?
    - rickstanley 4y ago
      I don't know exactly the "since when", but recently I was caught off guard when issuing `npm i` by mistake in a yarn project. It modifies "yarn.lock" by changing some, if not all, the registry from yarn pkg registry to npm package registry.
    - deleted 4y ago
      [deleted]
    - seniorsassycat 4y ago
      Npm 7+ can read and write yarn.lock https://github.com/npm/cli/blob/latest/workspaces/arborist/lib/shrinkwrap.js#L1148 https://github.com/npm/cli/blob/latest/workspaces/arborist/l...
- tempest_ 4y ago
  People building language tooling often use the language itself, even if it is not very suitable for the task at hand. This happens because the tooling often requires domain knowledge which they have and if they set out to write tooling for a language they tend to be experienced in that language.
  - kaba0 4y ago
    JS definitely doesn’t help, but it is a surprisingly fast language with modern runtimes. The problem lies elsewhere.
- nicoburns 4y ago
  In fairness, I suspect your average node_modules folder has a lot more files than your average git repo (maybe even an order of magnitude more)
- nine_k 4y ago
  But who still uses npm for that, and for what reason? Yarn seems much faster.
- pxc 4y ago
  > it can take actual minutes to find out where to place a dependency in a simple tree with npm. God, why ? Is it even a tree? Does NPM still allow circular dependencies?
solardev 4y ago
Is yarnv2 any better for this? I haven't tried it because it wasn't backward-compatible, but for fresh projects, would that be a better choice?
- metaldrone 4y ago
  Yes, yarn v2 (now v3) is a bit faster than yarn v1, but don't expect miracles. Yarn v2 is backwards compatible though. You just need to use the node_modules "linker" (not the default one) and it's ready to go.
  - conaclos 4y ago
    I have just discovered yarn2 / yarn3. The main advantage over npm / pnpm seems be the Zero Install philosophy [1] and the Plug'n'Play architecture [2]. Have you some feedback about these features? By the way, the yarn2 / yarn3 project is hosted on a distinct repository [3]. [1] https://yarnpkg.com/features/zero-installs https://yarnpkg.com/features/zero-installs [2] https://yarnpkg.com/features/pnp https://yarnpkg.com/features/pnp [3] https://github.com/yarnpkg/berry https://github.com/yarnpkg/berry
  - eyelidlessness 4y ago
    > Yarn v2 is backwards compatible though. You just need to use the node_modules "linker" (not the default one) and it's ready to go. Last I checked, not quite. Yarn 2+ patches some dependencies to support PnP, even if you don’t use PnP. I discovered this while trying out both Yarn 2 and a pre-release version of TypeScript which failed to install—because the patch for it wasn’t available for that pre-release. I would have thought using node_modules would bypass that patching logic, but no such luck.
niutech 4y ago
Maybe the problem lies in too many dependencies in the first place? Nowadays JS development is super bloated.

oreilles 4y ago

This is one of if not the most insane thing in web dev at the moment. Git can diff thousand of files between two commit in less time than it takes to render the result on screen. But somehow it can take actual minutes to find out where to place a dependency in a simple tree with npm. God, why ?

capableweb 4y ago
> it can take actual minutes to find out where to place a dependency in a simple tree with npm. God, why ? npm is famous for a lot of things & reasons, but none of those are "because it's well engineered". To this day, npm still runs the `preinstall` script after dependencies have actually been downloaded to your disk. It modifies a `yarn.lock` file if you have it on disk when running `npm install`. Lots of things like these, so that the install is slow, is hardly surprising.
- hoten 4y ago
  Since when would npm install modify a yarn lock file?
  - rickstanley 4y ago
    I don't know exactly the "since when", but recently I was caught off guard when issuing `npm i` by mistake in a yarn project. It modifies "yarn.lock" by changing some, if not all, the registry from yarn pkg registry to npm package registry.
  - deleted 4y ago
    [deleted]
  - seniorsassycat 4y ago
    Npm 7+ can read and write yarn.lock https://github.com/npm/cli/blob/latest/workspaces/arborist/lib/shrinkwrap.js#L1148 https://github.com/npm/cli/blob/latest/workspaces/arborist/l...
tempest_ 4y ago
People building language tooling often use the language itself, even if it is not very suitable for the task at hand. This happens because the tooling often requires domain knowledge which they have and if they set out to write tooling for a language they tend to be experienced in that language.
- kaba0 4y ago
  JS definitely doesn’t help, but it is a surprisingly fast language with modern runtimes. The problem lies elsewhere.
nicoburns 4y ago
In fairness, I suspect your average node_modules folder has a lot more files than your average git repo (maybe even an order of magnitude more)
nine_k 4y ago
But who still uses npm for that, and for what reason? Yarn seems much faster.
pxc 4y ago
> it can take actual minutes to find out where to place a dependency in a simple tree with npm. God, why ? Is it even a tree? Does NPM still allow circular dependencies?

solardev 4y ago

Is yarnv2 any better for this? I haven't tried it because it wasn't backward-compatible, but for fresh projects, would that be a better choice?

metaldrone 4y ago
Yes, yarn v2 (now v3) is a bit faster than yarn v1, but don't expect miracles. Yarn v2 is backwards compatible though. You just need to use the node_modules "linker" (not the default one) and it's ready to go.
- conaclos 4y ago
  I have just discovered yarn2 / yarn3. The main advantage over npm / pnpm seems be the Zero Install philosophy [1] and the Plug'n'Play architecture [2]. Have you some feedback about these features? By the way, the yarn2 / yarn3 project is hosted on a distinct repository [3]. [1] https://yarnpkg.com/features/zero-installs https://yarnpkg.com/features/zero-installs [2] https://yarnpkg.com/features/pnp https://yarnpkg.com/features/pnp [3] https://github.com/yarnpkg/berry https://github.com/yarnpkg/berry
- eyelidlessness 4y ago
  > Yarn v2 is backwards compatible though. You just need to use the node_modules "linker" (not the default one) and it's ready to go. Last I checked, not quite. Yarn 2+ patches some dependencies to support PnP, even if you don’t use PnP. I discovered this while trying out both Yarn 2 and a pre-release version of TypeScript which failed to install—because the patch for it wasn’t available for that pre-release. I would have thought using node_modules would bypass that patching logic, but no such luck.

niutech 4y ago

Maybe the problem lies in too many dependencies in the first place? Nowadays JS development is super bloated.