15 ms·

Hello Yarn 2, Goodbye node_modules

201 points by jonas21 5y ago

robinj6 5y ago
Kind of frustrating that this article doesn’t really describe in a nutshell how they do it, despite it being the title. It kind of assumes you already read about dropping node_modules.
- ComputerGuru 5y ago
  For anyone curious, that’s mainly described in the linked article here: https://yarnpkg.com/features/pnp https://yarnpkg.com/features/pnp
  - TazeTSchnitzel 5y ago
    It's interesting that node.js packaging has improved by borrowing an idea from the PHP world.
technicolorwhat 5y ago
Yes excited for this. I do believe having yarn introduced was the exact push NPM needed in order to tidy up their consistent installs game at that time. Hope these additions will make the ecosystem better.
codegladiator 5y ago
Why cant i scroll down on that page ?
- progval 5y ago
  There's a log-in wall: "Read the rest of this story with a free account.". But it's not shown unless you enable JS.
errantspark 5y ago
So the cost is you add another tool (yarn) to your toolchain and this is better than NPM because your deps are all in one folder and just mapped through a header file instead of being duplicated for every project, am I understanding that right?
- zdragnar 5y ago
  For a good while most people were better served by yarn anyway. I don't know exactly the current state of npm, but part of yarn's draw (even prior to the pnp aspect) was having deterministic builds. Even with a package lockfile, npm couldn't do that.
  - errantspark 5y ago
    Ahh, interesting, I wasn't aware of that issue. Personally the problem with npm for me has never been "the node_modules directory is too big" or "npm is non-deterministic" but rather "the quality of other people's code is terrible" or "this library solves 99 other cases in addition to the one I care about". I think if you're careful about what you pull in from npm it's totally fine.
    - zdragnar 5y ago
      Having deterministic builds is a godsent boon in an ecosystem without proper semantic versioning enforcement. The fact that minor version number changes can be breaking means that every build can break (a bit tautological but a point worth emphasizing). I also dont like that some packages are too big, or too small, or have too many transitive dependencies. It is a fact of life in npm land, though, making deterministic builds simultaneously minor and significant.
  - Zababa 5y ago
    > Even with a package lockfile, npm couldn't do that. NPM now has npm ci (which I think should be the default) for that.
    - 5y ago
encryptluks2 5y ago
Why is it that when I look at Yarn releases on GitHub it doesn't show v2?
- the_duke 5y ago
  That repo is for Yarn v1. V2 (now v3 ...) lives at https://github.com/yarnpkg/berry https://github.com/yarnpkg/berry. No idea why they did it that way. Probably because Yarn 1 is still very actively used, and things would get confusing otherwise.
jbverschoor 5y ago
How can you prevent from accidentally running npm or if a script runs it?
jbverschoor 5y ago
It’s insane that only now we have bundler
- Trasmatta 5y ago
  If that. Yarn 2 still has a bunch of issues, and they've already released Yarn 3. So now we have this fragmented scenario where tons of people are still on Yarn 1 because of compatibility issues, Yarn 2 can still be a pain to work with, and now there's a Yarn 3... Whereas Bundler just works (generally speaking).
derN3rd 5y ago
My company still has issues using yarn, as elastic beanstalk and their node environments only support npm... Does someone know an easy way without using own docker containers to fix that?
- ratww 5y ago
  Maybe this can help you: https://davidcalhoun.me/writing/using-yarn-on-elastic-beanstalk https://davidcalhoun.me/writing/using-yarn-on-elastic-beanst...
vvoyer 5y ago
Yarn 3.0 was released in July 2021 (https://dev.to/arcanis/yarn-3-0-performances-esbuild-better-patches-e07 https://dev.to/arcanis/yarn-3-0-performances-esbuild-better-...) Meanwhile, as much as I respect the work of Mael (yarn maintainer) I’ve switched back to npm. Mostly because: - isaacs (original creator of npm) did some commits lately - now backed by GitHub - they introduced workspaces (monorepo support) in npm 7 - it’s still easier to use the standard tooling in dev/ci/prod - like many, I had issues trying to convert my projects to yarn v2/3 The only thing yarn did I want in npm is an easy way to enforce the npm version across all environments (https://classic.yarnpkg.com/en/docs/cli/policies/ https://classic.yarnpkg.com/en/docs/cli/policies/).
- mauflows 5y ago
  Can you fix transitive dependencies in npm natively yet? That's why I switched to yarn.
- trinovantes 5y ago
  What kind of issues did you encounter converting to yarn 2/3?
  - theshrike79 5y ago
    Not the OP, but there was no clear tutorial on how to do it when I first tried it. I've got actual work to do, I don't have time to spend a day or two figuring out how to do yarn2 on our dev machines + CI env + prod. So we just stuck with yarn1. Now when Yarn 3 is out, it's either back to npm, which is supposedly on par with yarn1 in speed or I'll try a yarn 1 ->3 transition.
    - fuzzy2 5y ago
      Not that I’m a fan of Yarn 2+, but this view strikes me as narrow. Setting up and upgrading the development environment is actual work, too! If you don’t treat it as such, developer efficiency will suffer in the long run.
oraphalous 5y ago
I started working on converting our npm managed FE codebase to Yarn 2 just to get a sense of how much work it would take. The requirement of ensuring all deps are explicit before the build passes has exposed heaps of implicit dependencies in our code base - from devs back in the day that never npm install -S, yet it still worked somehow because some other package relying on the dep I presume. I haven't got the build working yet - and not sure it's even achievable - because so many of our explicit deps have implicit dependencies themselves (and some refuse to fix) - but it's been a valuable exercise.
- sildur 5y ago
  > because so many of our explicit deps have implicit dependencies themselves (and some refuse to fix) I usually create a fork in those cases.
- hasmolo 5y ago
  pnpMode: loose and log filters will help immensely
satya71 5y ago
Last time I tried, yarn 2 was totally unusable. Apparently, it would work in a future world where all packages in the registry were perfect. Has that future arrived?
- dexterhaslem 5y ago
  heck no. but hopefully before the sun extinguishes
- acemarke 5y ago
  My experience is that Yarn v2 works great as long as you stick with the `node_modules` linker. It's the "Plug 'n Play" functionality that is great in theory but still rough in practice. I've been using Yarn v2 on a decent-sized mixed-codebase JS app since the fall, and we just switched over to using it for both of the Redux Toolkit and React-Redux repos.
  - satya71 5y ago
    Thanks. I'll try with node-linker
moralestapia 5y ago
It is truly a shame that yarn is "the other package manager" in node as npm is an utter load of c*ap and the only reason it succeeded is because of nepotism favoring its "creator", Isaac Schlueter. Wish you the best with this and hope the tide turns on your favor eventually :)
- handrous 5y ago
  A few years ago, but long after people'd already started praising Yarn as a superior and completely-ready replacement for npm, it was still very easy to run into missing functionality if you went off the "happy path" of just installing stuff from NPM, and even then there was a decent chance of hitting bugs, often due to missing features or lack of extra safety checks that NPM was doing. Nepotism wasn't the reason the agency I was at kept starting projects with yarn (JS devs, whatcha gonna do) and pretty much always switching to NPM about the time each project was seriously getting going. It's because the likelihood of Yarn being the cause of some mysterious runtime or build bug you'd spend a couple hours chasing was at least 10x higher than NPM, or because we'd find we needed a feature Yarn didn't support. Yarn proponents have a history of pushing it despite its not being ready. Perhaps it's actually good now, though, I dunno. NPM's definitely not perfect, so there's plenty of room to improve on it.
wilg 5y ago
This is probably a good direction, but it seems like it's still contributing to JS dependency management becoming even hotter of a mess. There really shouldn't be two standard package managers for JS. This also reminds me how much we keep reinventing the wheel on this. I know it's likely infeasible for various reasons but it seems like it would be great to have a language-agnostic package manager that worked more or less like Bundler and can solve these problems once. (The most obvious downside of that is that – well – what language is it written in, and does it cause you to have to install multiple toolchains. Maybe the solution is some kind of standard spec for how package management should work, and the manifest formats, etc, and then each language ecosystem implements that standard. Or something. IDK.) Edit: A cool feature of such a system could be installing dependencies from multiple language ecosystems via one manifest file.
- enumjorge 5y ago
  > There really shouldn't be two standard package managers for JS. Amen to this. This type of stuff leads to so much confusion especially for beginners. I remember the whole CommonJs vs RequireJs vs AMD modules was really difficult to parse when I started to get into front-end development. Yes having multiple choices lets us experiment with alternatives, but I think we underestimate the costs of complicating the ecosystem as a result.
  - _fat_santa 5y ago
    I can't speak to other package managers but one silver lining about all of them (npm, yarn and pnpm) is they all are pretty interoperable. Between yarn and npm for example, I'm pretty sure they have virtually the same API so migrating from one to the other is a simple matter of using `yarn <command>` instead of `npm <command>`
    - idoubtit 5y ago
      No, yarn and npm don't have the same CLI API. For instance, the most basic command that adds a dependency : npm install [package] yarn add [package] Even the options for this command are different (--save-dev vs --dev). In fact, the situation is even more confusing, since `yarn install` exists but does not accept parameters: yarn install [1/4] Resolving packages... success Already up-to-date. yarn install something error `install` has been replaced with `add` to add new dependencies. Run "yarn add something" instead.
darepublic 5y ago
Gonna sit this one out, I'll see you at deno in 2023
simlevesque 5y ago
I see absolutely no reason to upgrade. I don't have any problems with npm7 right now and I fail to see why I'd need to change all these things and learn a new tool. Maybe the patch thing. But to say that you eliminated node_modules and then introduce a new folder which will have the same files inside it... sounds like madness to me. No thanks.
- acemarke 5y ago
  That's the point, it's _not_ the same files. Instead of extracting 75K individual files on disk, the Plug 'n Play concept keeps every package as its zipped tarball on disk. So, N tarballs, one per package, instead of hundreds of files per package. Conceptually, it's great. The problem comes when all the other tools in the ecosystem are still expecting a `node_modules` folder to exist and to be able to read those individual files directly, including Node itself. From what I've seen, the compat story has improved over the last couple years, but there's still rough edges. Last time I tried it I ran into problems with things like import aliases, and of course VS Code and TypeScript and other tooling all had to be tweaked to work correctly with Yarn's filesystem overrides. But, next time I start a new project from scratch, I'll probably give it another shot.
  - alerighi 5y ago
    The archive must be extracted to run the program. So they maybe consume less space on disk, but then you pay a penalty every time you run the program since (I assume) all these packages are extracted when you type yarn start (otherwise how can it work?) The problem of wasted disk space... start using a modern filesystem like ZFS (or BTRFS, but it's less stable) that does Copy on Write and you shouldn't have these problems. Or deal with it, nowadays disk is very cheap and fast so I don't see why worrying that much. I live with npm and that is fine, it's the most compatible solution.
    - plorkyeran 5y ago
      Decompressing files from a single tar.gz into memory can very easily be faster than opening a very large number of small files.
ezekg 5y ago
So... we're committing dependencies now, like Go before dep?
claytongulick 5y ago
I suppose I'm fairly unique in that I actually like node_modules. I enjoy the simplicity of having a project level folder that has all the dependencies. I like being able to easily expand it in my project tree and set a breakpoint in a dependency source file to debug strange issues. For deployments, I like being able to see exactly what's being executed. I would enjoy some more clear (to me) version locking though. I seem to constantly run into issues between environments with slight version changes, even though I commit my lockfile.
- ahuth 5y ago
  I too like node_modules. What's nice is you can use Yarn 3 and keep node_modules by setting the https://yarnpkg.com/configuration/yarnrc#nodeLinker https://yarnpkg.com/configuration/yarnrc#nodeLinker config option.
FractalHQ 5y ago
pnpm’s strategy of having a single global node_modules folder of cached deps it links to is still my favorite. Installs are way faster than both npm and Yarn, and it even works offline if you’ve ever installed the packages before. It’s supported workspaces long before npm, and works as a drop in replacement. I don’t like working without it these days. I just want Vercel to support it so I can use it to speed up build times with relying solely on build cache.
e1g 5y ago
I just moved our ~50 repos + 4 highly complex applications to Yarn 3. It works well, with a few gotchas - - In practice, using PnP and removing node_module creates more problems than it solves. Disk storage is cheap, so we use Yarn with "nodeLinker: node-modules" - Some "core" functionality like "yarn outdated" and "yarn install --production" is no longer available by default, and you need external plugins to mimic it - Custom pre/post hooks in package.json are not supported. So if you have "prestart" or "postrelease", that will no longer run automatically. But overall, it works well. We chose Yarn over NPM because we often use its "resolutions" feature that NPM has not yet implemented.
- hasmolo 5y ago
  1. pnpMode: loose has worked well for us 2. the install production requires the workspace focus thing for a litany of reasons but yeah 3. yarn actually supports plugins with hooks into the entire lifecycle if an install, it’s much more expressive than the default lifecycle stuff
drannex 5y ago
I may be the only one around who cares about this, but I _like_ node_modules. There are so many times where I am working on a project and either 1) I don't understand an API in a package so I quickly click into the module to learn more or 2) I find bugs, and can do a quickfix right there and then in the node_modules and test them out before submitting either a PR or work on a fork. I guess it would be possible(?) to go into the new .yarn folder but its just not the same. Sure, node_modules is a behemoth and there is still a likely better way - but I _love_ having all of my packages on a per-project basis. I used to be a user of Yarn 1 and 2, and their debugging and installation system is insanely better than NPM's, but I quite like having everything in one place for every project.
- vippy 5y ago
  They're not going away; they're just moving somewhere else.
- danellis 5y ago
  According to the article, it's still there, but they just renamed it to .yarn.
  - devinrhode2 5y ago
    You have .yarn/cache full of .tar.gz files. I do think yarn 2/3 has some way to specially cater to easily editing your npm packages.
    - rslabbert 5y ago
      This is vscode specific, but yarn suggests a zipfs extension which lets you explore the packages in exactly the same way you would with node_modules (with the added benefit of not having a massive file tree being expanded).
    - toastal 5y ago
      Too bad it's not at $XDG_CACHE_HOME/yarn. NPM recently shot that down.
somehnacct3757 5y ago
My favorite thing about yarn is how it lights a fire under npm's ass to implement community requests. But I don't actually want to _use_ yarn! More specifically I don't want to waffle my scripts, team, and keyboard habits btwn npm and yarn commands every time yarn has a new hot take on package management. Which is like, a lot.
- Rapzid 5y ago
  Yes but it's hard to know exactly how much credit to attribute.. GitHub acquired npm and Microsoft, who create their own package manager project as well, owns GitHub. Lot of pressure there as well.
FactoryReboot 5y ago
I honestly don’t want this. Each project having their own copy of the dependencies is a blessing.
pcthrowaway 5y ago
Unpaywalled: https://archive.is/IhHOz https://archive.is/IhHOz
amw-zero 5y ago
Didn’t Yarn 3 just come out?
- ratww 5y ago
  Yes, but version 3 doesn't differ from v2 as much as v2 differed from v2. It is a bit of a AngularJS/Angular situation.
deleted 5y ago
[deleted]
devinrhode2 5y ago
Yarn 2 problems: uses patched version of typescript. Using latest version of typescript is not reliable IME. Bugs with npm scripts are really hard to notice, the terminal output is much more pretty, but imo has regressed. Many npm scripts are not supported. You don't get a warning for which ones are ignored. Simply viewing the output of your npm scripts is quite annoying, if you can figure it out. Where yarn 1 "just worked" for my co-worker on windows (I'm on mac), yarn 2 did not "just work" - he got some security prompt (Yes, Corporate Sludge) Our security scanner (blackduck) could not properly parse the newly formatted yarn.lock file, but, thankfully, I could create a work-aroung by telling yarn2 to use `yarn2.lock` instead... and just maintain 2 lock files. (Yes, more corporate Sludge) Also, Svelte has a variety of issues with yarn2. In theory should be solved with yarn3. I am hopeful for yarn3. May try again once it's been iterated on a bit. Maybe once yarn4 beta is out, yarn3 will be well worth trying again. Biggest thing to me is the typescript situation. Yarn should be completely up-front about hacks like this. Yarn2 should immediately tell you to modify your package.json to point directly to the forked version of typescript when you first run yarn install, and in the migration guide. zero installs is very neat, but, upload speeds are always slower than download speeds. Proper CI caching+not uploading truckloads to git will be the fastest+best strategy (as long as upload speeds are slower than download)
- hasmolo 5y ago
  1. the patch just interjects zip file resolution, it shouldn’t be causing these problems 2. the npm lifecycle changes are very well laid out with many simple remedies see the docs 3. the list of deprecated lifecycle scripts are in the docs and very clear. the reasons are also interesting 4. yeah yarn requires you use process.stdout.write or write a yarn plugin 5. all of the issues you describe can be fixed via the sdk plugin sounds like the corp stuff is the worst, though we’ve had similar issues with our security scans
noway421 5y ago
For patching packages, I've been using `npx patch-package` workflow, and it's honestly very nice. It creates and deletes the temporary directory for you, so all you need to do is to change the files in node_modules and run `npx patch-package`!
nikeee 5y ago
Why doesn't nom adopt the symlinking that pnpm does? Always asked myself that, as it doesn't seem to have any downsides to me.
edf13 5y ago
And Medium now asks me to upgrade to read??
apatheticonion 5y ago
TypeScript support is hacky. I'd rather my `node_modules` folder be a symlink to a folder within a global package cache.
picardo 5y ago
The biggest pain point of Yarn 2 for me has been configuring CI servers. With `node_modules` there is one folder you need to copy between build server to CI server, but Yarn uses some virtual directory system to link the packages, and I have never been able to make it work. I ended up using `nodeLinker: node-modules`, which keeps node_modules folder, and that completely negates the benefit of using Yarn 2 in the first place.
wdb 5y ago
Sadly, currently, you can't use Yarn v1 or v2 or NPM v7 with GitLab private npm registry due to outstanding issues which is quite a shame. I love to use yarn workspaces but I have to wait for GitLab to fix the npm metadata issue first. But they have postponed it to October.
laangregor 5y ago
This is just awesome. It saves a lot of time.
Uehreka 5y ago
I’m gonna pull the trigger and do this on a project I’m currently working on by myself (lest anyone respond to my next sentence by calling me coward) but here’s my concern: I don’t check in node_modules for two reasons: 1- It can be fully regenerated (at the cost of time) from a lockfile. 2 - It’s huge, and large git repos are tough to work with (and can get you a nastygram from github if you’re on a free plan). Last time I tried yarn 2’s “zero installs mode”, the gist seemed to be “no more node_modules! Just check in this .yarn folder! Yes it’s technically the same thing, but we’ve always wanted to check in our deps and now we’re giving ourselves permission!” The .pnpthing file and .yarn (the parts that weren’t gitignored) added up to like, 40MB. That’s 100x larger than the rest of my project. Am I missing something here? Is this any better than just checking in node_modules? (Which would also technically get you a “zero installs mode”)
jamesappledev 5y ago
I moved our massive lerna monorepo to yarn v2 with PnP workspaces and cut 20 minutes off our CI time, deploy/build time (typescript) cut dramatically and reduced even our frontend bundle sizes thanks to the new and improved dedupe. It took all of maybe 10hrs but I've already made that back easily. Also checking out branches and syncing takes all of 3s versus the massive link process we used to have to do. Love yarnv2
deleted 5y ago
[deleted]
aabbcc1241 5y ago
How does it compares to pnpm, in turn of disk consumption and install speed?

201 points by jonas21 5y ago

Kind of frustrating that this article doesn’t really describe in a nutshell how they do it, despite it being the title. It kind of assumes you already read about dropping node_modules.

ComputerGuru 5y ago
For anyone curious, that’s mainly described in the linked article here: https://yarnpkg.com/features/pnp https://yarnpkg.com/features/pnp
- TazeTSchnitzel 5y ago
  It's interesting that node.js packaging has improved by borrowing an idea from the PHP world.

technicolorwhat 5y ago

Yes excited for this. I do believe having yarn introduced was the exact push NPM needed in order to tidy up their consistent installs game at that time. Hope these additions will make the ecosystem better.

codegladiator 5y ago

Why cant i scroll down on that page ?

progval 5y ago
There's a log-in wall: "Read the rest of this story with a free account.". But it's not shown unless you enable JS.

errantspark 5y ago

So the cost is you add another tool (yarn) to your toolchain and this is better than NPM because your deps are all in one folder and just mapped through a header file instead of being duplicated for every project, am I understanding that right?

zdragnar 5y ago
For a good while most people were better served by yarn anyway. I don't know exactly the current state of npm, but part of yarn's draw (even prior to the pnp aspect) was having deterministic builds. Even with a package lockfile, npm couldn't do that.
- errantspark 5y ago
  Ahh, interesting, I wasn't aware of that issue. Personally the problem with npm for me has never been "the node_modules directory is too big" or "npm is non-deterministic" but rather "the quality of other people's code is terrible" or "this library solves 99 other cases in addition to the one I care about". I think if you're careful about what you pull in from npm it's totally fine.
  - zdragnar 5y ago
    Having deterministic builds is a godsent boon in an ecosystem without proper semantic versioning enforcement. The fact that minor version number changes can be breaking means that every build can break (a bit tautological but a point worth emphasizing). I also dont like that some packages are too big, or too small, or have too many transitive dependencies. It is a fact of life in npm land, though, making deterministic builds simultaneously minor and significant.
- Zababa 5y ago
  > Even with a package lockfile, npm couldn't do that. NPM now has npm ci (which I think should be the default) for that.
  - 5y ago

encryptluks2 5y ago

Why is it that when I look at Yarn releases on GitHub it doesn't show v2?

the_duke 5y ago
That repo is for Yarn v1. V2 (now v3 ...) lives at https://github.com/yarnpkg/berry https://github.com/yarnpkg/berry. No idea why they did it that way. Probably because Yarn 1 is still very actively used, and things would get confusing otherwise.

jbverschoor 5y ago

How can you prevent from accidentally running npm or if a script runs it?

jbverschoor 5y ago

It’s insane that only now we have bundler

Trasmatta 5y ago
If that. Yarn 2 still has a bunch of issues, and they've already released Yarn 3. So now we have this fragmented scenario where tons of people are still on Yarn 1 because of compatibility issues, Yarn 2 can still be a pain to work with, and now there's a Yarn 3... Whereas Bundler just works (generally speaking).

derN3rd 5y ago

My company still has issues using yarn, as elastic beanstalk and their node environments only support npm... Does someone know an easy way without using own docker containers to fix that?

ratww 5y ago
Maybe this can help you: https://davidcalhoun.me/writing/using-yarn-on-elastic-beanstalk https://davidcalhoun.me/writing/using-yarn-on-elastic-beanst...

vvoyer 5y ago

Yarn 3.0 was released in July 2021 (https://dev.to/arcanis/yarn-3-0-performances-esbuild-better-patches-e07 https://dev.to/arcanis/yarn-3-0-performances-esbuild-better-...) Meanwhile, as much as I respect the work of Mael (yarn maintainer) I’ve switched back to npm. Mostly because: - isaacs (original creator of npm) did some commits lately - now backed by GitHub - they introduced workspaces (monorepo support) in npm 7 - it’s still easier to use the standard tooling in dev/ci/prod - like many, I had issues trying to convert my projects to yarn v2/3 The only thing yarn did I want in npm is an easy way to enforce the npm version across all environments (https://classic.yarnpkg.com/en/docs/cli/policies/ https://classic.yarnpkg.com/en/docs/cli/policies/).

mauflows 5y ago
Can you fix transitive dependencies in npm natively yet? That's why I switched to yarn.
trinovantes 5y ago
What kind of issues did you encounter converting to yarn 2/3?
- theshrike79 5y ago
  Not the OP, but there was no clear tutorial on how to do it when I first tried it. I've got actual work to do, I don't have time to spend a day or two figuring out how to do yarn2 on our dev machines + CI env + prod. So we just stuck with yarn1. Now when Yarn 3 is out, it's either back to npm, which is supposedly on par with yarn1 in speed or I'll try a yarn 1 ->3 transition.
  - fuzzy2 5y ago
    Not that I’m a fan of Yarn 2+, but this view strikes me as narrow. Setting up and upgrading the development environment is actual work, too! If you don’t treat it as such, developer efficiency will suffer in the long run.

oraphalous 5y ago

I started working on converting our npm managed FE codebase to Yarn 2 just to get a sense of how much work it would take. The requirement of ensuring all deps are explicit before the build passes has exposed heaps of implicit dependencies in our code base - from devs back in the day that never npm install -S, yet it still worked somehow because some other package relying on the dep I presume. I haven't got the build working yet - and not sure it's even achievable - because so many of our explicit deps have implicit dependencies themselves (and some refuse to fix) - but it's been a valuable exercise.

sildur 5y ago
> because so many of our explicit deps have implicit dependencies themselves (and some refuse to fix) I usually create a fork in those cases.
hasmolo 5y ago
pnpMode: loose and log filters will help immensely

satya71 5y ago

Last time I tried, yarn 2 was totally unusable. Apparently, it would work in a future world where all packages in the registry were perfect. Has that future arrived?

dexterhaslem 5y ago
heck no. but hopefully before the sun extinguishes
acemarke 5y ago
My experience is that Yarn v2 works great as long as you stick with the `node_modules` linker. It's the "Plug 'n Play" functionality that is great in theory but still rough in practice. I've been using Yarn v2 on a decent-sized mixed-codebase JS app since the fall, and we just switched over to using it for both of the Redux Toolkit and React-Redux repos.
- satya71 5y ago
  Thanks. I'll try with node-linker

moralestapia 5y ago

It is truly a shame that yarn is "the other package manager" in node as npm is an utter load of c*ap and the only reason it succeeded is because of nepotism favoring its "creator", Isaac Schlueter. Wish you the best with this and hope the tide turns on your favor eventually :)

handrous 5y ago
A few years ago, but long after people'd already started praising Yarn as a superior and completely-ready replacement for npm, it was still very easy to run into missing functionality if you went off the "happy path" of just installing stuff from NPM, and even then there was a decent chance of hitting bugs, often due to missing features or lack of extra safety checks that NPM was doing. Nepotism wasn't the reason the agency I was at kept starting projects with yarn (JS devs, whatcha gonna do) and pretty much always switching to NPM about the time each project was seriously getting going. It's because the likelihood of Yarn being the cause of some mysterious runtime or build bug you'd spend a couple hours chasing was at least 10x higher than NPM, or because we'd find we needed a feature Yarn didn't support. Yarn proponents have a history of pushing it despite its not being ready. Perhaps it's actually good now, though, I dunno. NPM's definitely not perfect, so there's plenty of room to improve on it.

wilg 5y ago

This is probably a good direction, but it seems like it's still contributing to JS dependency management becoming even hotter of a mess. There really shouldn't be two standard package managers for JS. This also reminds me how much we keep reinventing the wheel on this. I know it's likely infeasible for various reasons but it seems like it would be great to have a language-agnostic package manager that worked more or less like Bundler and can solve these problems once. (The most obvious downside of that is that – well – what language is it written in, and does it cause you to have to install multiple toolchains. Maybe the solution is some kind of standard spec for how package management should work, and the manifest formats, etc, and then each language ecosystem implements that standard. Or something. IDK.) Edit: A cool feature of such a system could be installing dependencies from multiple language ecosystems via one manifest file.

enumjorge 5y ago
> There really shouldn't be two standard package managers for JS. Amen to this. This type of stuff leads to so much confusion especially for beginners. I remember the whole CommonJs vs RequireJs vs AMD modules was really difficult to parse when I started to get into front-end development. Yes having multiple choices lets us experiment with alternatives, but I think we underestimate the costs of complicating the ecosystem as a result.
- _fat_santa 5y ago
  I can't speak to other package managers but one silver lining about all of them (npm, yarn and pnpm) is they all are pretty interoperable. Between yarn and npm for example, I'm pretty sure they have virtually the same API so migrating from one to the other is a simple matter of using `yarn <command>` instead of `npm <command>`
  - idoubtit 5y ago
    No, yarn and npm don't have the same CLI API. For instance, the most basic command that adds a dependency : npm install [package] yarn add [package] Even the options for this command are different (--save-dev vs --dev). In fact, the situation is even more confusing, since `yarn install` exists but does not accept parameters: yarn install [1/4] Resolving packages... success Already up-to-date. yarn install something error `install` has been replaced with `add` to add new dependencies. Run "yarn add something" instead.

darepublic 5y ago

Gonna sit this one out, I'll see you at deno in 2023

simlevesque 5y ago

I see absolutely no reason to upgrade. I don't have any problems with npm7 right now and I fail to see why I'd need to change all these things and learn a new tool. Maybe the patch thing. But to say that you eliminated node_modules and then introduce a new folder which will have the same files inside it... sounds like madness to me. No thanks.

acemarke 5y ago
That's the point, it's _not_ the same files. Instead of extracting 75K individual files on disk, the Plug 'n Play concept keeps every package as its zipped tarball on disk. So, N tarballs, one per package, instead of hundreds of files per package. Conceptually, it's great. The problem comes when all the other tools in the ecosystem are still expecting a `node_modules` folder to exist and to be able to read those individual files directly, including Node itself. From what I've seen, the compat story has improved over the last couple years, but there's still rough edges. Last time I tried it I ran into problems with things like import aliases, and of course VS Code and TypeScript and other tooling all had to be tweaked to work correctly with Yarn's filesystem overrides. But, next time I start a new project from scratch, I'll probably give it another shot.
- alerighi 5y ago
  The archive must be extracted to run the program. So they maybe consume less space on disk, but then you pay a penalty every time you run the program since (I assume) all these packages are extracted when you type yarn start (otherwise how can it work?) The problem of wasted disk space... start using a modern filesystem like ZFS (or BTRFS, but it's less stable) that does Copy on Write and you shouldn't have these problems. Or deal with it, nowadays disk is very cheap and fast so I don't see why worrying that much. I live with npm and that is fine, it's the most compatible solution.
  - plorkyeran 5y ago
    Decompressing files from a single tar.gz into memory can very easily be faster than opening a very large number of small files.

ezekg 5y ago

So... we're committing dependencies now, like Go before dep?

claytongulick 5y ago

I suppose I'm fairly unique in that I actually like node_modules. I enjoy the simplicity of having a project level folder that has all the dependencies. I like being able to easily expand it in my project tree and set a breakpoint in a dependency source file to debug strange issues. For deployments, I like being able to see exactly what's being executed. I would enjoy some more clear (to me) version locking though. I seem to constantly run into issues between environments with slight version changes, even though I commit my lockfile.

ahuth 5y ago
I too like node_modules. What's nice is you can use Yarn 3 and keep node_modules by setting the https://yarnpkg.com/configuration/yarnrc#nodeLinker https://yarnpkg.com/configuration/yarnrc#nodeLinker config option.

FractalHQ 5y ago

pnpm’s strategy of having a single global node_modules folder of cached deps it links to is still my favorite. Installs are way faster than both npm and Yarn, and it even works offline if you’ve ever installed the packages before. It’s supported workspaces long before npm, and works as a drop in replacement. I don’t like working without it these days. I just want Vercel to support it so I can use it to speed up build times with relying solely on build cache.

e1g 5y ago

I just moved our ~50 repos + 4 highly complex applications to Yarn 3. It works well, with a few gotchas - - In practice, using PnP and removing node_module creates more problems than it solves. Disk storage is cheap, so we use Yarn with "nodeLinker: node-modules" - Some "core" functionality like "yarn outdated" and "yarn install --production" is no longer available by default, and you need external plugins to mimic it - Custom pre/post hooks in package.json are not supported. So if you have "prestart" or "postrelease", that will no longer run automatically. But overall, it works well. We chose Yarn over NPM because we often use its "resolutions" feature that NPM has not yet implemented.

hasmolo 5y ago
1. pnpMode: loose has worked well for us 2. the install production requires the workspace focus thing for a litany of reasons but yeah 3. yarn actually supports plugins with hooks into the entire lifecycle if an install, it’s much more expressive than the default lifecycle stuff

drannex 5y ago

I may be the only one around who cares about this, but I _like_ node_modules. There are so many times where I am working on a project and either 1) I don't understand an API in a package so I quickly click into the module to learn more or 2) I find bugs, and can do a quickfix right there and then in the node_modules and test them out before submitting either a PR or work on a fork. I guess it would be possible(?) to go into the new .yarn folder but its just not the same. Sure, node_modules is a behemoth and there is still a likely better way - but I _love_ having all of my packages on a per-project basis. I used to be a user of Yarn 1 and 2, and their debugging and installation system is insanely better than NPM's, but I quite like having everything in one place for every project.

vippy 5y ago
They're not going away; they're just moving somewhere else.
danellis 5y ago
According to the article, it's still there, but they just renamed it to .yarn.
- devinrhode2 5y ago
  You have .yarn/cache full of .tar.gz files. I do think yarn 2/3 has some way to specially cater to easily editing your npm packages.
  - rslabbert 5y ago
    This is vscode specific, but yarn suggests a zipfs extension which lets you explore the packages in exactly the same way you would with node_modules (with the added benefit of not having a massive file tree being expanded).
  - toastal 5y ago
    Too bad it's not at $XDG_CACHE_HOME/yarn. NPM recently shot that down.

somehnacct3757 5y ago

My favorite thing about yarn is how it lights a fire under npm's ass to implement community requests. But I don't actually want to _use_ yarn! More specifically I don't want to waffle my scripts, team, and keyboard habits btwn npm and yarn commands every time yarn has a new hot take on package management. Which is like, a lot.

Rapzid 5y ago
Yes but it's hard to know exactly how much credit to attribute.. GitHub acquired npm and Microsoft, who create their own package manager project as well, owns GitHub. Lot of pressure there as well.

FactoryReboot 5y ago

I honestly don’t want this. Each project having their own copy of the dependencies is a blessing.

pcthrowaway 5y ago

Unpaywalled: https://archive.is/IhHOz https://archive.is/IhHOz

amw-zero 5y ago

Didn’t Yarn 3 just come out?

ratww 5y ago
Yes, but version 3 doesn't differ from v2 as much as v2 differed from v2. It is a bit of a AngularJS/Angular situation.

deleted 5y ago

[deleted]

devinrhode2 5y ago

Yarn 2 problems: uses patched version of typescript. Using latest version of typescript is not reliable IME. Bugs with npm scripts are really hard to notice, the terminal output is much more pretty, but imo has regressed. Many npm scripts are not supported. You don't get a warning for which ones are ignored. Simply viewing the output of your npm scripts is quite annoying, if you can figure it out. Where yarn 1 "just worked" for my co-worker on windows (I'm on mac), yarn 2 did not "just work" - he got some security prompt (Yes, Corporate Sludge) Our security scanner (blackduck) could not properly parse the newly formatted yarn.lock file, but, thankfully, I could create a work-aroung by telling yarn2 to use `yarn2.lock` instead... and just maintain 2 lock files. (Yes, more corporate Sludge) Also, Svelte has a variety of issues with yarn2. In theory should be solved with yarn3. I am hopeful for yarn3. May try again once it's been iterated on a bit. Maybe once yarn4 beta is out, yarn3 will be well worth trying again. Biggest thing to me is the typescript situation. Yarn should be completely up-front about hacks like this. Yarn2 should immediately tell you to modify your package.json to point directly to the forked version of typescript when you first run yarn install, and in the migration guide. zero installs is very neat, but, upload speeds are always slower than download speeds. Proper CI caching+not uploading truckloads to git will be the fastest+best strategy (as long as upload speeds are slower than download)

hasmolo 5y ago
1. the patch just interjects zip file resolution, it shouldn’t be causing these problems 2. the npm lifecycle changes are very well laid out with many simple remedies see the docs 3. the list of deprecated lifecycle scripts are in the docs and very clear. the reasons are also interesting 4. yeah yarn requires you use process.stdout.write or write a yarn plugin 5. all of the issues you describe can be fixed via the sdk plugin sounds like the corp stuff is the worst, though we’ve had similar issues with our security scans

noway421 5y ago

For patching packages, I've been using `npx patch-package` workflow, and it's honestly very nice. It creates and deletes the temporary directory for you, so all you need to do is to change the files in node_modules and run `npx patch-package`!

nikeee 5y ago

Why doesn't nom adopt the symlinking that pnpm does? Always asked myself that, as it doesn't seem to have any downsides to me.

edf13 5y ago

And Medium now asks me to upgrade to read??

apatheticonion 5y ago

TypeScript support is hacky. I'd rather my `node_modules` folder be a symlink to a folder within a global package cache.

picardo 5y ago

The biggest pain point of Yarn 2 for me has been configuring CI servers. With `node_modules` there is one folder you need to copy between build server to CI server, but Yarn uses some virtual directory system to link the packages, and I have never been able to make it work. I ended up using `nodeLinker: node-modules`, which keeps node_modules folder, and that completely negates the benefit of using Yarn 2 in the first place.

wdb 5y ago

Sadly, currently, you can't use Yarn v1 or v2 or NPM v7 with GitLab private npm registry due to outstanding issues which is quite a shame. I love to use yarn workspaces but I have to wait for GitLab to fix the npm metadata issue first. But they have postponed it to October.

laangregor 5y ago

This is just awesome. It saves a lot of time.

Uehreka 5y ago

I’m gonna pull the trigger and do this on a project I’m currently working on by myself (lest anyone respond to my next sentence by calling me coward) but here’s my concern: I don’t check in node_modules for two reasons: 1- It can be fully regenerated (at the cost of time) from a lockfile. 2 - It’s huge, and large git repos are tough to work with (and can get you a nastygram from github if you’re on a free plan). Last time I tried yarn 2’s “zero installs mode”, the gist seemed to be “no more node_modules! Just check in this .yarn folder! Yes it’s technically the same thing, but we’ve always wanted to check in our deps and now we’re giving ourselves permission!” The .pnpthing file and .yarn (the parts that weren’t gitignored) added up to like, 40MB. That’s 100x larger than the rest of my project. Am I missing something here? Is this any better than just checking in node_modules? (Which would also technically get you a “zero installs mode”)

jamesappledev 5y ago

I moved our massive lerna monorepo to yarn v2 with PnP workspaces and cut 20 minutes off our CI time, deploy/build time (typescript) cut dramatically and reduced even our frontend bundle sizes thanks to the new and improved dedupe. It took all of maybe 10hrs but I've already made that back easily. Also checking out branches and syncing takes all of 3s versus the massive link process we used to have to do. Love yarnv2

deleted 5y ago

[deleted]

aabbcc1241 5y ago

How does it compares to pnpm, in turn of disk consumption and install speed?