47 ms·

Yarn – A new package manager for JavaScript

• ef4 10y ago
  Kudos to the Yarn contributors for launching with a public RFC governance model right from the start: https://github.com/yarnpkg/rfcs https://github.com/yarnpkg/rfcs Practical, usable, deterministic lockfiles are wonderful and the lack of them has been the single biggest pain point in the npm ecosystem.
• steveklabnik 10y ago
  It looks like this addresses the biggest issues people have with npm's CLI, and it's coming from such huge names: Facebook, Google, and Tilde. Reproducible builds are a _huge_ issue, and this gives you that. Looks great! One interesting little tidbit I found from diving into the source: https://github.com/yarnpkg/yarn/blob/master/src/constants.js#L15 https://github.com/yarnpkg/yarn/blob/master/src/registries/yarn-registry.js It's not mentioned in the post, but looks like they're running their own registry as well... Oh! And open governance: https://github.com/yarnpkg/rfcs https://github.com/yarnpkg/rfcs !
  • jlongster 10y ago
    Huh, interesting! I thought this was going to just be a client. I wonder if it resolved to npm's servers or something.
    • steveklabnik 10y ago
      A curl -I shows CloudFlare, at least. npm's does not. So it's at _least_ doing some sort of cache...
  • thejameskyle 10y ago
    Oh hi Steve! https://registry.yarnpkg.com https://registry.yarnpkg.com is just a proxy to the normal npm registry provided by Cloudflare so we can add additional caching and work on network performance (lots of stuff we could layer on top). It should hopefully help those in China as well which would be awesome. But everything you do still lives on the normal npm registry.
    • steveklabnik 10y ago
      Makes total sense, thanks for elaborating! I don't do very much node dev, but when I do, it'll be yarn from now on.
  • ag_dubs 10y ago
    hi! they are running a mirror of the npm registry which contains all the package data but none of the permissions/user mgmt. this means that they are still dependent on npm infrastructure. npm, Inc strongly encourages people to build and use mirrors. for example, cloudflare (where the mirror lives, and where seb used to work) has been running https://npmjs.cf/ https://npmjs.cf/ for a long time.
    • wycats 10y ago
      At least speaking for myself, the npm package ecosystem is critical and I wouldn't want to do anything to undermine the integrity of the npm registry.
  • k__ 10y ago
    It's not the first one addressing these problems. Nix and ied (which borrowed from Nix) got these problems pretty much solved. I don't understand what spoke against these approaches? I mean okay, nix has its own language, which is probably a turnoff for JS devs, but ied?
    • ag_dubs 10y ago
      ied didn't work on windows because of the linking strategy. many people who need to use npm use windows.
      • blubbi2 10y ago
        That's not 100 % accurate. Arguably symlinks are kinda weird under Windows, but that doesn't mean symlinking node_modules doesn't work (you just need admin privileges). (Disclaimer: I'm the author of ied.)
    • deleted 10y ago
      [deleted]
• chrismorgan 10y ago
  See also: · http://yehudakatz.com/2016/10/11/im-excited-to-work-on-yarn-the-new-js-package-manager-2/ http://yehudakatz.com/2016/10/11/im-excited-to-work-on-yarn-... · https://yarnpkg.com/ https://yarnpkg.com/
• michaelsbradley 10y ago
  I think the write-up should specify `npm install -g yarnpkg` not `npm install -g yarn`.
  • thejameskyle 10y ago
    Lots of ways you can install Yarn over here: https://yarnpkg.com/en/docs/install https://yarnpkg.com/en/docs/install
    • pluma 10y ago
      Just a heads up: someone tried to comment out the Windows choco instructions with HTML comments but the angle brackets were escaped so the instructions are still there (with the escaped brackets).
      • dan15 10y ago
        Oops, that's my fault. It looked fine in the Github editor's preview when I commented it out, so I assumed it'd actually work properly. Chocolatey is coming soon, once they approve the package.
  • michaelsbradley 10y ago
    The yarn team has corrected the blog post[+]. [+] https://github.com/yarnpkg/yarn/issues/599#issuecomment-252948027 https://github.com/yarnpkg/yarn/issues/599#issuecomment-2529...
  • nailer 10y ago
    Could be worth finding the author of 'yarn' on npm and asking them for the name - it's been unmaintained for 4 years: https://github.com/samholmes/yarn https://github.com/samholmes/yarn
    • deleted 10y ago
      [deleted]
    • 10y ago
• colemannerd 10y ago
  I hope this becomes the default way to build npm projects... and then npm install becomes yarn :-) Exactly reproducible builds is a godsend to the js community.
• egeozcan 10y ago
  > Yarn, a collaboration with Exponent, Google, and Tilde. They should mention this at the very beginning. Multiple big players investing in this package manager means that we should maybe inspect a little bit more before chanting xkcd.com/927.
  • selectnull 10y ago
    927 was also my first thought. But what made me to reconsider was not more than one big name behind it (but it helped), but the fact that they rely on npm backend and did not reinvent everything from scratch. Basically, yarn is npm client done right reusing the same npm package repo.
  • Semiapies 10y ago
    That deprives me of the fun of downvoting people mindlessly misusing xkcd.com/927.
  • gkop 10y ago
    npm on it's own is that bad - this is nothing less than water in the desert.
• jbverschoor 10y ago
  "Linking: Finally, Yarn links everything together by copying all the files needed from the global cache into the local node_modules directory." Stop copying stuff. Just make it global and link. Next step is to make packages immutable and signed. I'm happy with this step and the fact that facebook will be able to push this.
  • wycats 10y ago
    This is something I really want to explore more through less-compatible modes. It was the original way yarn worked, but it wasn't compatible enough to be the default mode: https://github.com/yarnpkg/yarn/issues/57 https://github.com/yarnpkg/yarn/issues/57
    • jbverschoor 10y ago
      Hmm.. is it still available as a configuration option somewhere? I couldn't find it on the site. I'm a huge fan of bundler - it's dependency heaven. I'm also a big fan of the rubygems repo. It does not allow changes in released versions. Even without symlinks it's a much needed improvement in the javascript ecosystem
      • wycats 10y ago
        This is something I hope to work on over the next few months. It's a priority for Ember :)
    • endergen 10y ago
      If you do, would love see experiments on that. I do wish NPM would start adding badges or some other meta data to packages that signify that a module has constraints like: - No native code - No module state (No multiple singletons, and easily load variants of a module) - Pure installation, aka no build steps and is fully cacheable via hashing/immutable module patterns - Is all pure functional - 100% code coverage - Etc.
  • ef4 10y ago
    Yeah, that would be much better. But unfortunately the node_modules structure (which yarn is attempting to be fully compatible with) makes that impossible. The reason is that each package only finds its dependencies relative to its own location. So your second level dependencies cannot vary from project to project unless you do copying. (Example: AppA and AppB depend on LibX. LibX depends on LibY. Through their deterministic lockfiles, AppA and AppB disagree on which version of LibY to use. There is no way to symlink things together to satisfy that case without copying LibX or altering node's package resolution algorithm.)
• deleted 10y ago
  [deleted]
• wycats 10y ago
  I wrote a post explaining why I'm psyched to be working on it: TLDR: - open, community governance that will support long-term evolution - the technical details get a lot right out of the gate (decent performance, predictability, and security)
  • zamalek 10y ago
    Have you guys approached the ridiculous folder nesting situation? E.g. breaking out of the current/broken node_modules structure?
    • petetnt 10y ago
      There has been no problems with folder nesting in `npm` in general since the version@3 came out over a year ago.
      • steveklabnik 10y ago
        (other than its non-determinism)
        • thejameskyle 10y ago
          And Yarn it deterministic! :party_parrot:
      • spankalee 10y ago
        This isn't true though. npm chooses the first version of a package it encounters to install at the top level. Every other version is installed nested. If you have N uses of version A and one use of version B, but version B is installed first, then you get N copies of the package at version A.
      • ksherlock 10y ago
        It's improved but it's not fixed. if a depends on c 1.0 and b depend on c 2.0, both versions of c need to be installed and one of them will be nested.
• epmatsw 10y ago
  This looks awesome. But I have to wonder why create a whole new project rather than fork or upstream these changes to NPM? It doesn't seem like it's doing anything fundamentally different or outside of NPM's scope of responsibility.
  • jguimont 10y ago
    If you look at ruby there is "gem" that you could equal to "npm". On top of that there is "bundler" that is you could equal to "yarn". It is the same pattern.
    • epmatsw 10y ago
      Doesn't npm effectively replicate bundler's functionality (poorly) via shrinkwrap? Why not fix that feature rather than replacing it with a separate utility?
      • deleted 10y ago
        [deleted]
  • pluma 10y ago
    Because the npm client is a hot mess? The official npm cli is quite old and evolved along with all the different coding styles and architecture choices of the node and JS ecosystems. It's also full of dependencies on originally purpose-built modules that suffer from the same problems. This compounds various issues, resulting in long-standing bugs like `npm publish` sometimes not actually including all files in the tar bundle it generates[0]. NPM 3 was a partial rewrite but it's still built on the same mountain of code as NPM 2 and 1. npm Inc developers have in the past complained about not being able to address issues for a lack of resources despite having a ton of developers on the team[1]. The official npm client also doubles as an account management CLI to the npm registry. Which apparently npm Inc now considers a design flaw as they recently started creating a standalone tool for managing other aspects of their commercial services[2]. And finally it's important to understand that having a new package manager that is not tied into npm Inc means it's easier to replace the npm registry as the official node module registry in the future. Which would allow the community to get rid of the privileged position npm Inc currently holds in the node distribution (which is otherwise governed by the Node Foundation). Rewriting the client from scratch actually turned out to be the sanest approach in this situation, I think. [0]: https://github.com/npm/npm/issues/5082 https://github.com/npm/npm/issues/5082 [1]: https://github.com/npm/humans https://github.com/npm/humans [2]: https://npmjs.org/package/wombat https://npmjs.org/package/wombat
• tomdale 10y ago
  This is a huge leap forward for the JavaScript community—probably more than many people will realize right away. I loved Bundler's deterministic builds but chafed against the Ruby limitation of only having a single version of a dependency at once. npm solved this problem elegantly, but still struggles with non-determinism. I had resigned myself to thinking that maybe these were just fundamental tradeoffs in package management that could never be resolved. Then I had the opportunity to use Cargo, the package manager for Rust. It synthesized what, in my mind, are the best features of npm and Bundler into a single package manager, with terrific speed to boot. Yarn looks like it will be Cargo for JavaScript. After Cargo improved on many of the great ideas of npm, those ideas are returning to help improve the JavaScript ecosystem. Cross-pollination at its best. This also highlights a shrewd move on the part of npm: a clear decoupling between the npm registry and client, with a well-defined protocol between the two. The strength of Node is in the staggering size of its ecosystem; how those bits end up on disk is an implementation detail. This smart separation allows for this kind of experimentation on the client side, without causing the ecosystem fragmentation that happens when a new package manager requires a new registry. I'm also happy to see that a significant amount of work has already gone into the governance model. Despite the largest contributors being Facebook employees, it looks like they've really outdone themselves making sure this is a community-run project. A BSD license, no PATENTS file, and an Ember/Rust RFC process[1]; this has all the hallmarks of a community open source project. That's critical for open infrastructure projects like this, and they've nailed it. [1]: https://github.com/yarnpkg/rfcs https://github.com/yarnpkg/rfcs I'm very much looking forward to using Yarn in my own projects because it looks like it solves a lot of real problems I encounter every day. Thanks for all the hard work!
  • k__ 10y ago
    I find it strange that the time isn't invested in already existing projects. But at least it's a move away from NPM. I think the most problems I had with JavaScript develompent in the last 2 years came from NPM.
    • tranvu 10y ago
      Yarn isn't a replacement for npm itself. It's a client that can read/write to npm, and other registries such as Bower.
• simplify 10y ago
  Am I correct in understanding Yarn works similar to Ruby's bundler?
  • thejameskyle 10y ago
    Similar, but even more similar to Rust's Cargo https://crates.io/ https://crates.io/
• LoSboccacc 10y ago
  so, where are the packages stored? how is this more secure than npm? how does this solve the leftpad problem?
  • nailer 10y ago
    leftpad was a social issue. It's been solved by policy on the registry side stopping packages depended on by many others being unpublished.
    • ag_dubs 10y ago
      actualyl at this point, unless there is a legal request or a serious security issue, we don't allow any unpublishes. we encourage transferring the package to the npm user and deprecating it. yay immutable registry!
      • steveklabnik 10y ago
        It is unclear why this is downvoted: this is an npm employee who made the requisite policy changes after left pad happened. http://blog.npmjs.org/post/141905368000/changes-to-npms-unpublish-policy http://blog.npmjs.org/post/141905368000/changes-to-npms-unpu...
    • pluma 10y ago
      If by "social" you mean bad decisions on settling a naming dispute, yes it was a social issue. That issue hasn't really been addressed publicly (the official policy still doesn't specify how disputes are decided by npm when no amicable agreement can be reached -- other than what essentially boils down to "we'll do what we feel is right"). The actual disruption however was solved by npm Inc disabling unpublishing (except for some very specific circumstances) and by creating a dummy account unpublished modules are reassigned to to avoid abuse (whereas previously anyone could simply claim unpublished modules and publish new versions).
      • nailer 10y ago
        > If by "social" you mean bad decisions on settling a naming dispute, yes it was a social issue. No, I mean someone retroactively removing a package others were using out of spite.
• jshawl 10y ago
  >We decided to zip the entire node_modules folder and upload it to an internal CDN so that both engineers and our continuous integration systems could download and extract the files consistently. why not mirror npm modules there then?!
• szimek 10y ago
  Recently I got annoyed how hard it is to use shrinkwrap in npm and started working on a npm wrapper that would make npm as easy to work with as Ruby Bundler by copying its workflow as closely as possible (https://github.com/szimek/bundlerjs https://github.com/szimek/bundlerjs). Thankfully, I don't have to develop it anymore ;) Big thanks to all Yarn developers!
• swang 10y ago
  wow already one of the features i'm loving in yarn is that it tells you which package is firing warnings about package incompatability. warning electron-prebuilt-compile > electron-compilers > jade@1.11.0: Jade has been renamed to pug, please install the latest version of pug instead of jade in npm, that would have just said the part after "jade@1.11.0" which was really vague and didn't really make you want to "fix" it because which npm module do you have to go into? who knows because npm (the package manager) didn't tell you.
  • ateevchopra 10y ago
    Just experienced the same. Loved it! npm was giving vague errors about dependency's dependency. Now it's so much clearer that which package needs to be updated. Also the speed of installation has reduced a lot !
  • algesten 10y ago
    i seriously want npm to stop warning me of outdated deps 3 levels down my hierarchy. so what if some shitty old unmaintained lib we started relying on a few years back use lodash v3.x? i'm not going to worry about it as long as it works.
• kzahel 10y ago
  "The React Native package.json currently lists just 68 dependencies, but after running npm install the node_modules directory contains 121,358 files." That, to me, is what is wrong with npm. The problem stems from node.js not coming with "batteries included" so there is a proliferation of tiny libraries that do the most trivial things.
  • sotojuan 10y ago
    90% of those files are probably Babel-related.
  • justinsaccount 10y ago
    As I understand it, the related problem is the poor support in build tools for dead code elimination which encourages people to publish libraries that contain a single function like left-pad.
  • christophilus 10y ago
    Yeah. Inspecting the node_modules folder for pretty much any front-end project is depressing. Still, I prefer npm to no package manager at all. One problem is that packages often bundle everything, rather than including an npmignore. I'm guilty of this myself. Do you think that the problem of over-dependence is reversible, and if so, how?
  • mschuster91 10y ago
    > That, to me, is what is wrong with npm. The problem stems from node.js not coming with "batteries included" so there is a proliferation of tiny libraries that do the most trivial things. This is in no way a fault of node.js but of the whole JS standard. People these days use npm modules to run in a browser and ship stuff bundled together with webpack (or other bundlers), so even if nodejs had a proper stdlib, you'd still need to depend on a polyfill so that your stuff works in a browser environment, too. And due to the fact that even if a sane stdlib would ever be standardized, it would take YEARS of time until it reaches significant market share (looking at you, Android, Safari and IE), so you'd always have to ship a polyfill.
    • 10y ago
• pilif 10y ago
  Yarn is currently being powered by a service running at https://registry.yarnpkg.com https://registry.yarnpkg.com - at least that's what's referenced in the yarn.lock file that's created. I see no documentation nor code for the service powering this, nor is there a way to tell the command line tool to use a registry at a different address. Aren't we just replacing the dependency on npm, Inc.'s registry with a new dependency on Facebook Inc's registry with this? Yes. The client has a few advantages, but the registry is still closed and while with npm there's an option of running a private registry if you pay them, here, there's no option at all.
  • deleted 10y ago
    [deleted]
  • ag_dubs 10y ago
    that registry is a mirror of npm's registry. the mirror contains all package data but does not include any of the permissions or user management, so this tool still depends on npm infra for both that, and as a replication source.
    • thejameskyle 10y ago
      registry.yarnpkg.com is a proxy, it doesn't contain any package data except for what gets cached by Cloudflare. Just a Cloudflare'd "CNAME registry.yarnpkg.com. registry.npmjs.org."
• andrewvijay 10y ago
  The 'attempts at scaling npm client' section is incredibly cringy even to read. Those are real complex problems that every one who works on a massive scale will face at some point in time.
  • cpojer 10y ago
    Can you make yarn out of hair?
• currywurst 10y ago
  I wish some kind of "auto-bundle" feature prevented the creation of multiple hundred MB's worth of tests, readmes and docs on disk when something like single "babel.bundle.js" file would do. Is it really important that one knows that a tool dependency uses left-pad ? What could be the drawbacks of such bundling?
  • epmatsw 10y ago
    In theory, those submodules should be npmignoring those files and subdirectories, right?
  • thejameskyle 10y ago
    You can run `yarn clean` and it will clear out all the extra files you don't need: https://yarnpkg.com/en/docs/cli/clean https://yarnpkg.com/en/docs/cli/clean
    • currywurst 10y ago
      Great !! That sounds like it should help atleast .. hope the heuristics improve to capture most npm patterns
• wale 10y ago
  While Yarn sounds like a nice wrapper with extra stuff. I want to ask if this collaboration couldn't happen with the NPM team?
  • thejameskyle 10y ago
    npm's been involved since the early days, they wrote up a blog post here: http://blog.npmjs.org/post/151660845210/hello-yarn http://blog.npmjs.org/post/151660845210/hello-yarn
    • wale 10y ago
      Oh Okay. Good one. Thanks for the blog ref #openess
• geofree 10y ago
  Nice name too: https://www.getyarn.io/yarn-clip/6f28eef4-98fe-4ae5-96c0-da3aa65ce319 https://www.getyarn.io/yarn-clip/6f28eef4-98fe-4ae5-96c0-da3...
• petetnt 10y ago
  Rather impressed with this with the few tests I've run. Kudos to the creators and can't wait to see this project progress (and to contribute into!) in the near future.
• spankalee 10y ago
  Yarn is particularly great for front-end web apps because of its flat installation mode. ES6 module imports and HTML Imports both require that dependencies are imported by URL. This means that the only reliable way to import another module is by relative URL, like: import * as $ from '../jquery/jquery.js'; This requires that packages are installed flat, as siblings. Yarn is going to enable native JS modules and projects like Polymer to use npm instead of Bower. :)
  • MatthewPhillips 10y ago
    This is what I was most curious about, but it looks like --flat is a cli option but not the default. Definitely moving in the right direction though!
    • spankalee 10y ago
      --flat is the cli option, but there's also "flat": true support in package.json. Used at the top level this forces a flat install, but used in a dependency it means that that package requires a flat install and it throws an error if it's not. This means that HTML imports and ES6 modules can force flat installs and start moving the front-end package ecosystem in that direction.
  • dismantlethesun 10y ago
    You can use webpack or JSPM to resolve directories at the top level. That way you can just import "web/X" even if the web/ directory isn't a sibling to your current file. It's not very helpful if you are distributing an NPM package, but for people who aren't authoring libraries, its a godsend.
    • spankalee 10y ago
      Do you mean import '/web/X'? Import URLs have to start with a `/`, `./`, or `//`. Even so, that requires a build tool. I want to be able to load working sources directly out of my packages directory.
• wildpeaks 10y ago
  Note that it doesn't work with private modules yet, but it's coming soon: https://github.com/yarnpkg/yarn/issues/521 https://github.com/yarnpkg/yarn/issues/521
• danielrbradley 10y ago
  Overall looks good, but it's a shame that flat mode is opt-in instead of the default - this is the sane way of doing package management. Would have been nice to flip the options over so you have to explicitly allow multiple versions of the same package - which you could automatically enable if you're converting from npm.
  • steveklabnik 10y ago
    Elsewhere in this thread, they said that its because it doesn't work well with enough of the ecosystem yet.
• andreygrehov 10y ago
  I just tried to install yarn and ran it to against a relatively large React application. What's interesting is that it found an incompatible module and could not proceed, whilst npm works fine. The article states that running `yarn` is the same as `npm install`, which doesn't look like the case. At that point, I'm curious what are the differences.
  • thejameskyle 10y ago
    Sorry about that, there are very likely still some bugs out there we weren't able to catch in all our testing which will be ironed out by the community in the next few days. Can you open an issue with details about which package failed to install? https://github.com/yarnpkg/yarn/issues/new https://github.com/yarnpkg/yarn/issues/new
    • andreygrehov 10y ago
      Thank you. Done. The issue is reproducible with an entirely new project.
      • steveklabnik 10y ago
        A fun side effect of reproducible builds: recreating bugs for bug reports is easier.
  • quarterto 10y ago
    npm treats the package.json engines field as advisory, but yarn is strict about it. maybe that's you're problem
• jrs235 10y ago
  Here is npm's response: http://blog.npmjs.org/post/151660845210/hello-yarn http://blog.npmjs.org/post/151660845210/hello-yarn
  • Touche 10y ago
    Looks like they are trying to write off yarn as just another alternative install tool. I think this is short cited and hope they are taking this threat more seriously internally.
  • BinaryIdiot 10y ago
    I'm glad they're okay with it but then again what else would they post? npm has some major, major issues with it and it has barely moved in years. This Yarn looks like it solves quite a few issues with npm and it took another company to build it. That's insane. It wouldn't surprise me if yarn becomes popular and its proxy to npm slowly turns off and boom, everyone would be migrated and npm would be left with little to no users. Yeah that's probably unlikely for the immediate future but Yarn is positioned perfectly to siphon off npm's users very painlessly.
    • sotojuan 10y ago
      Does anyone know why npm moves so slowly?
      • softawre 10y ago
        they have millions of installs and can't break them?
        • striking 10y ago
          Well, but remember that time they shipped an SSL cert that broke all updated clients? I'm not sure they move slowly because they don't want to break things, because that would have been a pretty easy bug to catch... http://blog.npmjs.org/post/78165272245/more-help-with-selfsignedcertinchain-and-npm http://blog.npmjs.org/post/78165272245/more-help-with-selfsi...
• nawitus 10y ago
  Should you commit the yarn lockfile to version control?
  • steveklabnik 10y ago
    I don't know about yarn specifically, but for all the systems it's modeled after, the answer is "yes for binaries, no for libraries".
  • FLGMwt 10y ago
    Docs here [1] say yes. It also seems to imply that this is edited by the `yarn` command so it would be invalidated if you manually edited your `package.json`? Dunno if it's destructively invalid though. 1: https://yarnpkg.com/en/docs/configuration#toc-use-yarn-lock-to-pin-your-dependencies https://yarnpkg.com/en/docs/configuration#toc-use-yarn-lock-...
  • quaunaut 10y ago
    Yes. This makes it part of code reviews and is what keeps everyone who might ever use the project(whether human or CI) using the same libraries.
• sdegutis 10y ago
  For what it's worth, the name is really cool. I just looked up the etymology of Node and it comes from Latin `nodus` meaning knot. So if a Node program is the knot, then the stuff that makes it up and that it comes from must be the yarn! Language is fun.
• koolba 10y ago
  > Finally, updating a single dependency with npm also updates many unrelated ones based on semantic versioning rules. This makes every change much larger than anticipated, and having to do things like committing node_modules or uploading it to a CDN made the process less than ideal for engineers. You shouldn't check in node_modules. You should use a shrinkwrap file to indicating the specific sub-dependency versions you're using.
  • epmatsw 10y ago
    They address pretty throroughly why that does not work for them in the linked article.
    • koolba 10y ago
      > They address pretty throroughly why that does not work for them in the linked article. Not really. They just mention it in passing: From the article >> Shrinkwrap files aren't generated by default and will fall out of sync if engineers forget to generate them, so we wrote a tool to verify that the contents of the shrinkwrap file matches what's in node_modules. These files are huge JSON blobs with unsorted keys, though, so changes to them would generate massive, difficult-to-review commits. To mitigate this, we needed to add an additional script to sort all the entries. Comparing node_modules to shrinkwrap isn't necessary. Building the app will re-generate node_modules and tests should catch whether or not the app works as intended. The sorting issue for shrinkwrap is legit, but the answer to that is sort it! Sure you'll get some deep diffs if the dependencies change, but that reflects the real changes to the app. The simpler answer seems to be to add a build test to mandate the existence of a shrinkwrap file, verify it's sorted, and (optionally) verify it's up to date. The latter doesn't even have to be done every build.
• bouk 10y ago
  I'm so excited about this. Having wasted lots of time migrating Shopify to use npm for JS package management, it seems that yarn fixes everything that had been a nuisance with it. Woo!
• tolmasky 10y ago
  My initial tests with this are very positive (I tried this use case here: https://github.com/npm/npm/issues/10999 https://github.com/npm/npm/issues/10999 ). It handled it quite well and we'll definitely be considering this for our package. I'm curious about the "flattening" approach though -- and why something like ied ( https://github.com/alexanderGugel/ied https://github.com/alexanderGugel/ied ) wasn't done instead. ied only ever installs one copy of a module, but allows each module to think they have their own copy of it through clever symlinkery (essentially making it FEEL like npm 2 while getting the benefits of npm3/yarn). I'm not a fan of (explicit) flattening for a number of reasons: 1. Mimicking recursive dependencies while still only having on physical copy on disk has the benefit (or downside I suppose) of isolating and protecting different module's dependencies from each other. AKA, if you require("x").mutate = true, this won't be seen by other modules. Again, depends whether you like this feature or not, but with node's treat-symlinks-as-different-files configurable switch, you can actually control it yourself in ied I believe. 2. You don't run into tricky situations where require("x") can accidentally pick something up you didn't mean to (since it is now in the parent path of ALL your dependencies). So, if A depends on x, and you install B, it will be able to require("x") (maliciously?) and modify it, affecting A's behavior. Perhaps this is considered a strange possibility, but it gets particularly weird with peer dependencies, where you expect the user to choose their own top level dependency, but this can now be accidentally "chosen for you" by a subdependency of another package. 3. I just don't like clicking on my node modules folder and seeing 100 top level directories when conceptually I typed "pkg-manager install one-thing". This can lead to you yourself making the above mistake, by require("x") and having it "just" work, then having it "just break" when you remove the dependency that included x. I'm willing to be convinced though (on RunKit we had module-fs simulate npm2 install behavior, super simple recursive installation, conceptually similar to what ied does).
  • sebastianmck 10y ago
    Hey! I'm Sebastian McKenzie (@kittens on GitHub) and I'm the lead develop on Yarn at Facebook. We initially used a method similar to ied when we first started experimenting with internal usage at Facebook. We ran into a lot of issues with OS compatibility around symlinks and existing tools not supporting it. Windows lacks support for symlinks on non-Admin accounts, NTFS junctions work but they're slightly quirky in their behaviour and don't behave exactly the same way. A lot of tooling relies on the existing node_modules structure. ESLint for example will load tools relative to itself which means that they need to be reachable in the tree, you can solve this with hardlinks but there's caveats with that too. Existing tools aren't very well suited for handling cycles either which in a system like npm are extremely common as a transitive module could depend on one that's higher in the graph. We care a lot about existing ecosystem compatibility and because of these limitations we went with the current status quo. We'll continue to explore others options though and if there's a chance we can do it without any of these caveats then we're more than happy to explore it. Let me know if you have any further questions!
• fold_left 10y ago
  Brilliant. I've been trying to tackle this problem for the last year or so with shrinkpack (https://github.com/JamieMason/shrinkpack https://github.com/JamieMason/shrinkpack) with _some_ degree of success and a nagging feeling that much more could be done. It's great to see attention is being spent on what is a really important area, one which has been largely under-appreciated in Node.js until now (aside from for a short period after the left-pad incident). Kudos to Facebook, I look forward to giving it a try.
• manojlds 10y ago
  Clashes with Apache YARN - https://hadoop.apache.org/docs/r2.7.2/hadoop-yarn/hadoop-yarn-site/YARN.html https://hadoop.apache.org/docs/r2.7.2/hadoop-yarn/hadoop-yar...
  • vlunkr 10y ago
    They are different problem domains, I think confusing them is unlikely
    • loukrazy 10y ago
      Tell that to my google searches
      • __david__ 10y ago
        I'm sure that will fix itself very soon.
        • andrewprock 10y ago
          I'm sure it will. Search "go" on google, and the first link is the programming language. Search on any other search engine, and as one might expect, the definition or the game of go is ahead of the language.
          • smrtinsert 10y ago
            Wait till I release my new programming language, 'the'. It's the go killer.
            • imtringued 10y ago
              the haskell eliminator
      • rizky05 10y ago
        Use "apache yarn" to search spark related problem, otherwise "yarn js" *untested theroy
• thunfisch 10y ago
  This is a very good step in the right direction. It still doesn't solve the biggest problem the node.js and javascript ecosystem has IMHO: dependency hell, thousands of trivial packages, just utter chaos.
• thecity2 10y ago
  So this is not a Hadoop thing?
  • andmarios 10y ago
    Just an easy way for javascript devs to add big data keywords into their CV. :p
• karliky 10y ago
  https://twitter.com/k4rliky/status/785875228710858753 https://twitter.com/k4rliky/status/785875228710858753 please no
• mrmrben 10y ago
  What does this mean for NPM Inc the company? Is a replacement / mirror / rethink of the NPM infrastructure coming next? If I was NPM Inc I would think about adopting yarn if possible.
• pfooti 10y ago
  There's a lot to like here - deterministic builds are great. However, yarn doesn't currently seem to support installing straight from github or private packages [0], [1]. I'm sure this will be added in the future, but it is currently a dealbreaker for me - I'm using a single package that's just straight hosted in a private git repo, as the package itself isn't ready to be published yet. I'm sure other people are using private packages for more normal reasons (closed source, pre-production, etc). If yarn actually made it simpler to refer to a local package during development, I'd be on board with that. (I am developing the dependency, but want to also work on the thing that depends on it, so I'd rather just save locally and refresh npm on the outer project, but that's hard to get right - file urls don't always update, the easiest workflow seems to be delete the package from node_modules, and npm install, with the package having a git url rather than a version in package.json). 0: https://github.com/yarnpkg/yarn/issues/573 https://github.com/yarnpkg/yarn/issues/573 1: https://github.com/yarnpkg/yarn/issues/521 https://github.com/yarnpkg/yarn/issues/521
  • cpojer 10y ago
    You are right, we aren't 100% compatible right now. Please track the issues on the Yarn GitHub repo that you linked or better yet: help us fix it. This is a community project and we are excited for the community to jump in and help us out and complete the few missing pieces. I agree local development with many packages can be hard. For Jest we adopted `lerna` See https://github.com/facebook/jest https://github.com/facebook/jest and https://github.com/lerna/lerna https://github.com/lerna/lerna which makes cross-package development in a mono-repo a lot of fun. Maybe that solution will work for you?
  • evolve2k 10y ago
    This. I came here to say this also. In ruby if you want to patch a repo for yourself you just fork it and make changes and then in bundler point to the git url (often on github). Then run bundle install and you're set. With npm I've been frustrated a few times whereby you can fork the repo and point package.json to the git repo but after running npm I usually get a bunch of compilation errors. It would be wonderful to make this as simple as with bundler within the ruby ecosystem, point to forked url and running yarn handles the rest.
• foxwoods 10y ago
  yarn.lock[0] is similar to composer.lock[1] [0] https://github.com/yarnpkg/yarn/blob/99dd4504469273332f01ce5ad846c995378cc048/yarn.lock https://github.com/yarnpkg/yarn/blob/99dd4504469273332f01ce5... [1] https://github.com/composer/composer/blob/20ee689bb464edfe7e57af6be8f03fb664f5db1d/composer.lock https://github.com/composer/composer/blob/20ee689bb464edfe7e...
  • dham 10y ago
    Pretty close to https://github.com/discourse/discourse/blob/master/Gemfile.lock https://github.com/discourse/discourse/blob/master/Gemfile.l...
• mvdanj 10y ago
  Its disheartening to see so many duplicate efforts to solve the same problems everyone faces. Instead of supporting an existing open-source project that attempts to solve the problem in pretty much the same way (jspm), a conglomerate once again builds their own from scratch. I do not think the underlying motivations for doing so are questioned enough. Sure there is control, but it is of course anyway-you-slice-it an aggrandizement of their brand. We should be ashamed as a community to support the idea of yet ANOTHER client side dependency management system. How oh how did we ever solve this problem before Facebook came along and made Yarn in late 2016?
  • tomdale 10y ago
    I don't think it's fair to compare jspm and Yarn. jspm is an interesting project, but is tightly coupled with SystemJS. Telling everyone to rewrite their apps to use SystemJS and jspm is a huge lift. Yarn, on the other hand, has focused on npm compatibility from the get-go. The entire point is to be a drop in replacement, with no changes (or maybe very very minimal changes) required to gain a lot of benefit. In this slice of the industry where everyone is told they have to rewrite to the latest thing every year or two, I find it very refreshing to see Yarn's focus on backwards compatibility, improving existing infrastructure, and having a clear, painless adoption path.
    • mvdanj 10y ago
      I don't agree with any of your statements, and I think they are inaccurate. jspm aims to be type-agnostic with respect to modules (supports all major types, hence why it is called a universal module loader). jspm certainly doesn't advocate rewriting your modules (you can use exports, AMD, ES6, whatever). In fact, when the browsers natively support module loading, systemJS will go away (which is really just a polyfill for this functionality). jspm also has npm compatibility.
• bandrami 10y ago
  This may come off as a troll, but it's an honest question. I'm not a javascript guy. It's not a language I deal with at all. Why on God's green earth does it need as much tooling as it seems to have? Are people really making projects with dozens (hundreds? more?) of dependent libraries? Are there aspects of the language or runtime that reward multiple layers of configuration management? In short, what the hell is up with this ecosystem?
  • pjmlp 10y ago
    JavaScript is the new BASIC and every millennial wants to populate his/her Github portfolio for Silicon Valley interviews.
  • ed312 10y ago
    A very limited standard library with no native (well, just approved but not ratified) support for importing/libraries. Coupled with inconsistent implementations and you end up need more layers of abstraction.
  • fivesigma 10y ago
    Browsers are the new OSes. Everything on them runs on javascript. You do the math.
  • ubercore 10y ago
    I came to the Javascript world late, starting with React. I think its tooling has gotten out of hand, but there are a couple of reasons it's not a _total_ disaster in my mind: * Partly, there are a larger number of Javascript developers that haven't been exposed to other ecosystems, and end up reinventing some wheels * More importantly though, I think the pace of innovation in the javascript ecosystem is actually good in a way, because other more mature languages (I'm not talking about age of the language, just maturity of the tooling and ecosystem) have gone through the same growing pains, it just took a lot longer. It's easy to forget, coming from the Python world for instance, the pains of easy_install and distutils vs seuptools, etc etc. I still find Java dependency management a bit of a pain, because I don't know it very well. We're just watching a sped up version of a language maturing, and it's painful as individual developers trying to keep up, but I don't think it's as negative as HN often makes it.
• Kiro 10y ago
  So, as a casual user of node... Should I switch? I don't see any immediate benefits.
  • ohitsdom 10y ago
    You don't? Consistent and reliable dependency versioning across all machines, and speed: https://yarnpkg.com/en/compare https://yarnpkg.com/en/compare
    • Kiro 10y ago
      Sure, not saying there aren't any benefits but as a casual user they don't feel big enough to justify a switch. I may be wrong though (hence why I'm asking).
      • ohitsdom 10y ago
        It took less than 20 minutes for me to change over a decently large project. The builds on the CI server increased by about 25 seconds by switching from npm install to yarn install. It was definitely worth it for me.
        • seandavidfisher 10y ago
          Wait, your build times increased on the CI server? What makes that worth it?
  • Vinnl 10y ago
    Depends on how casual :P But probably not. (Edit: Note: I mean you don't _have_ to. You might like the performance increase, but if you only use it casually, even that might not be worth the trouble.)
• edibleEnergy 10y ago
  Just installed it and ran it on a largish project ( https://www.bugreplay.com https://www.bugreplay.com ), the parallel downloads really sped things up and outputted some great diagnostic info about problematic dependencies. BugReplay is a webapp for recording webapp bugs, so having some confidence that our dependency chain isn't introducing our own bugs would be awesome :)
• joepie91_ 10y ago
  Well, this is a problematic case of "throwing out the baby with the bathwater". Semantic versioning isn't the problem here (the lack of deterministic install trees is!), yet it's thrown out pretty much entirely, without even so much as considering why it exists in the first place. I wonder whether Facebook realizes just how much damage they are going to be doing to the JS ecosystem with this approach.
  • steveklabnik 10y ago
    How is this throwing away semver?
    • joepie91_ 10y ago
      As I understand it, it automatically locks in the (exact) version of everything you install, rather than having this as a separate, optional feature (like shrinkwrap does, despite it not working well - but that's an implementation problem). The big deal about semver and how it's implemented in NPM is that there is a choice of how to deal with it - either you automatically allow compatible updates and risk an occasional maintainer fuckup, or you explicitly pin everything and take on the burden of constantly updating your versions manually and tracking security releases. The latter might work for Facebook and other large organizations, but for many smaller developers, the consequences of an accidental break are much, much less serious than those of constantly having to keep pinned dependencies up to date. End result: people don't bother at all, and insecure versions never get upgraded for the majority of the userbase. This, in turn, removes the incentive for package authors to do anything with semver at all, once Yarn's approach becomes commonplace. Why bother with semantic versioning if everybody is expected to manually check and pin their dependencies anyway? Facebook also needs to take into account their influence over the ecosystem. The reality is that whatever project Facebook puts out is immediately seen as gospel and "naturally, a well-engineered thing and a good choice to use!" - this is literally a daily argument in #Node.js when somebody is trying to argue why a poorly-fitting tool is somehow appropriate for their usecase. "Well, Facebook uses it!" In reality, the Yarn workflow is designed to work for Facebook and organizations of similar size, and in all likelihood it will break down in other situations. People will not realize this until they've already bought into it, and by then it's too late.
• atomaka 10y ago
  Why another project? If npm has been suitable and only falls short in certain cases, why not contribute back to it?
  • bronson 10y ago
    It's unlikely npm would accept patches moving it in the direction Yarn is taking. Different priorities.
• skybrian 10y ago
  The algorithm sounds very much like how Dart's "pub" command works. Anyone know how they differ?
• z3t4 10y ago
  console.log("u got powned")
• bcherny 10y ago
  I use JS+Node+NPM for my day job and many side projects. Initial thoughts: - Why didn't Facebook contribute the updates to NPM directly? - They are coupling a package manager and registry proxy; the latter has many existing implementations already - The differenced between Yarn and NPM+Shrinkwrap do not seem substantive; NPM made the design decision to use a sometimes non-deterministic install algo in NPM3 to speed up install times - when network is involved, there is a serious trade off between idempotency and speed In general, FB seems to love building their own versions of existing tools: - Flow, introduced 3 months after TypeScript - Nuclide instead of Atom/Sublime/VSCode - Jest instead of Jasmine/Mocha - DraftJS instead of (insert of of the existing 100 text editors here) - ... I get that these are useful internally at FB, but I don't think they help the community much. It would be better to work with existing tools and contribute back to them than to reinvent the wheel every time something doesn't work perfectly for their use case. I get that FB uses these as recruiting tools, it's useful for them to have rights for these projects, and it gives their engineers something to do and be excited about, but I do not want a whole dev tool ecosystem controlled by big FB. Also, I find IED's approach to speeding up builds far more novel and interesting - https://github.com/alexanderGugel/ied https://github.com/alexanderGugel/ied
  • Klathmon 10y ago
    IMO it's the "javascript way" to invent new tools instead of trying to improve others (when it makes sense to do so). Let me explain that a little bit... The javascript "ecosystem" takes the unix philosophy to the extreme in many ways. And one of the points of the unix philosophy is to try to avoid "bloating" tools with tons of options, and instead trying to create new tools where appropriate. Many of those are perfect examples of where "improving" a current tool with the changes they wanted would end up with a more complicated and difficult to use tool. - NPM makes tradeoffs that the majority of developers in the JS world want right now. Like it or not, deterministic installs isn't something that most people want or need (at least, they don't know that they might want or need it). Making a new tool that anyone can use when that is needed is a great solution, and it avoids bloating NPM with a hundred little flags and tons of code to cover edge cases. It's the perfect example because anyone can switch to using yarn when it's needed without any extra work. - Nuclide is actually just a layer on top of Atom. They wanted a bunch of things to all work one way, so rather than try to get Atom to change to include what they wanted, or require developers install a ton of extensions, they packaged them up on top of Atom as one system. - Jest had different goals at the start. Testing react components was ugly and difficult, and Jest wanted to solve that first (IIRC, i'm not very familiar with jest unfortunately). Again, it's something that can be used if needed, and can even be combined with other testing tools if you want (we almost did that at my work, used Jest to test the front-end components, and our current mocha+friends setup to test the "business logic").
• STRML 10y ago
  This is a really big deal. We have a very large/complex web application & API with a large number of dependencies, and not a single npm alternative (pnpm, ied, npm-install) could figure out the tree properly. It has been working well with our npm proxy and is basically a faster, better-deduped drop-in replacement that removes our need for npm-cache, npm-check-updates, and all of our shrinkwrap hacks. This is finally the npm rewrite we've been waiting for. In looking at the yarn repository, it's nice to see a clean, modern build, ES6 + async/await, Flow typing, and so on. This is what a well-maintained JS project looks like in 2016 that's built for production use. npm, for all its flaws, was a revolution when it was first written. But since then, it has been large, unruly, difficult to maintain, and has largely unaddressed many important production use cases. It is nice to see a group of companies using Node in production take the lead on addressing problems that really matter to production JS apps.
• hswolff 10y ago
  This is super exciting! Really excited to try this out and glad to see innovation in this area. I'm also bummed cuz I've been working on a static site generator for over a year written in node called Yarn as well[1]. I guess it's time for a rename! Any suggestions? :D [1] https://github.com/yarnjs/yarn https://github.com/yarnjs/yarn
  • wayfarer2s 10y ago
    Oh that sucks..How about "yarnish"? It's available on npm, sticks to your roots, and you can play on varnish if you like. Or you can always go the thesaurus route and use fleece, wool, cotton fiber, flaxen thread...
• BinaryIdiot 10y ago
  I'm really liking yarn. It solves some of my main issues with npm, it's fast and it provides some great output in the CLI. It's also architected absolutely ingeniously. Seriously they have positioned themselves to seamlessly take over npm's entire business just like that. What do I mean? So today by default yarn goes through its servers which then proxy to npm's. As time goes Facebook can add more features to its own repo (custom submissions instead of to npm or maybe it does a federated publish to both at first, nice looking pages with community engagement tools built in, better metrics (npm provides me with almost worthless metrics), etc). Then when enough people are using Yarn Facebook could simply...flip the switch. Then boom, zero npm usage. I would be terrified if I were NPM right now. They've sat on their hands for years not improving npm. I mean npm3 had a silly CLI animation that made all downloads take orders of magnitude longer simply because of the animation! P.S. if you're trying to use yarn with a custom npm repository just drop a .yarnrc file into your project's directory then add the following in that file: registry "http://<my http://<my register>"
  • thejameskyle 10y ago
    npm should not be terrified, we've been working with them and they've been very encouraging. From their perspective it's very difficult to make breaking changes to the client because of the sheer number of people depending on them. (We've faced similar problems on Babel) Yarn still makes use of the npm registry and all the high quality infrastructure they've built and support they provide for the ecosystem. They are a critical piece of this.
    • BinaryIdiot 10y ago
      I mean I understand the intent but it turns npm into basically a dumb pipe / just infrastructure. Every business that gets reduced to that struggles to get out of that space and expand. Granted it's far too early to write npm off. But with how slow they've moved over the years I'm unconvinced that yarn won't take over the space unless it runs into some bad problems. Npm 3 and its launch was an absolute mess and, ultimately, it's almost just a package managing tool. I am unconvinced that breaking changes is much of an issue if at all for them. They could abandon shrinkwrap into a better system and I think everyone would be happy for it; no need to keep propping up that awful system.
• zuck9 10y ago
  The node_modules folder is my least favorite part of the JS ecosystem. I realize now that's mostly because of npm-cli. npm-cli's job was simple. Take in a `package.json` as input and output the same `node_modules` folder as fast as possible. But somehow it couldn't do that even after getting 6 years old. On top of that, every now and then with when you ran `npm update` or `npm dedupe`, it messed up the `node_modules` folder with extraneous and invalid packages.
• nimish 10y ago
  Excellent. The NPM cli is a dumpster fire-- cf https://github.com/npm/npm/issues/11283 https://github.com/npm/npm/issues/11283 A better std lib for node and module LTO would be a good step forward to removing the proliferation of tiny packages
• iamleppert 10y ago
  This is not a problem with the package manager. This is a problem with complexity. When did it start becoming reasonable for a front-end only part of the MVCC pattern to have 68 dependencies? Or for a transpiler like Babel to add 100k+ files? I'm sorry I just find it ridiculous that instead of taking a look at the disease (unbounded complexity), we are looking to engineer our way out of the problem by creating a package/dependency manager that "scales". Are the frontend problems at Facebook of showing HTML forms and buttons on a page really that complicated to warrant such a behemoth of a system? This harkons back to the days at LinkedIn where I spent my time in a misguided effort to create a distributed build system because our codebase had grown so massive that it literally took a distributed system to build it and still release within the day. I feel bad for people working on these systems because while it is fun and "technically interesting" to solve "problems at scale", you're really just digging the ditch deeper for your company. You are putting lipstick on the 500 lbs pig; making room in the dump to pour in more technical debt. I don't think anyone ever imagined we'd be in a situation where a javascript framework is approaching the complexity of an operating system in terms of individual source files and SLOC. Just boggles my mind.
  • deleted 10y ago
    [deleted]
  • daveidol 10y ago
    Have you seen some of the things we are building for the web these days? Full applications, with multiple "pages", server-side rendering above the fold content, minified builds that load not only content--but code as well--on demand from the server, etc. Absolutely: this is overkill for your blog or portfolio website, but part of the rising complexity with the tooling and dependency system is due to the rising complexity of the applications themselves.
  • callumlocke 10y ago
    What is your proposed cure for the disease?
  • dwaltrip 10y ago
    The Javascript ecosystem has to deal with unrelenting, multi-decade backwards compatibility requirements; supporting numerous, inconsistent implementations; and an incredibly wide set of stakeholders and participants with diverse motivations when advancing the spec. Do any other software platforms out there face the same magnitude of complications? To speak to one of your primary examples: Babel is freaking huge and somewhat absurd, but is there any other way to advance the Javascript language? Fortunately, this is only a development dependency, and the client doesn't have to see it. On a happier note, these are good problems to have. This is another indicator of how successful the web platform has become. You can build some absolutely incredible projects these days, and they are instantly available to billions of people who are operating thousands of different hardware devices (with different specs and dimensions) running dozens of operating systems. No gatekeeper, bureaucrat, or corporate curator must approve your work. Just simply deploy it, and anyone with the URL can access it. It is a mind-blowing human accomplishment. I look forward to seeing how the web continues to progress over the coming decades.
• eknkc 10y ago
  Oh god it's fast.. It's freaking fast. Thank you, Whoever touched this. Thank you.
• fooyc 10y ago
  Why not contributing these improvements to non instead ? Why is the JavaScript community continuously creating new tools instead of contributing to existing ones ?
• nodesocket 10y ago
  Does yarn support custom `run` commands like npm? For example: "scripts": { "dev": "node api.js --config config-dev.json" } $ npm run dev
  • steveklabnik 10y ago
    Yes. See the diff here for an example of moving a project from one to another: https://github.com/rust-lang/crates.io/pull/457 https://github.com/rust-lang/crates.io/pull/457 lots of "npm run" -> "yarn run" there.
    • nodesocket 10y ago
      Sweet. Would be cool to have yarn support passing dynamic arguments into scripts/run commands.
• theanswerer 10y ago
  A big +1 for yarn from me. I had no end of trouble with npm needing internet connectivity at all times and its non-determinism. Thanks to all involved!
• substack 10y ago
  One of the features for yarn mentioned is better offline support, but you can also do this with vanilla npm: npm install --cache-min Infinity I have this aliased to `npmi` on my system and use it all the time.
  • theanswerer 10y ago
    For unknown reasons it will still try to contact the npm servers once in a hundred runs even though none of the package versions changed.
• skevy 10y ago
  Exponent has been testing out Yarn for about a month, and it's been an incredibly pleasant experience, both in working with the Yarn team and using the tool. Exponent uses a monolithic code repository (rather than many small code repos) to manage our many interdependent projects. Our repo is actually setup very similarly to Facebook's mono-repo, though obviously considerably smaller in size. We were encountering _many_ of the same problems as Facebook with our NPM setup -- long CI/CD build times, indeterminate node_modules directories that caused our projects and tools to work for some people on our team but not for others, and the inability to do "offline-only" npm installs in CI. We actually talked with our friends at Facebook about these problems, and tried many of the same approaches they did -- shrinkwrap everything, checking in node_modules, uploading the node_modules folder to separate place (we used another completely separate Git repo), etc. All these approaches either didn't work well or were difficult to maintain, especially on a small team. Yarn has fixed all these issues for us. One not-yet-super-publicized feature of Yarn (though I'm told there is a blog post coming) that has been super useful at Exponent is the ability to build a distributable offline cache of dependencies. We have a "node_modules-tarballs" folder in our mono-repo that contains tarballs of all the dependencies for every single project that we maintain at Exponent. Yarn will pull from this cache first before fetching from the NPM registry. This offers HYPER-predictability...it's a level of confidence over and above the "yarn.lock" file, and let's me sleep well at night knowing that my team can get up and running quickly no matter where they're working, how reliable their internet connection is, etc. No more worrying about "did I forget to update the shrinkwrap" -- things just work. As JS developers, we all, beginners and experts alike, can experience this "JavaScript fatigue" that we always hear about. Fighting with tools is never fun. We just want to write code! As we've started to use Yarn at Exponent, both on our engineer's machines as well as in our continuous integration and deployment environments, we've at least had one less tool to fight with. Yarn works, works fast, uses the great parts of the NPM ecosystem that we know and love, and most importantly, is _predictable_.
  • acemarke 10y ago
    I've been using the Shrinkpack tool ( https://github.com/JamieMason/shrinkpack https://github.com/JamieMason/shrinkpack ) to do that with NPM, and am trying to figure out how to do the same thing with Yarn (per my request at https://twitter.com/acemarke/status/785886788493553664 https://twitter.com/acemarke/status/785886788493553664 ). Definitely looking forward to that blog post from the Yarn team, but I'd definitely be interested in your own comments on how to use that "checked-in tarballs" approach.
• cdnsteve 10y ago
  "Excited to to recommend the @yarnpkg JS package manager npm install – 2m5s yarn (no cache) – 9s yarn (cache) – 2s" https://twitter.com/yeoman/status/785888558544334848 https://twitter.com/yeoman/status/785888558544334848
• jfmercer 10y ago
  Dear Facebook, Thank you, thank you, thank you, thank you, thank you.
• AnsemWise 10y ago
  JavaScript has enough issues that I prefer to not touch it, focusing more on backend Python work. Yarn is one of the first steps I've seen that makes me interested in JavaScript again. Excited to see where this leads the community.
• nodesocket 10y ago
  Does registry.yarnpkg.com always just proxy to the npm registry, or is it caching results and serving directly as well.
  • steveklabnik 10y ago
    Elsewhere in the thread, the devs have said its just a CNAME.
• burgerdev 10y ago
  The acronym means "Yes: another repo's name!".
• smrtinsert 10y ago
  yarn global add typescript yarn link typescript error No registered module found called "typescript". info Visit http://yarnpkg.com/en/docs/cli/link for documentation about this command. what? edit: also, $ yarn add typescript yarn add v0.15.1 [1/4] Resolving packages... [2/4] Fetching packages... [3/4] Linking dependencies... error ENOENT: no such file or directory, open 'C:\Users\******REDACTED******\@types\react\index.d.ts' at Error (native) info Visit http://yarnpkg.com/en/docs/cli/add for documentation about this command.
  • vjeux 10y ago
    Can you open an issue in the repository please :)
• losvedir 10y ago
  Very interesting! How does the lockfile work with optional dependencies? We've run into this issue[0] on npm with a bad interaction between optional dependencies and the shrinkwrap. We develop on OS X and our CI server is linux, and fsevents works on one but not the other. [0] https://github.com/npm/npm/issues/2679 https://github.com/npm/npm/issues/2679
• b123400 10y ago
  So many package managers, we need a package manager for that!
• ateevchopra 10y ago
  My results on the speed of yarn vs npm. Same repo, side by side comparison. https://twitter.com/ateevchopra/status/785914522364026880 https://twitter.com/ateevchopra/status/785914522364026880
• dorianm 10y ago
  Github link: https://github.com/yarnpkg/yarn https://github.com/yarnpkg/yarn
• klosnet 10y ago
  awesome
• niahmiah 10y ago
  Many of these problems could have been avoided by using containers
• po1nter 10y ago
  They should update the getting started section. I get a warning when trying to install it: C:\Users\******>npm install -g yarnpkg npm WARN deprecated yarnpkg@0.15.1: Please use the `yarn` package instead of `yarnpkg`
• stincity 10y ago
  I get NPM for backend development but I don't understand why frontend developers require NodeJS just to utilize it's package management. It really does not make any sense to me considering that NodeJS is server side javascript. If my backend is Python or Go or .NET or Java or whatever, now I'll need NodeJS for the frontend and only for NPM (and now this). It's a reason why I don't enjoy dealing with "modern" frontend technologies.
  • gkoberger 10y ago
    Is requiring Node really that bad? I don't write Bash or Ruby or Python or Go, but I have them all installed because various tools need them.
    • stincity 10y ago
      It doesn't make sense to require a backend tool for frontend technologies. That's just me though. So that's why I'm asking. Couldn't they just create a package system that's not based on some backend tech?
      • gkoberger 10y ago
        I guess I don't understand. How would you fetch/build/etc frontend JS files without some sort of backend technology? (Also don't forget that frontend JS has to target dozens of subtly different browsers, so we unfortunately need to "build" JS files to account for backwards compatibility, etc)
      • qudat 10y ago
        We have that and it's called `bower`. We moved away from it because having two package managers to manage javascript (one for front-end and one for back-end) is more insane than the issue you described.
        • stincity 10y ago
          Love that. Let's include a backend tool for just so our frontend stuff can work. Makes sense.
• slobbermaster 10y ago
  It appears that yarn.lock resolves exactly one hash version of a package for one semvar version of it. What happens when your project depends on packages A and B, which both depend on the same semvar version, but due to manipulation by the author, rely on different hash versions?
• johncoltrane 10y ago
  If only emojis were optional…
• tehchromic 10y ago
  It's a great read and feels good to know FB is solving problems that we all run into
• tehchromic 10y ago
  It's a great read and feels good to know FB is solving problems that we all run into!
• rbanffy 10y ago
  Yet Another... What does the R and the N stand for?
  • __raccoons__ 10y ago
    Yet Another Redundant NPM
    • Gigablah 10y ago
      Redundancy for package registries? Sounds like a good thing to me ;)
      • rbanffy 10y ago
        Redundancy needs consistency. Not sure how that will be done.
  • Doctor_Fegg 10y ago
    Guessing "Replacement for NPM".
• electic 10y ago
  Doesn't seem to be very bug free atm: yarn yarn install v0.15.1 info No lockfile found. [1/4] Resolving packages... error https://registry.yarnpkg.com/babel-code-frame https://registry.yarnpkg.com/babel-code-frame: socket hang up at createHangUpError (_http_client.js:252:15) at TLSSocket.socketCloseListener (_http_client.js:284:23) at emitOne (events.js:101:20) at TLSSocket.emit (events.js:188:7) at TCP._handle.close [as _onclose] (net.js:493:12) info Visit http://yarnpkg.com/en/docs/cli/install http://yarnpkg.com/en/docs/cli/install for documentation about this command.
• gell_mann 10y ago
  This is a somewhat unrelated question. I'm a systems programmer (I spend most of my days in the land of C and Go). I do understand the basic concepts of web apps and how you develop one. I have even built a few websites using Rails. Can somebody please explain to me: 1) How does npm fit in in the process of developing a web app. How does it manage the js files / libraries and where does it place them? 2) How is yarn going to make your life easier Please go easy on me.
  • steveklabnik 10y ago
    1. npm == bundler. One difference: it puts the source of your dependencies into a node_modules folder in the project itself, rather than in some global spot 2. npm doesn't have bundler's lockfile, but this does. It's also faster.
    • gell_mann 10y ago
      I never guessed building a package manager / dependency tracker could be such a complex problem.
      • steveklabnik 10y ago
        It's really tough! I like this post, if you want to learn more: https://medium.com/@sdboyer/so-you-want-to-write-a-package-manager-4ae9c17d9527 https://medium.com/@sdboyer/so-you-want-to-write-a-package-m...
• beefsack 10y ago
  Is it just me, or does Facebook seem to create new competing tools and technologies more often than the other majors? My inner-cynic makes me feel like everyone at Facebook is really young, as I've noticed a trait of inexperienced developers is to try to create new things over finding and improving existing tools. In reality it could be something as simple as Facebook releasing more FOSS software than others, or maybe their releases having more visibility. Either way, feels like Facebook like to pave their own way, for better or worse.
  • cpojer 10y ago
    We don't think this way. We needed to solve this problem for Facebook, explored alternatives and made the case to build Yarn. We work at Facebook to solve Facebook's engineering challenges; when we can collaborate with other companies and the open source community, that's an added bonus.
    • dacjames 10y ago
      That mindset would seem to explain why Facebook creates more competing technologies than other players.
  • nmblackburn 10y ago
    They absolutely do, it's all a big game to see who can get the biggest market share.
• throwaway6497 10y ago
  Why did we have to pick the name Yarn which clashes with Apache YARN on which Hortonworks and Cloudera depend for their survival?
• vacri 10y ago
  How is checksumming 'mega-secure'? (looking at https://yarnpkg.com/ https://yarnpkg.com/)
• Benjamin_Dobell 10y ago
  Hmm, NIH Syndrome? The justification for Yarn starts by ignoring the existence NPM Shrinkwrap. It then proceeds to mention Shrinkwrap and refer to its supposed short-comings. Whilst there absolutely were issues with Shrinkwrap in the past, they've already been addressed by NPM. NPM is slow, perhaps Yarn is faster and has some extra features. But why these couldn't be contributions to NPM seems like a serious case of NIH...
  • brandonbloom 10y ago
    > NIH Syndrome No. NPM is awful in ways that could not be fixed without braking changes. They have a need, so they satisfied it. A need which, by the way, every one of the dozen-or-so Node-using projects I've worked on have shared. > Whilst there absolutely were issues with Shrinkwrap in the past, they've already been addressed by NPM. This isn't even remotely true. Shrinkwrap is still a usability nightmare and doesn't begin to address the non-deterministic install behavior that crops up when you add new dependencies. > why these couldn't be contributions to NPM seems like a serious case of NIH There's no way that they could have got these changes in to NPM in anywhere near the time it will take for Yarn to become popular; if at all! Politics (or the avoidance thereof) are practically _the_ perfectly valid reason to justify NIH. I'll wait a while for Yarn to bake, but personally, I hope NPM dies a swift and total death.
    • Benjamin_Dobell 10y ago
      > No. NPM is awful in ways that could not be fixed without braking changes. They have a need, so they satisfied it. A need which, by the way, every one of the dozen-or-so Node-using projects I've worked on have shared. Examples, please. > This isn't even remotely true. Shrinkwrap is still a usability nightmare and doesn't begin to address the non-deterministic install behavior that crops up when you add new dependencies. Again, examples, please. > There's no way that they could have got these changes in to NPM in anywhere near the time it will take for Yarn to become popular; if at all! Politics (or the avoidance thereof) are practically _the_ perfectly valid reason to justify NIH. Did the Yarn developers make any attempt to contribute to NPM at all? If they did and were turned back, then sure, I understand. However, everything I've read about Yarn thus-far is just vague finger-pointing at NPM. I'm by no means suggesting NPM is perfect, for one I find it incredibly slow. However, surely that can be optimised. However, regarding dependency version locking, which is the problem Yarn is trumpeting, what are the actual issues with Shrinkwrap? I'd like something specific. Since Shrinkwrap was updated to keep a flat sorted structure (a while ago) I've been using it no fuss at all. What are the specific issues people are running into now? I've seen comparisons with Bundler, and how Yarn will be more like Bundler's Gemfile.lock files. However, the thing is the current version of Shrinkwrap works almost identically to Bundler's lockfile e.g. When I update a dependency the shrinkwrap file automatically updates (at least that's the default behaviour).
• nikolay 10y ago
  Coming soon to Homebrew: https://github.com/Homebrew/homebrew-core/pull/5832 https://github.com/Homebrew/homebrew-core/pull/5832
• ilaksh 10y ago
  Maybe instead of reinventing npm use a more general staging or deploment tool/strategy like Docker.
• philwhln 10y ago
  Just glad to see that there is no "yarn isntall" command.
• itsbits 10y ago
  Surely reliability will be improved with lockfile concept but why do we need another package manager with this. Can't we suggest same to npm team?
• pylinux 10y ago
  Anybody know what yarn does when it encounters a "*" version?
• kabes 10y ago
  NPM must have been the only thing the JS community has standardised on. Now that's gone too
• incredulous 10y ago
  It's incredible that a tool that we've tolerated for years, npm, is instantly and unceremoniously dumped for a superior offering, yarn. Technology changes on a dime.
• ensiferum 10y ago
  I don't know JS and I've never used these js package managers, but are these issues such that they couldn't have been fixed in the original package manager? Smells like NIH that is just going to create head ache and misery for everyone in the JS community in the long term.
• romanovcode 10y ago
  Doesn't work for me. My gulp builds are failing because they are missing some dependencies. I'll continue using npm thank you very much.
• ephimetheus 10y ago
  Hm I like the idea, but in my testing, yarn is about 2 times slower than regular npm3... EDIT: Even when it's hitting the cache, it's only ~5% faster than clean NPM with networking for me...
• z3t4 10y ago
  I think it's naive to not review dependency updates. Also if a minor or patch update touches tens of thousands of files, you are dealing with a monster. And monsters need to be in cages.
• jiyinyiyong 10y ago
  In China, we have using part of yarn for long, installation speed can be boosted with https://github.com/cnpm https://github.com/cnpm .
• niix 10y ago
  Awesome! So far this is working great with a Dockerized Node application I have. Great job Facebook and the rest who contribute!
  • mfornasa 10y ago
    It could finally solve the major pain in using Node over Docker for CI and local development. https://medium.com/@mfornasa/using-yarn-with-docker-c116ad289d56 https://medium.com/@mfornasa/using-yarn-with-docker-c116ad28...
• ostater 10y ago
  yarn's immediate acceptance by the community over npm appears to be a wholesale rejection of semver and an embrace of dependency determinism. Every dependency is locked down to the patch level with yarn. Instead of major.minor.patch, it might just as well be a single incrementing number now.
• sontek 10y ago
  Has anyone successfully got this to work? I was excited to try it so I did: $ rm -rf ./node_modules $ npm install yarnpkg $ ./node_modules/yarnpkg/bin/yarn and it just freezes up and never finishes: [4/4] Building fresh packages... [1/3] ⠈ node-sass [2/3] ⠈ radium [3/3] ⠈ phantomjs-prebuilt [-/3] ⠈ waiting... [-/3] ⠈ waiting... This is not a particularly complex app, the dependencies: http://paste.openstack.org/show/585495/ http://paste.openstack.org/show/585495/
• ranyefet 10y ago
  As a JavaScript developer I can't say how much excited I am about this new tool. Most of my problems with developing modern JavaScript apps are around npm client, long installs and huge dependencies folder. I hope it will improve everybody workflows.
• deleted 10y ago
  [deleted]
• akhatri_aus 10y ago
  So how do you install packages like `npm install` without adding it to the package.json file?
  • skrebbel 10y ago
    You don't. If you want to be able to do stuff like that (and inevitably run into "hmm, works on my machine"-type bugs once you start sharing your code with others), you need to keep using npm. This kind of liberty is the sort of stuff npm was designed for. It is also why npm has been causing trouble in many cases.
    • akhatri_aus 10y ago
      I was under the impression that this was about speed. There are many use cases for this functionality, the design of a software package shouldn't instill dogma into its users. Users use software for their needs, not to sign up to a religion.
      • skrebbel 10y ago
        No, speed is bonus. Yarn is about reproducible dev/test/build setups. Same yarn.lock - same node_modules content. That's the primary goal. If you think consistency across computers is a religion then I don't think we're going to find a lot of common ground here though. Finally, yarn is not intended to replace the npm client. It is intended to replace it for a certain use case, which happens to be one many companies have. Unlike Yarn, Npm is highly flexible and obviously you like it like that. I strongly doubt it's going to stop being developed all of a sudden because some other people made Yarn. So what, exactly, is the problem?
        • akhatri_aus 10y ago
          > So what, exactly, is the problem? Follow users needs, not your idea of what they are or the problems with npm are.
• ChoHag 10y ago
  You're going to need these: "nother"
• spasquali 10y ago
  Or just use shrinkpack: https://github.com/JamieMason/shrinkpack https://github.com/JamieMason/shrinkpack But that would be too easy...
  • remi 10y ago
    Did you read the article paragraph about shrinkwrap?
    • spasquali 10y ago
      Not `shrinkwrap` -> `shrinkpack`. Not sure why Facebook spent so much time and money to create a more complicated "solution", or why so many people don't realize that.
• hiendv 10y ago
  Yarn shouldn't be installed from the package repository.The inconsistency between the version of nodejs which yarn is running on and the version of nodejs which npm is running on brings significant effects. https://github.com/yarnpkg/rfcs/issues/9 https://github.com/yarnpkg/rfcs/issues/9 Any idea how to reach out to one of the authors?
• tofupup 10y ago
  neat