5 ms·

I wrote a post explaining why I'm psyched to be working on it: TLDR: - open, community governance that will support long-term evolution - the technical detai

by wycats 10y ago

I wrote a post explaining why I'm psyched to be working on it: TLDR: - open, community governance that will support long-term evolution - the technical details get a lot right out of the gate (decent performance, predictability, and security)

zamalek 10y ago
Have you guys approached the ridiculous folder nesting situation? E.g. breaking out of the current/broken node_modules structure?
- petetnt 10y ago
  There has been no problems with folder nesting in `npm` in general since the version@3 came out over a year ago.
  - steveklabnik 10y ago
    (other than its non-determinism)
    - thejameskyle 10y ago
      And Yarn it deterministic! :party_parrot:
  - spankalee 10y ago
    This isn't true though. npm chooses the first version of a package it encounters to install at the top level. Every other version is installed nested. If you have N uses of version A and one use of version B, but version B is installed first, then you get N copies of the package at version A.
  - ksherlock 10y ago
    It's improved but it's not fixed. if a depends on c 1.0 and b depend on c 2.0, both versions of c need to be installed and one of them will be nested.
    - Kenan 10y ago
      What's the solution for that case though (one which requires no nesting)? I don't think a solution with no nesting exists given the current module resolution algorithm of Node.js, which allows for only a single version of a particular package to exist at a given level.
      - ksherlock 10y ago
        nesting can't be prevented in that case (at least with the current node require() design). It gets really bad if you have 10 packages that depend on c@1.0.0 and 10 packages that depend on c@2.0.0 -- one of them will install in the root directory and the other 10 will be duplicated. ied stores a single copy of each package@version and uses symlinks which is an improvement. Apparently yarn tried something similar but it breaks some packages (that check where they're located for example). https://github.com/alexanderGugel/ied https://github.com/alexanderGugel/ied
        Osiris 10y ago
        I could be prevented. Another project does this by using a hash of the project to all all dependencies at one level then symlinks the dependencies from hash to name so it references the correct version.
    - l1ambda 10y ago
      ...which is actually good. With Java and maven, you cannot use multiple versions of a library.
  - mvdanj 10y ago
    fwiw npm really needs an `exec` command like bundler has (ie `npm exec bin-cmd`). "$(npm bin)/bin-cmd" is getting really old
    - josephfrazier 10y ago
      You can use `npm run env bin-cmd` for that. It's not as convenient as `npm exec bin-cmd` would be, but it might be easier to type than `$(npm bin)/bin-cmd`.
  - mvdanj 10y ago
    fwiw npm really needs an `exec` command like bundler has (ie `npm exec bin-cmd`). "$(npm bin)/bin-cmd" is getting really old
  - zamalek 10y ago
    > no problems Absolute and utter hogwash. I don't even need to get into versioning to make this argument but I'll exclude it for clarity's sake. Simple scenario: root -> A root -> B root -> C A -> B B -> C What folder structure is created? root/node_modules/A/node_modules/B/node_modules/C root/node_modules/B/node_modules/C root/node_modules/C Symlink that all you want, how is it in any way whatsoever thoughtful and elegant? It's not a real solution by any stretch of the imagination. Myriads of pre-existing systems that correctly deal with this scenario existed at the time that NPM was concoted. Ignored. Considering that node is touted as a cross-platform solution, how well does version@3 deal with filesystems that don't support symlinks? I've run into this: not at all. NPM version@3 could not be further from the ethos of "engineering." It's an outright hack, and it quickly betrays its limitations. It could have been far simpler, far more considerate of pre-existing wisdom.
    - secoif 10y ago
      what you described is npm@2's behaviour. npm@3 will flatten dependencies as much as possible
      - zamalek 10y ago
        How does it flatten dependencies? Symlinks?
        secoif 10y ago
        no, it just installs everything it can into the top level node_modules. npm@2 would do this to a degree as well, it was just less sophisticated about it.
        zamalek 10y ago
        > into the top level node_modules. Just tried using npm3 explicitly and I stand corrected.
- woah 10y ago
  What is the problem with nested folders?
sophiebits 10y ago
You forgot the link. :) http://yehudakatz.com/2016/10/11/im-excited-to-work-on-yarn-the-new-js-package-manager-2/ http://yehudakatz.com/2016/10/11/im-excited-to-work-on-yarn-...
- wycats 10y ago
  Yikes, thanks!
raesene6 10y ago
Interested to see the emphasis on security, it's definitely something that could use more focus in this area. On that note, are there any plans to implement TUF (https://theupdateframework.github.io/ https://theupdateframework.github.io/) or similar into Yarn?
mvdanj 10y ago
You say "decent performance" and "predictability". What is the basis of this claim? I've heard these all before, but unless you've actually shipped a product using this tool I don't know how you can back this up.
- steveklabnik 10y ago
  > unless you've actually shipped a product using this tool Given that wycats _worked on_ this tool, I would assume it's his experience helping build it that's the basis of the claim. (I don't work at Tilde so I can't tell you if it's been shipping with yarn, but it would shock me if it wasn't.)
  - wycats 10y ago
    We're not using Yarn at Tilde yet, but I've been using it on Yarn itself and when working on Ember packages, and it's worked well. As far as performance, we have good benchmarks tracking yarn's performance in a whole bunch of different scenarios (https://yarnpkg.com/en/compare https://yarnpkg.com/en/compare)
  - mvdanj 10y ago
    Ok, didn't realize he worked on it. Bundler is great so that gives me some hope for Yarn. But in general I think its more stable in the long-term for projects to use open standards instead of vendor solutions for browser package management.
    - steveklabnik 10y ago
      I can understand this sentiment in general, but don't think it applies here. npm is also a private company, and this does work with their registry. There currently isn't a vendor-neutral standard to follow.
- jameside 10y ago
  As for performance, Yarn is about 3-5x faster installing all of the dependencies and devDependencies of react-native. The benchmarks are here: https://yarnpkg.com/en/compare https://yarnpkg.com/en/compare. It's much faster in most scenarios, especially the ones that used to take minutes.
gjolund 10y ago
When do you expect ember to transition to yarn? Are there any branches of ember-cli implementing yarn?

by wycats 10y ago

zamalek 10y ago

Have you guys approached the ridiculous folder nesting situation? E.g. breaking out of the current/broken node_modules structure?

petetnt 10y ago
There has been no problems with folder nesting in `npm` in general since the version@3 came out over a year ago.
- steveklabnik 10y ago
  (other than its non-determinism)
  - thejameskyle 10y ago
    And Yarn it deterministic! :party_parrot:
- spankalee 10y ago
  This isn't true though. npm chooses the first version of a package it encounters to install at the top level. Every other version is installed nested. If you have N uses of version A and one use of version B, but version B is installed first, then you get N copies of the package at version A.
- ksherlock 10y ago
  It's improved but it's not fixed. if a depends on c 1.0 and b depend on c 2.0, both versions of c need to be installed and one of them will be nested.
  - Kenan 10y ago
    What's the solution for that case though (one which requires no nesting)? I don't think a solution with no nesting exists given the current module resolution algorithm of Node.js, which allows for only a single version of a particular package to exist at a given level.
    - ksherlock 10y ago
      nesting can't be prevented in that case (at least with the current node require() design). It gets really bad if you have 10 packages that depend on c@1.0.0 and 10 packages that depend on c@2.0.0 -- one of them will install in the root directory and the other 10 will be duplicated. ied stores a single copy of each package@version and uses symlinks which is an improvement. Apparently yarn tried something similar but it breaks some packages (that check where they're located for example). https://github.com/alexanderGugel/ied https://github.com/alexanderGugel/ied
      - Osiris 10y ago
        I could be prevented. Another project does this by using a hash of the project to all all dependencies at one level then symlinks the dependencies from hash to name so it references the correct version.
  - l1ambda 10y ago
    ...which is actually good. With Java and maven, you cannot use multiple versions of a library.
- mvdanj 10y ago
  fwiw npm really needs an `exec` command like bundler has (ie `npm exec bin-cmd`). "$(npm bin)/bin-cmd" is getting really old
  - josephfrazier 10y ago
    You can use `npm run env bin-cmd` for that. It's not as convenient as `npm exec bin-cmd` would be, but it might be easier to type than `$(npm bin)/bin-cmd`.
- mvdanj 10y ago
  fwiw npm really needs an `exec` command like bundler has (ie `npm exec bin-cmd`). "$(npm bin)/bin-cmd" is getting really old
- zamalek 10y ago
  > no problems Absolute and utter hogwash. I don't even need to get into versioning to make this argument but I'll exclude it for clarity's sake. Simple scenario: root -> A root -> B root -> C A -> B B -> C What folder structure is created? root/node_modules/A/node_modules/B/node_modules/C root/node_modules/B/node_modules/C root/node_modules/C Symlink that all you want, how is it in any way whatsoever thoughtful and elegant? It's not a real solution by any stretch of the imagination. Myriads of pre-existing systems that correctly deal with this scenario existed at the time that NPM was concoted. Ignored. Considering that node is touted as a cross-platform solution, how well does version@3 deal with filesystems that don't support symlinks? I've run into this: not at all. NPM version@3 could not be further from the ethos of "engineering." It's an outright hack, and it quickly betrays its limitations. It could have been far simpler, far more considerate of pre-existing wisdom.
  - secoif 10y ago
    what you described is npm@2's behaviour. npm@3 will flatten dependencies as much as possible
    - zamalek 10y ago
      How does it flatten dependencies? Symlinks?
      - secoif 10y ago
        no, it just installs everything it can into the top level node_modules. npm@2 would do this to a degree as well, it was just less sophisticated about it.
        zamalek 10y ago
        > into the top level node_modules. Just tried using npm3 explicitly and I stand corrected.
woah 10y ago
What is the problem with nested folders?

sophiebits 10y ago

You forgot the link. :) http://yehudakatz.com/2016/10/11/im-excited-to-work-on-yarn-the-new-js-package-manager-2/ http://yehudakatz.com/2016/10/11/im-excited-to-work-on-yarn-...

wycats 10y ago
Yikes, thanks!

raesene6 10y ago

Interested to see the emphasis on security, it's definitely something that could use more focus in this area. On that note, are there any plans to implement TUF (https://theupdateframework.github.io/ https://theupdateframework.github.io/) or similar into Yarn?

mvdanj 10y ago

You say "decent performance" and "predictability". What is the basis of this claim? I've heard these all before, but unless you've actually shipped a product using this tool I don't know how you can back this up.

steveklabnik 10y ago
> unless you've actually shipped a product using this tool Given that wycats _worked on_ this tool, I would assume it's his experience helping build it that's the basis of the claim. (I don't work at Tilde so I can't tell you if it's been shipping with yarn, but it would shock me if it wasn't.)
- wycats 10y ago
  We're not using Yarn at Tilde yet, but I've been using it on Yarn itself and when working on Ember packages, and it's worked well. As far as performance, we have good benchmarks tracking yarn's performance in a whole bunch of different scenarios (https://yarnpkg.com/en/compare https://yarnpkg.com/en/compare)
- mvdanj 10y ago
  Ok, didn't realize he worked on it. Bundler is great so that gives me some hope for Yarn. But in general I think its more stable in the long-term for projects to use open standards instead of vendor solutions for browser package management.
  - steveklabnik 10y ago
    I can understand this sentiment in general, but don't think it applies here. npm is also a private company, and this does work with their registry. There currently isn't a vendor-neutral standard to follow.
jameside 10y ago
As for performance, Yarn is about 3-5x faster installing all of the dependencies and devDependencies of react-native. The benchmarks are here: https://yarnpkg.com/en/compare https://yarnpkg.com/en/compare. It's much faster in most scenarios, especially the ones that used to take minutes.

gjolund 10y ago

When do you expect ember to transition to yarn? Are there any branches of ember-cli implementing yarn?