10 ms·

Umask setting is ignored by npm for some directories

56 points by kpcyrd 10y ago

nathancahill 10y ago
Reminds me of PHP back in the day. Doesn't work? chmod -R 777 /var/www
- nolok 10y ago
  Because that's wasn't PHP, that was (and still is) shitty devs
  - deleted 10y ago
    [deleted]
- ars 10y ago
  That has nothing to do with PHP and everything to do with that the webserver runs as a different user than the creator of the files. Unless you have root it's rather difficult to make a directory that the webserver can write to. It's not specific to PHP, except that PHP is used for web scripting. The correct way to do it, if you don't have root is: Make a directory 777 (yes, you do need to). Have the webserver create a subdirectory owned by the webserver. Change the parent directory away from 777 to 755 or 775. Double check that no one else created something in there while it was wide open. Be aware that ALL users on that machine who can run web scripts have access to that directory. An even better way is with mpm-itk
- mst 10y ago
  freenode #perl still periodically has to talk somebody trying to get a CGI script running out of using 777 to avoid having to figure out permissions properly.
  - rhizome 10y ago
    It's an evergreen topic, I've seen this in so many contexts in almost 20 years in IRC help channels that it's a shibboleth for rookie status in configuring systems. Not only that, but it's a communicable weakness in that it's always passed on from person to person: if you only taught yourself using Google, it's actually hard to find 777 as advice. If you see an SO comment/answer recommending 777, it's invariably downvoted into oblivion. This increased when devops came on the scene, and it was probably one of the first things that made me conclude that devops was about pushing sysadmin skills on unprepared developers. /rant
  - kuschku 10y ago
    It’s also an interesting task: How do you configure a folder so that two users, who are in no common groups, can read, write and execute everything? Even if the other user uploaded it with 700 set?
    - Ded7xSEoPKYNsDd 10y ago
      If there's really no way to run these processes as the same group at least, there's always ACLs: setfacl -m u:NON-OWNER:rwX,d:u:NON-ONWER:rwX,d:u:OWNER:rwX DIR (Allows NON-OWNER to read, write, execute if already executable for anyone to that directory, and allows NON-OWNER and OWNER to do the same for newly created files.)
      - kuschku 10y ago
        I’m not sure that solves the issue of a user (user:user) uploading and creating files via SFTP, but www-data:www-data having to execute, read, and modify them.
        Ded7xSEoPKYNsDd 10y ago
        It works pretty well for me, to get Web DAV access to my files on my NAS. The default ACLs make sure that any newly created files can be accessed by both users.
tyingq 10y ago
This bit is interesting: What I observed (even with newest npm 3.10.9) that it sometimes creates node_modules directories with permission 777. when doing the npm install multiple times the results vary, most of the time it results in 755 but sometimes in 777). This seems to have nothing to do with the source tarballs content (retrieved from registry.npmjs.org) but a more general issue. As mentioned, its not deterministic and the tarballs definitively don't contain any files/directories with such permissions. If that's true, it may take a while to unearth the root cause.
- varikin 10y ago
  I thought further up it mentioned that it starts with the permissions of the files in the tarball, then grabs process.umask (of the npm-tarball subprocess maybe?) and then overrides the execute bit. If that is correct, that is 2 layers of questionable permissions being applied leading to non-deterministicness.
  - tyingq 10y ago
    Perhaps, but the author of the comment I quoted seemed to be performing the install over and over, on the same machine. I assume neither the tarball permissions nor the umask would be changing, so it's odd.
    - varikin 10y ago
      Well that further complicates it.
      - jacquesm 10y ago
        Does it fork off any background processes? If so that would be the first place to look, the code executing in parallel with the main program.
  - kpcyrd 10y ago
    This bug was rediscovered 1 year later after a distro shipped a package with /usr/lib/node_modules/ being set to 777 in a deterministic build environment. This seems to be a race condition since it may or may not happen on multiple rebuilds.
xiwenc 10y ago
Yesterday after I did `pacman -Syu` on my Arch Linux I noticed the following: ( 95/121) installing grunt-cli [########################################] 100% warning: directory permissions differ on /usr/bin/ filesystem: 755 package: 777 warning: directory permissions differ on /usr/lib/node_modules/ filesystem: 755 package: 777 :facepalm:
- clux 10y ago
  Managing node with package managers feels like a a bit of a fools errand at the moment. They are going to be wrapping npm somehow, but when npm can't even do things right..
  - dima55 10y ago
    They absolutely should not be wrapping npm. npm is attempting to do a job that package managers should be doing to begin with
    - j3097736 10y ago
      Real package managers should be left to manage system packages, where packages are often supported for a few years. Package managers for scripting languages and similar should be left to manage user-wide packages, or virtualenv-wide for webapps.
    - int_19h 10y ago
      The problem is that there is no single package manager that covers every platform. If you are writing an inherently portable library in node.js, as an author, are you supposed to also provide packages for RPM, dpkg, pacman, brew etc? Or do you expect distros to package every tiny thing? So in practice there is a niche that npm covers, that wouldn't be filled if we removed it. On the question of whether the system package manager should wrap npm - the reason why you want to do so is because it lets npm dependency resolution work regardless of how packages are installed. If the system package manager just does its own thing, then next time you do need to npm install a package (because it's a relatively obscure package that's not in your distro), you don't want it to install copies (or worse, yet overwrite) all the dependencies that you've already installed by other means.
Matachines 10y ago
The lack of activity in the issue is sadly not rare for npm.
- AgentME 10y ago
  Yeah, I reported https://github.com/npm/npm/issues/10463 https://github.com/npm/npm/issues/10463 (npm fails to install everything from a shrinkwrap sometimes -- the feature specifically for locking down the dependencies to make things deterministic/reproducible!) about a year ago and it hasn't received much attention. It was labeled "big-bug" once but got downgraded to "support" somewhere along the line. I use npm in a continuous integration/deployment system for a project. The system used to do a git checkout, `npm install` to get the dependencies, and then a build, but then we ran into a few extremely confusing issues where we actually had a couple old dependencies tested and deployed into production because npm install failed to respect the shrinkwrap fully (yet it still exited successfully). We finally had to write a script which compares the node_modules directory against the shrinkwrap, and if there's a mismatch, it removes the entire directory and re-runs `npm install`. Thankfully `npm install` tends to install everything fine from a fresh slate, but it's just soo slow (which is also surprising to me, because shouldn't most everything be cached into ~/.npm if I've installed nearly all of the same dependencies recently? I would think that installing from a shrinkwrap of many things I've installed before would mostly be un-tarring things from cache)...
- labster 10y ago
  Not just npm. My umask bug in Firefox has just celebrated its fifth birthday. https://bugzilla.mozilla.org/show_bug.cgi?id=688944 https://bugzilla.mozilla.org/show_bug.cgi?id=688944
- doublerebel 10y ago
  Unfortunately I have noticed the same with multiple issues. Seems that's an organizational problem that yarn can't solve.

56 points by kpcyrd 10y ago

nathancahill 10y ago

Reminds me of PHP back in the day. Doesn't work? chmod -R 777 /var/www

nolok 10y ago
Because that's wasn't PHP, that was (and still is) shitty devs
- deleted 10y ago
  [deleted]
ars 10y ago
That has nothing to do with PHP and everything to do with that the webserver runs as a different user than the creator of the files. Unless you have root it's rather difficult to make a directory that the webserver can write to. It's not specific to PHP, except that PHP is used for web scripting. The correct way to do it, if you don't have root is: Make a directory 777 (yes, you do need to). Have the webserver create a subdirectory owned by the webserver. Change the parent directory away from 777 to 755 or 775. Double check that no one else created something in there while it was wide open. Be aware that ALL users on that machine who can run web scripts have access to that directory. An even better way is with mpm-itk
mst 10y ago
freenode #perl still periodically has to talk somebody trying to get a CGI script running out of using 777 to avoid having to figure out permissions properly.
- rhizome 10y ago
  It's an evergreen topic, I've seen this in so many contexts in almost 20 years in IRC help channels that it's a shibboleth for rookie status in configuring systems. Not only that, but it's a communicable weakness in that it's always passed on from person to person: if you only taught yourself using Google, it's actually hard to find 777 as advice. If you see an SO comment/answer recommending 777, it's invariably downvoted into oblivion. This increased when devops came on the scene, and it was probably one of the first things that made me conclude that devops was about pushing sysadmin skills on unprepared developers. /rant
- kuschku 10y ago
  It’s also an interesting task: How do you configure a folder so that two users, who are in no common groups, can read, write and execute everything? Even if the other user uploaded it with 700 set?
  - Ded7xSEoPKYNsDd 10y ago
    If there's really no way to run these processes as the same group at least, there's always ACLs: setfacl -m u:NON-OWNER:rwX,d:u:NON-ONWER:rwX,d:u:OWNER:rwX DIR (Allows NON-OWNER to read, write, execute if already executable for anyone to that directory, and allows NON-OWNER and OWNER to do the same for newly created files.)
    - kuschku 10y ago
      I’m not sure that solves the issue of a user (user:user) uploading and creating files via SFTP, but www-data:www-data having to execute, read, and modify them.
      - Ded7xSEoPKYNsDd 10y ago
        It works pretty well for me, to get Web DAV access to my files on my NAS. The default ACLs make sure that any newly created files can be accessed by both users.

tyingq 10y ago

This bit is interesting: What I observed (even with newest npm 3.10.9) that it sometimes creates node_modules directories with permission 777. when doing the npm install multiple times the results vary, most of the time it results in 755 but sometimes in 777). This seems to have nothing to do with the source tarballs content (retrieved from registry.npmjs.org) but a more general issue. As mentioned, its not deterministic and the tarballs definitively don't contain any files/directories with such permissions. If that's true, it may take a while to unearth the root cause.

varikin 10y ago
I thought further up it mentioned that it starts with the permissions of the files in the tarball, then grabs process.umask (of the npm-tarball subprocess maybe?) and then overrides the execute bit. If that is correct, that is 2 layers of questionable permissions being applied leading to non-deterministicness.
- tyingq 10y ago
  Perhaps, but the author of the comment I quoted seemed to be performing the install over and over, on the same machine. I assume neither the tarball permissions nor the umask would be changing, so it's odd.
  - varikin 10y ago
    Well that further complicates it.
    - jacquesm 10y ago
      Does it fork off any background processes? If so that would be the first place to look, the code executing in parallel with the main program.
- kpcyrd 10y ago
  This bug was rediscovered 1 year later after a distro shipped a package with /usr/lib/node_modules/ being set to 777 in a deterministic build environment. This seems to be a race condition since it may or may not happen on multiple rebuilds.

xiwenc 10y ago

Yesterday after I did `pacman -Syu` on my Arch Linux I noticed the following: ( 95/121) installing grunt-cli [########################################] 100% warning: directory permissions differ on /usr/bin/ filesystem: 755 package: 777 warning: directory permissions differ on /usr/lib/node_modules/ filesystem: 755 package: 777 :facepalm:

clux 10y ago
Managing node with package managers feels like a a bit of a fools errand at the moment. They are going to be wrapping npm somehow, but when npm can't even do things right..
- dima55 10y ago
  They absolutely should not be wrapping npm. npm is attempting to do a job that package managers should be doing to begin with
  - j3097736 10y ago
    Real package managers should be left to manage system packages, where packages are often supported for a few years. Package managers for scripting languages and similar should be left to manage user-wide packages, or virtualenv-wide for webapps.
  - int_19h 10y ago
    The problem is that there is no single package manager that covers every platform. If you are writing an inherently portable library in node.js, as an author, are you supposed to also provide packages for RPM, dpkg, pacman, brew etc? Or do you expect distros to package every tiny thing? So in practice there is a niche that npm covers, that wouldn't be filled if we removed it. On the question of whether the system package manager should wrap npm - the reason why you want to do so is because it lets npm dependency resolution work regardless of how packages are installed. If the system package manager just does its own thing, then next time you do need to npm install a package (because it's a relatively obscure package that's not in your distro), you don't want it to install copies (or worse, yet overwrite) all the dependencies that you've already installed by other means.

Matachines 10y ago

The lack of activity in the issue is sadly not rare for npm.

AgentME 10y ago
Yeah, I reported https://github.com/npm/npm/issues/10463 https://github.com/npm/npm/issues/10463 (npm fails to install everything from a shrinkwrap sometimes -- the feature specifically for locking down the dependencies to make things deterministic/reproducible!) about a year ago and it hasn't received much attention. It was labeled "big-bug" once but got downgraded to "support" somewhere along the line. I use npm in a continuous integration/deployment system for a project. The system used to do a git checkout, `npm install` to get the dependencies, and then a build, but then we ran into a few extremely confusing issues where we actually had a couple old dependencies tested and deployed into production because npm install failed to respect the shrinkwrap fully (yet it still exited successfully). We finally had to write a script which compares the node_modules directory against the shrinkwrap, and if there's a mismatch, it removes the entire directory and re-runs `npm install`. Thankfully `npm install` tends to install everything fine from a fresh slate, but it's just soo slow (which is also surprising to me, because shouldn't most everything be cached into ~/.npm if I've installed nearly all of the same dependencies recently? I would think that installing from a shrinkwrap of many things I've installed before would mostly be un-tarring things from cache)...
labster 10y ago
Not just npm. My umask bug in Firefox has just celebrated its fifth birthday. https://bugzilla.mozilla.org/show_bug.cgi?id=688944 https://bugzilla.mozilla.org/show_bug.cgi?id=688944
doublerebel 10y ago
Unfortunately I have noticed the same with multiple issues. Seems that's an organizational problem that yarn can't solve.