10 ms·
I understand that this is a hairy, messy problem they are solving, and that they traded off one aspect of the problem for another (Install order dependency! as
by kylecordes 11y ago
I understand that this is a hairy, messy problem they are solving, and that they traded off one aspect of the problem for another (Install order dependency! as a new "feature" in 2015!).
But I wish they had aimed higher. The better goal would be that the entire state of the node modules directory is a pure function of the contents of the package.json file plus the platform details (compiler used for native modules).
While they're at it, the ecosystem could be improved considerably if there was some sort of obvious "penalty" applied to any package that compiles native code, because such things cause considerable extra trouble for Windows users. A visible penalty, transitively carried up the dependency tree, would discourage use of such modules with native code; projects would use them (depend on them) only if absolutely necessary instead of accidentally, all over, all the time.
- city41 11y agoWhy do dev communities have to continually pay the Windows tax? If Microsoft keeps insisting on being an outlier in the world of OSes, shouldn't they be the ones burdened? I gotta say, as someone who never ever uses Windows but maintains a couple of open source packages, I'm really sick of the Windows only problems that crop up.
- tvanantwerp 11y agoAs someone who has no choice but to use Windows at work, I'm really sick of all the problems that crop up because so many tools use were written by people who won't use Windows and haven't tested anything on it.
- city41 11y agoI feel your pain, but am not convinced open source maintainers should have to be the ones to resolve the issue. Especially since so many projects are done using spare time and with no compensation.
- saosebastiao 11y agoIf I had no choice but to use Windows at work, and using Windows meant I couldn't use a large amount of development tools that I rely on, then the last ones I would blame would be the open source developers that won't use or develop for Windows. Your employer is the problem here.
- krisdol 11y agoWhy do you expect developers to pay for software, OSes, and licenses they don't use in order to develop open source software for free? Help them out by contributing, or blame Microsoft who has the resources to make their own OS more compatible Doesnt Microsoft have their own V8 or node.js fork now?
- ploxiln 11y agoKinda funny isn't it, after a decade where one might be required to use a proprietary VPN or email client that only runs on windows. And those things didn't use standard, published, or even consistent protocols, nor were they open-source. In fact they were hostile to compatible implementations. They had to be reverse-engineered. So now you get just a little, watered-down taste of using the non-preferred platform for something.
- tsuraan 11y agoThat's a funny point of view. Windows is made by a multi-billion dollar company, rented by countless wealthy software development shops, but it's volunteer open-source programmers who are supposed to take on the burden of supporting it? Seriously, I can't even comprehend that thought process. I write software for fun, I release it on github, and some people find it useful. If there are other people who don't, they can improve it for their own purposes, or they can find or write something better. I don't think anybody has a right to expect me to support Microsoft's product. That's insane.
- Silhouette 11y agoI don't think anybody has a right to expect me to support Microsoft's product. That's insane. True enough, though it's also fair to consider a language/platform/ecosystem inferior for some work if it relies on that kind of individual and often less portable contribution for its effectiveness. A lot of FOSS advocates will promote community support and a broad contributor base as advantages of that style of development, but there is another side to that coin, which is that sometimes you get just what you (didn't) pay for and you're on your own in terms of support or working around any problems.
- ag_dubs 11y agohi! i totally know this is a common feel for many npmjs users. this is why i've been running an empathy campaign for Windows, Windows Wednesday, where i do all my work (with/for npm) on windows. it's been good for finding bugs, either technical or UX. i'm working on improving the docs to make them friendlier to all platforms (not just OSX, which is the most common platform at npmjs). npmjs cares about the enterprise + windows.
- city41 11y agoI think this is the wrong approach. We shouldn't pick up MS's slack, MS should. This just enforces the current situation and keeps enabling MS to ignore the problem.
- gelasio 11y agoSo, are you of the opinion that OSS projects should never be built for any commercial, proprietary OS or just Windows (i.e. because it's not Unix)? For instance - OS X window managers suck. Do you think that a project like Spectacle should have never been built because they're just picking up Apple's slack?
- city41 11y agoI don't see how a project like Spectacle, which is OSX specific, relates. If someone wants to make an OS specific project, more power to them. The issue arises on platforms that span OSes, and they just never quite span onto Windows correctly. Microsoft is very happy that Node, Java, Clojure, etc work on Windows. It helps them a lot. But they make little to no effort to help bridge the gap, they usually leave that to everyone else. That's my gripe, right there. And to be fair, this is not a black and white situation. It's complex and Microsoft's role in all of this is complex too. And sometimes this issue does crop up on other OSes, for example Docker on OSX is sometimes a little painful.
- gelasio 11y ago"But they make little to no effort to help bridge the gap, they usually leave that to everyone else." OK, but that's also where you're wrong, right there. In general, but also particularly for this exact issue - Microsoft considered attempting to fix the file path limit for the first release of Windows 10. Maybe you don't understand, but it's going to take a herculean effort to fix this. They want to do it and if you need proof of that I can find posts of them talking about what I just said.
- Maarten88 11y agoYes, npm is very problematic/buggy on Windows. I wasted many hours because of https://github.com/npm/npm/issues/9696 https://github.com/npm/npm/issues/9696 (meaning any build on visual studio online breaks ~50% of the time simply running 'npm install'). Microsoft should finally fix the MAX_PATH issue in Windows, but npm should just fix their software: it's too important to be that buggy.
- frik 11y agoMicrosoft initially aimed POSIX compatibility with Windows NT 3.1. There were POSIX subsystems for WinNT 3-5.1. Though, things changed and got worse.
- um_ya 11y agoBetter yet, why don't they just make a tool that fixes the windows problem, as a standalone patch. No need to change NPM, just need to fix the problem on Windows... The whole flat folder thing looks really bad.
- gelasio 11y agoIt sounds like you're getting annoyed that the devs of certain projects like Node.js have chosen to support an OS that you don't use without asking you about it first... If you're in control of the project - you have every right to ignore Windows users. If you're not in control, then you really have zero right to complain. Zero. Devs certainly aren't forced to "pay the Windows tax" in any way. Plenty of OSS projects completely ignore Windows. I'm sure you're aware though that Windows has the majority market share of desktop operating systems. If your software is interesting to the vast, vast population of Windows users, you might get some requests to support Windows. And since it's not an outlier in the market, other desktop OSes should obviously strive to be more like Windows ;)
- city41 11y agoWindows is very much an outlier in the world of software development. Unless you're specifically using Microsoft tech (ie .NET and all that), then you're in a world dominated by Linux and OSX. I'm annoyed that Microsoft insists on going forward with a proprietary OS that does not conform to standards that pretty much every single other OS does. Sure, platform specific issues do happen for other OSes, but they are rare. They are extremely common for Windows. For sure, I have no obligation to fix Windows specific issues in my projects. All of my projects are done in my spare time and given away for free. But Windows specific issues are still annoying and disappointing. Another aspect of this that is annoying is that major, mission critical open source projects have to dedicate so many resources to Windows. For example this bug in ClojureScript: https://github.com/clojure/clojurescript/commit/80d46bdb7969c6a914324f1b9c81f8246d01f0a7 https://github.com/clojure/clojurescript/commit/80d46bdb7969... Imagine if the Clojure team didn't have to worry about file separator differences, and the multitude of other Windows annoyances. How much extra energy, time and resources would they gain? I suspect a lot. EDIT: And I would also say I have every right to complain about Microsoft. Ignoring standards and doing their own thing for their own financial gain at everyone else's expense is definitely worthy of complaint. Also see Internet Explorer.
- moron4hire 11y ago>> Windows is very much an outlier in the world of software development. Unless you're specifically using Microsoft tech (ie .NET and all that) I think you need to get out more. Or at least go to meetups that don't only meet at coffee shops. You're cutting out a huge chunk there, and there are still huge chunks that use Windows to do Java, C++, etc. I mean, game development is a multi billion dollar industry that is almost exclusively Windows-but-not-.NET, outside of consoles, and even on consoles it's still Windows in a large number of cases. >> I'm annoyed that Microsoft insists on going forward with a proprietary OS that does not conform to standards that pretty much every single other OS does. This is different from Apple with OSX and iOS how? This is different from Google with Android how? This is different from Canonical with Unity how? This is different than RedHat and Debian and pretty much every other major Linux distro with systemd how? And for crying out loud, file separator issues? That's not even a hard one.
- Klathmon 11y agoI love the path that some take which is to maintain both a "pure" JS-only solution and a "native" solution, and have the native one be optionally installed. That way it works as best it can automatically. Now I know that can't always be the case as it adds a lot of work to the development and maintenance, but it is a really nice thing when i see it.
- davexunit 11y ago>The better goal would be that the entire state of the node modules directory is a pure function of the contents of the package.json file plus the platform details Yes, this would have been much better than this new, also-broken approach. Functional package managers like the one I work on, GNU Guix, have this problem solved and solved well. Language-specific package manager developers would do well to either implement similar systems in their projects, or realize the inherent problems with limiting the scope of dependency management to a single language rather than the full dependency graph and switch to using functional package managers.
- munificent 11y ago> Language-specific package manager developers would do well to either implement similar systems in their projects This is what we did with Dart's package manager[1]. It works like Bundler where it finds a single set of package versions that satisfy all of the constraints and there are no duplicate packages. It's a challenge because constraint solving is NP-complete, but it works very well in practice. [1]: https://pub.dartlang.org/ https://pub.dartlang.org/ > realize the inherent problems with limiting the scope of dependency management to a single language rather than the full dependency graph and switch to using functional package managers. The problem then is now your language has a dependency on an outside package manager, one which is often OS specific. Most modern languages need to support a variety of OSes.
- davexunit 11y ago>This is what we did with Dart's package manager[1]. It works like Bundler where it finds a single set of package versions that satisfy all of the constraints and there are no duplicate packages. Constraint solving is the status quo, and I am strongly opposed to it. Rather than saying "I want foo >= 1.0" in a package manager, you should have the build system test the environment for the features you need. This is what the Autotools and other build systems that have been around for decades do. This way, the project isn't tied to a single package manager and, in the case of free software, greatly eases the burden on package maintainers that want to add your software to their distribution. Functional package management doesn't need constraint solving because every package in the system precisely describes itself: exactly which dependencies are needed for build-time and runtime (which precisely describe themselves, recursively, all the way down to libc), precisely which source code (tarball/directory/whatever + checksum), and the exact build script that turns that source code into a (hopefully bit-reproducible) binary. There's no constraint solving to determine what the dependency tree is, it's already been encoded in the package objects themselves. This is what any robust package manager should enable, but the only two that I know of that do this are Nix and GNU Guix. >The problem then is now your language has a dependency on an outside package manager, one which is often OS specific. Most modern languages need to support a variety of OSes. A programming language should not depend on any specific package manager. This is what build systems are for! More and more language communities conflate the two, but this is a huge mistake that we are paying for by making our software nearly impossible to reproduce from source code. [0] Everyone just uses pre-built binaries full of circular dependencies and reproducibility issues because things are so tangled that no one actually knows how to build anything from source anymore. I think the value of language-specific package managers lies in providing an easy way to fetch pure (no native code) modules to faciliate simple code sharing and helping newcomers get bootstrapped quickly. However, for serious software development and deployment they are terrible and we desperately need better tools. I think functional package managers are the tools we need, as they have greatly simplified software building and deployment for me whilst also greatly increasing the reliability of the systems that use them thanks to transactional upgrades and rollbacks. [0] http://www.vitavonni.de/blog/201503/2015031201-the-sad-state-of-sysadmin-in-the-age-of-containers.html http://www.vitavonni.de/blog/201503/2015031201-the-sad-state...
- ploxiln 11y agoThis "penalty" you propose would be a cosmetic thing, it wouldn't really make a difference. Many devs would choose to use the module as a dependency of their modules. The real solution to the sub-dependency-insanity of npm is ... use fewer dependencies. Don't use dependencies that have sub-dependencies. Write more code to directly solve your problem. This is a viewpoint you won't often hear, because it comes from someone who doesn't write js or use node ... I've just had to fix the deployment of some node applications written by frontend devs a few times. I've seen npm trees with over 1000 modules to do some very trivial stuff. Node actually seems nice, and it has all the basics you need built in. I typically use C or Python, which pretty much never use nested dependencies, and have never felt the desire to install/update/manage/debug exponentially more dependencies.
- feross 11y agoI wouldn't recommend using fewer dependencies, but smaller ones. One of the best features of npm is that it solves the "dependency hell" problem. You can depend on modules freely and not worry about making it harder for users of your module to install it. The issue comes when people create these huge "do-everything" modules. This is worth a read: https://github.com/sindresorhus/ama/issues/10#issuecomment-117766328 https://github.com/sindresorhus/ama/issues/10#issuecomment-1...
- Touche 11y agoThat just creates a situation where you depend on an insane number of dependencies and your npm install takes 10 minutes.
- tracker1 11y agowhere this really starts to bite in my mind, is when you wind up seeing multiple modules in different dependencies that do the same thing. Lodash, Underscore, Ramda and other come to mind in particular here... It's really easy to wind up with at least two of those three. Part of why I wouldn't mind seeing a few "winners" even if slightly larger win out. We're at a point where we get to tear jQuery out of projects, but it's easy enough to wind up with bundles that are larger than if we'd just used jQuery or Zepto, or whatever.
- tehnorm 11y agoAimed higher is correct. I've always wondered why no one looked at the way git does this under the covers and applied it to npm. A directed graph that points to (with a sym link or similar mechanism) the correct package version on the file system. That way you have the best of both worlds - dependencies don't collide and a single copy of a package version on the file system in node_modules.
- tracker1 11y agoThe biggest bug is that Windows Explorer gui has troubles with long file paths.
- DrJokepu 11y agoMeh. Anyone who ever worked with the v8 C++ interop API knows that it's a rather unpleasant experience (unlike e.g. Python's C interop API, which is very nice). Nobody uses it out of pure laziness, but because They Have A Very Good Reason To Do So™.