23 ms·

Node.js packages don't deserve trust

• mikece 5y ago
  "The fundamental problem with npm is that any package you install has full access to do whatever it wants on your computer." Is it uncommon to use dedicated VMs for development for this very reason?
  • saurik 5y ago
    > Is it uncommon to use dedicated VMs for development for this very reason? ... or on the production deployment's computer (which presumably is also "your" computer, and has a similar set of problems). (... or, if you go there, in production inside a VM, but in the same context and with access to everything from and all the memory and capabilities as the rest of the code you wrote, such as access to networking and the database or arbitrary CPU utilization.)
  • XorNot 5y ago
    It's hard to make it practical without linking a whole lot of your local environment in. Remember it's death by a thousand cuts - every time you need some new thing, you just add it without considering the consequences too much. Probably lots of people doing this with their whole home directory linked in read/write. Recently I got a little concerned about this and made myself a basic safety harness with the bubblewrap[1] tool: rather then going all out, I just lock the mount namespace to readonly for everything except the directory I execute it in. Which is at least some protection against system mods or wide-spread home directory destruction. What's a lot more of a problem is trying to protect truly vital files - i.e. SSH keys and the like - which are also things you're likely to have bound into your VM anyway. selinux is a much better solution there (but so hard to administer as to be almost useless, though I do really like Fedora's default scopes and have used them successfully). [1] https://github.com/containers/bubblewrap https://github.com/containers/bubblewrap
  • kklisura 5y ago
    Are we bringing back mainframes, yet?
  • skybrian 5y ago
    I haven't actually done development within a VM. How practical is it? What's a good way to get started? (In this case, on a Mac.)
    • mikece 5y ago
      When I was still a consultant I spun up new VMs for each client engagement -- both macOS and Windows -- using Parallels. When the engagement was over I would dump that to an external HDD. It meant over-spending on my MacBook Pro to have the SSD and RAM space to support multiple VMs but there was never any question of client work corrupting my system (or vice versa) or needing to wipe and reload my machine after a client engagement.
    • nukemaster 5y ago
      You just do everything over ssh. It's one of the reasons everyone still like CLI/VT tools so much.
    • mhoad 5y ago
      Codespaces is great in my experience. VSCode devcontainers is another very similar approach that will keep it on your machine but at least containerize the development environment raising the bar fairly significantly for a rogue package to do serious damage on your machine.
    • deleted 5y ago
      [deleted]
  • mschuster91 5y ago
    These days, for performance reasons most of the old "development VMs" (=Vagrant or worse) got replaced by Docker containers - orders of magnitude less effort. And in any case a dedicated VM is not going to protect you against attacks on your network, unless you go the full route of using a VPN to provide internet connectivity to the VM, and let's be honest almost no developer is going to do that simply because how much effort and maintenance it requires.
  • progx 5y ago
    Linux solves a part of the problem, it has user accounts with restricted access. Create a dev account and run node within that account. This solves the full access problem, but not possible backdoors or leaks of your app.
    • bogwog 5y ago
      And an even more convenient and more safe option is to use rootless containers via podman or [toolbx](https://containertoolbx.org/ https://containertoolbx.org/)
• z3t4 5y ago
  There are only two solutions: 1) Trust someone 2) Carefully read through their changes node_modules should be banned from .gitignore npm should stop moving folders around. If I have placed a module in node_modules/foo/bar it should stay there! Not get moved to node_modules/foo@1.0.0 etc
  • zkldi 5y ago
    You aren't meant to put your own files manually inside node_modules. What are you trying to achieve with that?
    • silon42 5y ago
      Have it in git and be part of code review.
      • nishs 5y ago
        If the code is managed manually and not by package.json, I would consider putting such code in a "vendor" or "third_party" directory instead of putting into "node_modules".
        • z3t4 5y ago
          You use "npm install" to place modules in node_modules, and "npm update" to conveniently update them. But you have to either 1) trust the maintainers of the modules or 2) review the code changes.
• lucideer 5y ago
  TL;DR: There's nothing fundamentally less worthy of trust about node's supply chain than any other popular mainstream language ecosystem. They've all got some badness. --- Here's a trope I'm tired of: Take X general problem that affects a wide range of systems, attribute it to one narrow system Y. Usually because that system Y's general accessibility and success leads to a higher number of high profile incidents related to problem X. One of the practical outcomes of this trope is people trying to solve this problem in narrow, ecosystem-specific, non-portable ways.
  • hinkley 5y ago
    > They've all got some badness. They're all based on the same design, as far as I can see. Is there another architecture we could be using?
  • mikece 5y ago
    Is the trope inaccurate in this case? If so, how? What would a path toward solution look like?
    • lucideer 5y ago
      The post has two things to communicate: 1. The Node supply chain has serious problems when it comes to security: this is true 2. The strong implication (throughout the article but most prominently in the title) that these problems are either unique to Node or at least worse in Node than elsewhere. This is definitely not true. --- The solutions cited in the article are good, and definitely benefit the Node ecosystem: --ignore-scripts is a well known option that would be great on-by-default but tends to be enforced in large corp CICD currently. The capabilities model is not too different to what Deno do (this article's idea is somewhat more fine grained but much more complex which could hamper adoption or lead to misuse). Overall both would be of somewhat limited, but still very worthwhile benefit. The hard problem imo is solving more generally for supply chain trust. We implemented --ignore-scripts enforcement last year and have only really see it have a relatively small impact, despite it being relatively new as an idea in the past few years (which motivates more exploits due to its novelty). In reality, most supply chain attacks are either targeting prod envs, or integrated build dependencies, rather than piggybacking CICD repo install steps: the later attack gives you the same env access and has no package-manager-specific mitigation, so it needs a more general solution. Capabilities also have limited use outside install scripts because limiting them also limits the "benign" functionality of your normal application code.
• egeozcan 5y ago
  Why put a sensational headline instead of something that describes what the post is really about? Something like "A capabilities based system for node.js packages", perhaps?
  • geniium 5y ago
    Click bait is the reason
    • josephg 5y ago
      Author here. I totally hear the criticism. I had "Capabilities for nodejs packages" or something like it as a draft title for the post. But its a boring title. And I believe this article would have had far less reach with a name like that. I hear you. I don't believe the answer is to just sensationalize everything to 110%. But I personally absolutely can't stand boring writing. Like, holding my attention on anything which isn't at least a bit human and engaging feels like torture. My job is 70% researcher, and one of the worst parts of my job is reading academic papers - I just can't hold my attention on a boring paper for more than a few sentences. My old supervisor used to tease me about using 2 weeks in the lab to save 5 minutes in the library. I think he's more right than he knows. So, I really don't have a good answer. What feels like sensationalist clickbait for you might seem like a catchy title to me, with a promise of engaging writing for someone else. And what sounds like an accurate title for someone else might turn me off completely because of its blandness. I suspect there's no middleground here where everyone is happy, and at the end of the day when I write its up to me and my judgement. And you, the audience to complain more loudly if you feel like like I'm taking it too far. I need that, because only have my own judgement for calibration. But thats pretty unsatisfying! Veritasium had a great video talking about this for his youtube thumbnails. I found it quite thought provoking: https://www.youtube.com/watch?v=S2xHZPH5Sng https://www.youtube.com/watch?v=S2xHZPH5Sng
• donatj 5y ago
  I think Go modules version resolution opting for lowest common release rather than the standard highest is a reasonable and sane option though not a total solution. It prevents users of your library from pulling unvetted versions of your dependencies just by pulling your library alone.
  • jaitsu 5y ago
    https://go.dev/blog/supply-chain https://go.dev/blog/supply-chain is a very good write-up on this topic too.
• bspammer 5y ago
  This token system requires changing code, why not extend package.json with a declared list of permissions instead?
  • josephg 5y ago
    Author here. That could well be a better design - especially if we need to explicitly "bless" packages anyway. One tricky part about that approach is it generates some weird semver problems. Lets say package A uses package B to interact with the filesystem. Package B has some problems, so the author of package A replaces B with B2 (a fork of B). From a semver perspective, this is totally fine because the exposed API of package A hasn't changed. And this is also true with the capabilities system I explained. But how would we do it in package.json? If the root package needed to explicitly bless B2 instead of B, that means package A must have broken semver compatibility. Maybe each package expresses the permissions its direct dependencies have, and it sort of ripples out getting more specific in the dependency tree? I think its a good idea, and there probably is a solution here somewhere. But I'm not quite seeing it. Want to write up a sketch of how you imagine that working?
• deleted 5y ago
  [deleted]
• jameshart 5y ago
  This is a noble proposal but it requires far too many systems to collaborate on adopting it. A capabilities system based on optional package metadata seems easier to introduce (think about how typings have been able to progressively layer on top of existing packages)
• macspoofing 5y ago
  What's different between the NPM ecosystem, and, say, java/maven? What is the latter doing that the former isn't?
  • rootlocus 5y ago
    Most packages you'd pull from maven are developed by large companies or foundations like apache. The dependencies you're pulling are simple jar files that get loaded at runtime and don't execute anything at build or install time.
  • moffkalast 5y ago
    Throw apt/snap/pacman/whatever into the mix and the answer is still "nothing". People act like package managers are somehow the end all but they're no more secure than going to a random official site and downloading some shit, it's just streamlined the process somewhat. In fact the latter is probably more secure, since the more the packages depend between each other the worse it gets. One random dependency can be hijacked and will be autoinstalled everywhere. Or someone can delete it and break half the internet as we've seen time and time again.
    • capableweb 5y ago
      You're kind of right, but there is a different between the default repositories used by apt (Debian & Ubuntu) and pacman (Arch) and things like npm, in that they are indeed reviewed and can't disappear overnight. You have some guarantee that it won't disappear overnight, because of the organizations behind them. With npm, anyone can publish/unpublish without any sort of review, while packages in the default repositories are reviewed by others.
  • michaelt 5y ago
    The short answer is nothing - it would be very easy to introduce a malicious Maven package. The slightly longer answer is maven artefacts are immutable; the default behaviour is to pin precise versions; and norms among java programmers don't favour using libraries for one-liners like left-pad - meaning there are fewer people in a position to launch a supply chain attack. The dam still has cracks in it, but there are fewer cracks and some have sticking plasters over them.
• mark_and_sweep 5y ago
  I'm surprised the author doesn't mention Node's Policies: https://nodejs.org/dist/latest/docs/api/policy.html https://nodejs.org/dist/latest/docs/api/policy.html "Policies are a security feature intended to allow guarantees about what code Node.js is able to load."
  • josephg 5y ago
    Author here! I didn't even know that existed. Thanks for the link! That looks like it has the same problem as Deno's solution, in that its too coarse for my taste. I want to explicitly give permission to a library, not to the process as a whole. (Since I don't want some errant library deep in my dependency tree to nuke my production databases.) I love the definitions of scope though - that looks like exactly the sort of thing that I want here.
    • zozbot234 5y ago
      > I want to explicitly give permission to a library, not to the process as a whole. (Since I don't want some errant library deep in my dependency tree to nuke my production databases.) Doesn't Java have a SecurityManager feature that can do this? Perhaps we need a JS equivalent.
      • jameshart 5y ago
        The trouble with security manager is it’s only as good as it’s widespread support. If it worked, and was widely used, for example, nobody would have had to worry about the possibility of their logging library downloading code from an LDAP server and executing it.
      • gorjusborg 5y ago
        Also, the security manager is getting removed: https://openjdk.java.net/jeps/411 https://openjdk.java.net/jeps/411
      • vbezhenar 5y ago
        SecurityManager was a bad idea. It might be useful to prevent accidental bound trespassing. But it turned out too flaky to serve as a security foundation. Basically nobody uses it for security.
• vlunkr 5y ago
  Pasting a comment I made about this a while ago: The reason this keeps happening with NPM is because of absurd number of dependencies in the average node app. I have a tiny app I've been playing with using create-react-app. There are over 800 directories in node_modules. That absolutely dwarfs the number of any other language I've used. Even in a medium sized rails app, you likely have some awareness of what every dependency is. It's just impossible with npm. This makes it easier for someone to inject their package into the ecosystem whether it's actually very useful of not (like the colors package). One thought I've had to "reboot" the npm culture is to somehow curate packages that are proven to have minimal and safe dependencies, probably through manual review. Maybe it could be recursive, so that safe projects only rely on other safe projects.
  • zozbot234 5y ago
    Package curation exists in other languages, too. C++ has Boost, and Haskell has its Haskell Platform. It helps avoid the pitfalls of languages with large standard libraries (where stability guarantees make "batteries included" turn into obsolete and bitrotting "dead batteries").
    • regularfry 5y ago
      This is an idea that every ecosystem eventually realises it needs. Once you've got enough versions of enough libraries that A and B both need C but at different versions, you start to need curation, although that need might not become pressing enough to do anything about for a while. But once you've got curation, cryptographically trusting that curation becomes viable in a way that cryptographic trust of the original packages often isn't. Putting a layer of "distributions" over language ecosystems, in the same way that "distributions" solve the problem of putting enough mutually-compatible library versions together to get Linux to work, is, I think, inevitable.
      • ljm 5y ago
        Another aspect I think is, how much open source dependency do you actually need for your project. You can probably get by with fewer packages than you expect, which also makes it more feasible to run a curated package system. For example, you can't submit any old Haskell library you put together to Hackage, or you couldn't at the last time I checked. It had to meet certain minimum standards to be considered. I honestly don't know where you would start in terms of curating NPM, precisely because of the dependency on dependencies. You'd end up curating half of the ecosystem.
• porsager 5y ago
  This won't prevent packages you're already giving capabilities to - to later do something evil in a small patch update. So it's not enough - you should also not rely on "automatic security updates" through some sort of semver trust. Lock your dependencies up completely, choose packages with a small dependency tree or zero dependencies, and at the same time use the newly launched https://socket.dev https://socket.dev to know what packages do. Also reading through the source of your dependencies should be on the list.
  • josephg 5y ago
    > This won't prevent packages you're already giving capabilities to - to later do something evil in a small patch update. True - its not perfect! But the principle of least privilege should help limit the blast radius. How many packages in npm dependency tree need access to the filesystem? Or to the network? I bet its a vanishingly small percentage of the packages in the average nodejs project. Being vulnerable to malicious code in 3 hand selected packages is much, much better than being vulnerable to malicious code in any of the packages in your dependency tree. And even then, we can be very specific about what those packages have direct access to. Right now the situation is "every package can read and write to any file on my computer". The fine grained permissions I'm proposing in this post would let us say "only package X can access the filesystem at all, and when it does it only has access to this subdirectory". Much better!
  • feross 5y ago
    Thanks for sharing Socket.dev! Totally agree that a key part of any supply chain security strategy must be understanding what packages actually do when they run. For example, see the package `angular-calendar` which is a calendar/date picker web component. When you look it up on Socket.dev [1], you'll see that it actually uses: - Install scripts - Telemetry to track you - Network access - Shell access - Environment variable access - File system access Which is waaay more capabilities than you'd expect. All of these capabilities turn out to be caused by a single dependency which implements telemetry to track the package usage a la Google Analytics. [1]: https://socket.dev/npm/package/angular-calendar/issues/0.29.0?tab=dependencies https://socket.dev/npm/package/angular-calendar/issues/0.29....
• jFriedensreich 5y ago
  there is already a project that does most of this: deno. there are of course other motivations that differentiate it from nodejs but i think this should be part of a new runtime that was build with this in mind, deno is the way forward. Of course putting your projects in appropriate jails/zones/containers/vms is also an option, but that has always been true
• roryrjb 5y ago
  I was exploring the actual implementation[0] of a capabilities feature in Nodejs and was utilising seccomp (via libseccomp) on Linux at least to achieve a greater degree of security than might otherwise be possible by remaining in userland code. The idea is that you'd write your code, import whatever you like and define your capabilities upfront at initialisation. The problem is there's quite a big disconnect between what you are doing in JavaScript and what's happening with system calls in v8, libuv and the other native parts that it's difficult to predict what you need to block and what's actually going to happen. So I don't think my approach is really viable in a general sense, although capabilities in general I think would improve the situation if the wider community were to adopt the approach. [0]. https://github.com/roryrjb/node-seccomp https://github.com/roryrjb/node-seccomp
  • jgerrish 5y ago
    Cool. OS level code, realistic discussion of issues and approaches. Thanks for the contribution.
• deleted 5y ago
  [deleted]
• mc4ndr3 5y ago
  Got tired of constantly being the one to patch upstream problems, ended up deprecating the majority of my (alt)JS problems.
• ecmascript 5y ago
  I don't get why this happens to node all the time. They have a very extensive code of conduct that forbids this /s
• filleokus 5y ago
  Could a path forward also be to unify the most depended upon small packages into one large dependency, managed by some trustworthy entity? I guess some plug-in to npm could handle the resolution-mapping between the (9 line) strip-ansi package and the node-standard-library package? Of course this don’t solve all problem, but if a create-react style app could lower its number of dependencies by 80% or something, it would be easier to keep track of the remaining. Because even if you have some kind of capability system, you are still pretty vulnerable to miss behaving packages. Even if the scope of badness is dramatically lower, chaos would ensue in many build pipelines if some of these “core” package just started throwing exceptions / returning empty objects
  • a9h74j 5y ago
    ActiveState does something comparable for Python, Perl and TCL. https://www.activestate.com/ https://www.activestate.com/
  • hulitu 5y ago
    The tendency is to make for every function a library. Just look at lib directory in the Slackware install tree. The functionality (of the distribution) has not changed too much but the number of libraries needed is astonishing.
• jcmontx 5y ago
  I remember reading articles like this since 2016 or 2017. How come nothing has changed in so many years? Is this issue not important enough? Or is the solution just to expensive/unpractical to implement?
  • marcus_cemes 5y ago
    I think the machine has just gotten too big, it would require a huge effort, even the original creator or Node.js, Ryan Dahl, is strugling create sufficient displacement with Deno. Innovation over safety? Like cheap products I guess, either you join them or get driven out of business, leaving only the cheap products anyway. Also, if you want to spend a lot of time, effort or money to vastly improve something, would you put it into JS or would you perhaps focus on another language that has better foundations and hope it catches on, where there is the potential to outperform (in quality and consistency) the JS ecosystem? Perhaps those with more sense move to other ecosystems, such as C#, Go, Java, Elixir? I say this as a long-time user of JS who has recently enjoyed his foray into Elixir.
• didip 5y ago
  In the beginning of node.js life, NPM the company had a guerilla marketing that says: The smaller the package, the better for reusability. That stained the node.js ecosystem forever, until now. This is why I am excited about Deno, Ryan wants to break away from this culture.
  • silon42 5y ago
    IMO, npm is correct/better. The solution is to 'vendor' everything and have package updates as part of normal code review.
    • throw_m239339 5y ago
      > IMO, npm is correct/better. NPM was never correct. NPM as a business cared about growth. The more packages on NPM servers the more valuable their company, it's as simple as that.
  • elondaits 5y ago
    I think that’s throwing the baby out with the bathwater. Wouldn’t it be logical that when people use snippets from Stack Overflow or classical algorithms from Wikipedia the pasted code was actually treated as a dependency so you can get warnings and updates in case errors or security issues are found? It also helps for proper licensing and attribution. The problem is that code in npm should allow for trust. Either based on signatures, or code reviews by trusted parties, or something like that. Code signed by a long time trusted developer that’s been published for a month should not be treated the same as a 3 minute old commit to a repo by a mysterious developer. These verifications should be automated and npm could give a final ranking.
    • throw_m239339 5y ago
      > I think that’s throwing the baby out with the bathwater. Wouldn’t it be logical that when people use snippets from Stack Overflow or classical algorithms from Wikipedia the pasted code was actually treated as a dependency so you can get warnings and updates in case errors or security issues are found? It also helps for proper licensing and attribution. Why are you assuming devs don't just both use random snippets from Stack Overflow and also download packages with 900 transitive dependencies from NPM at the same time? It's not one or the other.
• mike_hearn 5y ago
  Hmm, where to begin? This is an old idea. It has all been tried before in the JVM world and yet support for it is now being removed, which is in my view a pity given that Now Is The Time. But the problems encountered trying to make it work well were real and would need to be understood by anyone trying the same in the JS world. Understand that Java had it relatively easy. Java was designed with a sandbox as part of the design from day one, the venerable SecurityManager. The language has carefully controlled dynamism and is relatively easy to statically and dynamically analyze, at least compared to JavaScript. The libraries were designed more or less with this in mind, and so on. So what went wrong? Firstly, the model whereby you start with a powerful "root" capability and then shave bits off doesn't have particularly good developer usability. It requires you to manually thread these little capabilities through the call stack and heap, which is a nightmare refactoring job even in a language like Java let alone something with sketchy refactoring tooling like JavaScript. Lots of APIs become awkward or impossible, something as basic as: var lines = readFile("library-data.txt"); is now impossible because there's no capability there, yet, developers do expect to be able to write such code. Instead it would have look like this: function readFile(appDataPath) { var url = appDataPath.resolve("library-data.txt"); var lines = appDataPath.readLines(); } readFile(rootFileSystem.resolve("/app/data")); Can you do it? Yes. Does it make code that was once concise and obvious verbose and non-obvious? Also yes. Consider also the pain that occurs when you need a module that has higher privileges than the code calling it (e.g. a graphics library that needs to load native code, but you don't want to let the sandboxed code do that). In the pure caps model you end up needing a master process that "tunnels" powerful caps through to the lower layers of the system, breaking abstractions all over the place. Secondly, this model means you can never add new permissions, change the permissions model or have different approaches because refining permissions == refactoring all your code, globally, which isn't feasible. Thirdly, this model imposes cap management costs on everyone even if they don't care about security because they know the code is trustworthy e.g. because their colleagues wrote it, it came from a trustworthy vendor, or because it'll run in a process sandbox. Even if you know the code is good it doesn't matter, you still have to supply it with lots of capabilities, you still have to implement callbacks to give it the capabilities it needs on demand and so on. These problems caused Java to adopt a mixed capability/ambient permissions model. In the SecurityManager approach you assigned permissions based on where code came from and stack walks were used to intersect all the sources on the stack. Java also allowed libraries to bundle data files within them, and granted libraries read access to their resources by default. That solved the above problems but introduced new ones, in particular, it lowered performance due to the stack walking, plus now library developers had to document what permissions they needed and actually test the code in a sandboxed context. They never did this. Also the approach was beaten from time to time by people finding clever ways to construct pseudo-interpreters out of highly dynamic code, such that malicious code could get run without the bad guy being on the stack at all. Fourthly, it's dependent on everyone playing defense all the time. If your object might get passed in to malicious code, then it has to be designed with that in mind. A classic mistake: class Foo { private ArrayList<String> commands; void addCommand(String command) { commands.add(command); } List<String> getCommands() { return commands; } } The author's intent was to make an object in which you can read the list of commands but not write them. But, they're returning the collection directly instead of using an immutable wrapper. Fine in normal code, but oops, in sandboxed code now you have a CVE. Bugs like this are non obvious and the tooling needed to find them isn't straightforward. These bugs are a drain on development. Fifthly, Spectre attacks mean that a library that can get data to an attacker via any route can exfiltrate data from anywhere in the process. You may not care about this, and for many libraries there may be no plausible way they can exfiltrate data. But it's another sharp edge. Finally, it all depends on the ecosystem having minimal native code dependencies. The moment you have native code in the mix, you can't do this kind of sandboxing at all. Now. All these are challenges but they don't mean it's impossible. Sandboxing of libraries is clearly and obviously where we have to go as an industry. The Java approach didn't fail only due to the fundamental difficulties outlined above - the SecurityManager was poorly documented and not well tuned for the malicious libraries use case, because it was really meant for applets. After the industry gave the Java team so much shit over that, they just sort of gave up on the whole technology rather than continuing to iterate on it. It may be that a team with fresh eyes and fresh enthusiasm can figure out solutions for the above issues and make in-process sandboxing really happen. I wish them the best, but anyone who wants to work on that should start by spending time understanding the SecurityManager architecture and how it ended up the way it did. https://dl.acm.org/doi/pdf/10.1145/2030256.2034639 https://dl.acm.org/doi/pdf/10.1145/2030256.2034639
• toastercat 5y ago
  The impression I get from the companies I've worked at is to not trust packages that are mostly maintained by one person. Ironically, those packages are usually the ones that don't have outrageously large dependency trees and can usually be audited by a developer on a weekend, and the more "trustworthy" larger packages with hundreds of maintainers typically have monstrous dependency trees. Not saying 1-man tools are inherently better, but in my experience, these tools seems to have a tighter focus and less chances of scope/dependency creep.
  • btmcnellis 5y ago
    The big problem with one-person packages isn't so much security as it is support. I have been burned more than once by old applications where key features rely on random packages with one maintainer who disappeared years ago. At least with a group, you have options to keep things moving without having to fork the library yourself. (Of course the root cause here is arguably too much reliance on third-party dependencies, but searchable dropdowns are _such_ a pain to make on your own, and it's so tempting...) The Sangria GraphQL library in Scala ran into a version of this. The libraries were primarily maintained by one person, who wrote the vast majority of the code and was the only person with write privileges in the main repos. Sadly, he passed away unexpectedly, and it took months (maybe a year or so) before his colleagues and other contributors were able to get access to the GitHub org.
    • ratww 5y ago
      Well, for what is worth, we have a lot of dependencies maintained by Microsoft of all companies, with lots of production-breaking bugs and they're not too interested in fixing or letting us fix. Even getting fully-functional PRs (with good test coverage and community support) looked at takes a lot of work and time, let alone getting fixes after reporting issues. One of those packages is a JS package that is hosted by them, so we can't even fork it and host ourselves. On the other hand, with simple packages that get abandoned, we just fork, publish ourselves with another name or namespaced, and it's solved.
      • 5y ago
• havkom 5y ago
  I would suggest that one tries to limit dependancies. The entire node ecosystem contraviene this so maybe people should not use it.
• andix 5y ago
  We need one or a few well maintained standard library for Node and/or Browser-JS. Those must not depend on any other libraries. And then packages can reduce their dependencies vastly, by just referencing one standard library, that provides a lot of features.
  • tempest_ 5y ago
    JS's small standard lib is the cause of some of these issues. When I look at one of the JS projects I have worked on there is a non trivial amount of 'is-*' packages whose only job is to identify the type of the object.
    • andix 5y ago
      Exactly. Somebody would need to put together a standard lib, maybe even by repacking some already de-facto standard libraries. But only one library per category (one „is“ library, one date library, …). It’s a very opinionated task, but it could really help.
    • shagie 5y ago
      is-odd, is-even... The radical code reuse where people don't ever want to write a one liner that exists in some other package is at issue too.
      • andix 5y ago
        There is even is-is-odd, lol
    • dolni 5y ago
      The small stdlib is part of it. The fact that identifying an object's type is so difficult is another part of it. It's also more foundational to JavaScript.
• cabirum 5y ago
  Can we identify nodejs projects, which - a) are trivial to implement and, b) are most depended upon? I'd love to contribute to a coordinated community effort to minimize the number of npm dependencies.
  • flanbiscuit 5y ago
    This is not the exact thing you're looking for but it's a place to start: https://gist.github.com/anvaka/8e8fa57c7ee1350e3491 https://gist.github.com/anvaka/8e8fa57c7ee1350e3491 That code in that gist is generated by this: https://github.com/anvaka/npmrank https://github.com/anvaka/npmrank Just found that from a quick google search. It would be nice if it had a way to search by max package size.
  • feross 5y ago
    We maintain an automatically updated list of "trivial packages" (defined as less than 10 lines of code) here: https://socket.dev/npm/issue/trivialPackage https://socket.dev/npm/issue/trivialPackage Is this the type of list you were looking for? It's not a perfect metric and isn't meant to disparage any of the maintainers of these packages, but it's one factor in our Socket.dev risk analysis.
    • cabirum 5y ago
      Yes, exactly!
      • feross 5y ago
        Great!
• j0057 5y ago
  Capabilities, chroots and whatnot look like containers, but with extra steps.
• Kaze404 5y ago
  This article made me wonder about packages calling user code. Using express' example, what if I don't want to give it filesystem access but need it in one of my callbacks? Is the engine capable of distinguishing my (privileged) code from express' (unprivileged) code?
  • kilovoltaire 5y ago
    Your callback would have your capabilities in scope, so your user code would still be able to do whatever you want, even when called by an untrusted library. I.e. something like: import capabilities; import fs; import someLibrary; const root = capabilities.claimRootToken(); someLibrary.onEvent(() => { fs.writeSync(root, ...); });
    • Kaze404 5y ago
      Makes sense, thank you
• a9h74j 5y ago
  Is there a need for ... NPM: The Good Parts
  • acheron 5y ago
    "Would you like something to read?" "Do you have anything light?" "How about this leaflet, NPM: The Good Parts?" https://www.youtube.com/watch?v=IoACIIz33II https://www.youtube.com/watch?v=IoACIIz33II
• systemvoltage 5y ago
  This is similar to electronics supply chain. If you source from shady component distributors, you’re going to get bitten with something like a faulty capacitor with an annoying frequency in volume. The difference here is that a single vulnerability can take down your entire app or worse. It’s like sourcing components that can potentially do irreversible fire damage to your customers. This is why they rely on trust worthy distributors and manufacturers. Without payment incentive and with high expectations for free stuff, this is not possible to solve. Billion dollar companies building on people’s hobby projects. In any other industry, this would be unprofessional.
  • zozbot234 5y ago
    > This is similar to electronics supply chain. Not a new idea. In fact, "Software IC's" as a notion goes back to the late 1980s.
• cosmiccatnap 5y ago
  Sort of a cake and eat it too mentality if you ask me. If you're not willing to build your own JS glue then you should at least take time to audit the glue you are pulling in from some thankless developer who probably started this as a hobby project in college and now it is the backbone of 10 fortune 500 frontends. I'm sorry that they can then do whatever the hell they want with that package but this ecosystem exists because companies want free stuff but don't want to provide back into it.
• kerneloops 5y ago
• numbsafari 5y ago
  If, instead of loading library code directly into the same process and heap, package code were instead loaded and run in their own dedicated processes with their own dedicated memory and you communicated via message passing (a la actors) then this approach would really start to make sense. But most node packages today are designed to run directly in the same process and memory space as the code that is being imported. Also, any package manager that runs arbitrary code should not be trusted.
• Hackbraten 5y ago
  Vanilla JS is not something we should be afraid of. I wish we’d consider it more often in projects. The latest generations, ES2020 and newer, are pretty useful and pleasant to use. If you leverage their features, it goes a long way. Implementing the odd missing function yourself along the way is perfectly doable if you embrace TDD. My feeling is that people in the JS ecosystem tend to overestimate how much time they’d allegedly waste re-implementing stuff, and grossly underestimate the true cost of deeply-nested dependency trees.
• marcus_cemes 5y ago
  It makes me deeply sad to see these sort of interactions in open source [1]. > Hmm, I think it's a worthwhile fix. Where did you see malware here? > I think the author of this repo is free to decide what code he publishes. Say thanks to that it's for free An incredible amount of people have dedicated sweat and tears and foreheads (from banging against the desk in frustration) to open source across the entire stack, from the contributers to OSs such as Linux to those working their arses off to create better frameworks, languages and runtimes, that we can all benefit from and use with a reasonable expectation of security, respect and privacy. As a university student, I feel privileged to have been able to grow up in a world where so much work and knowledge is provided for free with no strings attached, regardless of demographic/location, I would not be where I am without it. A century ago this would not have been possible. To all of you who have tirelessly and selflessly worked on OSS for others, without expecting anything in return or imposing politics, ideologies, infringing on privacy, causing damage, collecting vast quantities of marketable personal information or monopolisation, I give you my heartfelt thanks for your efforts, you know who you are. You have created something that will have forever helped to improve our society and empower those that want to learn and create their own designs. From my own personal experience, I want to give a shout-out to the smaller projects of Rust, Svelte and Elixir. I think it's incredible that the work and ideas of (often) a single person (Rich Harris, José Valim) can grow into larger extremely welcoming and helpful communities with many more motivated contributors that are proud of being parts of those projets and put in an extraordinary effort to try and do things better than before. I'm sure there are plently of other worthy names I'm too young/ignorant to know. Love it or hate it, Node.js has been very empowering for a large number of people to learn and publish their own full-stack applications, the JavaScript ecosystem has improved enormously since its beginnings, but has a tendancy to change slowly due to its size, unless a disruptive technology comes along such as TypeScript. Websites are a great way to introduce people to the joy of programming with its visual feedback, you can make a small penguin move across the screen, then move on to play tic tac toe. Even as a younger developer, I admit that the days of FTP, no-build-step pages with a sprinkle of JQuery were easier to understand and actually safer for newcomers than introducing someone to a SPA stack (which can easily have thousands of transient dependencies) nowadays. [1]: https://github.com/Yaffle/EventSource/issues/202 https://github.com/Yaffle/EventSource/issues/202
• danShumway 5y ago
  Good article. This is what I was excited about the first time I heard about Deno before I eventually learned what Deno's sandboxing model actually was. I'm not sure this is the exact proposal I would want, but I do want something vaguely like scopes or capabilities in Node, and even if it wasn't perfect I think it would go a long way towards mitigating at least some of the current risk in the ecosystem. Also agreed that for all their use, it would have been better in the long run if install scripts had never existed. It's not just that they're a security vulnerability, they also get in the way of vendoring code, and can introduce additional non-JS dependencies and errors on other systems/platforms. Again, not to say that they don't have any use, I get why they're there. I just wonder if the benefits are worth the downsides.
• lucacasonato 5y ago
  Deno core team member and TC39 delegate here. We have pondered about capability based security for Deno in the past. Our conclusion has always been that this is not possible to do securely in JS without freezing all prototypes and objects by default. The reasoning for this is that you need to make sure the capability token does not ever leak. For example as a malicious user I could override `globalThis.fetch` to exfiltrate the capability token destined for `fetch` and use it myself later. One could also override `Map.prototype.set` / `Map.prototype.get` to exfiltrate a token every time it is added or removed from a `Map` (people will want to store tokens in a `Map`). One could also override `Array.prototype[Symbol.iterator]` to exfiltrate tokens stored in arrays if those arrays are destructored, spread, etc. There are many more cases like this, where one can exfiltrate tokens because of the very dynamic nature of JavaScript. It is unlikely that freezing all intrinsic prototypes and objects is even enough. People will find ways to exfiltrate tokens.
  • zozbot234 5y ago
    These seem like features that could work a bit better in the context of WASM modules and/or components. AIUI, WASM was designed with the expectation that support for capabilities would be required.
    • lucacasonato 5y ago
      Yup, for sure. I think capability based ShadowRealms are also totally doable. I'm sure we'll add those for Deno once we add support for ShadowRealms.
  • 3np 5y ago
    Doesn't SES address that? The only fundamental barrier right now seems to be performance, which could be addressed by runtime support. https://github.com/tc39/proposal-ses https://github.com/tc39/proposal-ses https://github.com/endojs/endo https://github.com/endojs/endo
    • lucacasonato 5y ago
      Yup, SES would address this. But SES also needs to bring with it a paradigm shift for JS: a) Folks would have to load each bit of code they want separate permissions for in a separate compartment. This won't be easy. b) Runtimes will need to provide an immutable global realm, which is not something that is the case right now. As I said in a different commeent, I think a lot of this can already be addressed by ShadowRealms. Deno will likely allow users to specify per ShadowRealm permissions, which is probably as granular as most people will want to get.
• grnmamba 5y ago
  The offending event-source-polyfill version is still on npmjs.com.
• etchalon 5y ago
  Seeing how there are dozens of package management toolchains out there, why is NPM so uniquely absolutely horrible? The only behavioral difference I see is that with most other package tools, installing a dependency pins the specific version of the dependency you installed. So why does NPM instead pin "any version at least this, or greater"? Feels like a decision that was made when the tooling was young to help improve reliability that has since created this nightmare. Is Yarn better about it?
  • gedy 5y ago
    > So why does NPM instead pin "any version at least this, or greater"? It doesn't, it follows semver and you can pin to point, minor, major, etc versions depending on your package.json
  • hinkley 5y ago
    There is a flag you can override to be less aggressive, and it's in our onboarding docs, but only because I added it after a conversation with the only other individual on our team who was using it. It's the classical situation of the framework having defaults that end up being actively hostile to developer success. These Primrose Path situations get under my skin like few things do.
    • etchalon 5y ago
      Wasn't aware of the flag. Thanks!
  • CraigJPerry 5y ago
    >> why is NPM so uniquely absolutely horrible? I don’t think it’s uniquely bad. For example, Python’s story is just as bad. The Java world is the best i know of but even relatively modern stacks like Rust are still making the same errors long-solved in the java world. For example, nothing stops me publishing a google-grpc crate today (first come first served on names and that name’s not taken and i don’t need to prove ownership of my domain like in the java world). C# and nuget is a blind spot for me so I don’t know. Go gets it right arguably even better than Java. Deno borrows the same decentralised idea. The whole situation is a stack of cards right now though. I’m genuinely surprised we haven’t seen more specific targeting of high value maintainers. Github is arguably ahead of the curve here a little bit (e.g. the ability to flag and act on unsigned commits).
• nilsbunger 5y ago
  I'm surprised we haven't had many crypto key thefts via npm supply chain exploits.
• andrewmcwatters 5y ago
  No, Go solves this problem and Deno explicitly copies its design. Just use Go. Seriously. The stupidest thing npm still does today is naively assume SemVer is an appropriate strategy for automatic updates. There is so such thing as an appropriate strategy for automatic updates for developers. For end users? Sure. Because the assumption for them is that things are stable. Developers need to check. Developers need to pin. If you’re working in the Node.js ecosystem the solution isn’t to throw the baby out with the bath water and use Deno, it’s to `—save-exact`. I’m sure as hell not throwing away all of the hours invested in now stable JavaScript software.
• andrewmcwatters 5y ago
  I made another post here, but in a separate note, some of the developers of the most used software in the Node.js ecosystem, as people, don’t deserve your trust either. For the past decade or more I’ve watched and personally interacted with the personalities of some of these developers and the last thing they seem to have on their mind is the stability of your software. They just do not care. Many of them are totally willing to throw away years of work that you also built years of work on for you to chase after their new toy. These are guys that built popular packages in their early to mid 20s. They weren’t thinking about software that lasts any meaningful duration of time.
• bjpbakker 5y ago
  A capabilities system like pledge could be a way to safer use _existing_ packages. However, I think that it's not a very nice way to continue. Every application will end up doing its own capability pledging, and mistakes will be made. A lot. Another approach could be to use an effect system like PureScript does. The main problem with Node.js packages is that any function you use can execute arbitrary code (such as wiping systems with an IP that is from the Russian region). Having an effect system in place the library author has no other means than to come forward with the side-effect, or code won't compile.
• cryptica 5y ago
  It doesn't make sense to build this 'capabilities' feature into the program itself. It feels a bit like taping your mouth shut in order to lose weight. It doesn't make sense for a program to not trust its own code any more than it makes sense for a person to not trust their own thoughts. There is no need to pollute your code like this. It should be implemented as an external tool which analyzes dependencies when executed on demand. A company could just run this tool as part of their CI pipeline before code is deployed or executed. It could be a default hook which runs automatically as part of npm install. It should not be part of the code itself. It's ugly and adds unnecessary overhead and complexity. It's possible that an external tool executed at compile-time would not be able to verify modules which come with C/C++ bindings, but I think it would be difficult to stop these anyway (even at runtime). C/C++ bindings will always be less secure because it's harder to understand what's going on if you don't have access to the code. C/C++ is too powerful; you can do some crazy stuff with buffer overflows which would be difficult to detect anyway even at runtime. The solution is to try to stick to modules which rely only on native Node.js functionality and not on custom C/C++ bindings.
• jollybean 5y ago
  Systems not designed for security are hard to secure after the fact.
• chubot 5y ago
  I think this needs to be solved at the OS level, not the language level. It's a problem for every language -- packages you download can write to any file or make network connections. It might be a little worse in NPM because of the culture, but the problem is pervasive. Personally I'm interested in the direction of lightweight containers that behave like executables, and that have composable dependencies. Developing in Docker-like containers can work but there are a bunch of downsides to be mitigated. (copy of lobste.rs comment)
• phendrenad2 5y ago
  Of course this is a problem for all package repositories. Ruby, Python, Docker, Node, etc. But Node seems especially bad. Why? It seems to me that Node packages are mostly the work of a single person, and lack a community around them. You know you can trust Nokogiri because there are many developers working on it, and they're visible and accessible on Twitter et al. But many Node modules are just one person's passion project and/or resume padder. It's too easy for someone to have their npm publish keys stolen and not notice, whereas if there were more people on the project someone would probably notice sooner.
  • zozbot234 5y ago
    Many C libraries are maintained by a single person, too. But C/C++ programming has no equivalent to the Node culture of making trivial releases, so every single update to a C package gets quite a bit of attention from its users.
• deleted 5y ago
  [deleted]
• feross 5y ago
  Founder of Socket (https://socket.dev https://socket.dev) here, a new tool built by npm maintainers to help solve JavaScript supply chain security. I totally agree with the idea that we should assume all open source packages may be malicious. Socket.dev uses "deep package inspection" to characterize the behavior of an open source package. By actually analyzing the package code, Socket can detect when packages use security-relevant platform capabilities, such as the network, filesystem, or shell. For instance, to detect if a package uses the network, Socket looks at whether fetch(), or Node's net, dgram, dns, http or https modules are used within the package or any of its dependencies. This entails running static analysis (and soon, dynamic analysis) on a package – and all of its dependencies – to look for specific risk markers. In this way, Socket can detect the tell-tale signs of a supply chain attack, including the introduction of install scripts, obfuscated code, high entropy strings, or usage of privileged APIs such as shell, filesystem, eval(), and environment variables. We are taking an entirely new approach to one of the hardest problems in security in a stagnant part of the industry that has historically been obsessed with just reporting on known vulnerabilities.
  • caseyross 5y ago
    This, in my opinion, is the right answer for the problem identified in the parent blogpost. Rather than trying to get every single package author to adopt some unified capability token scheme in their code, just statically analyze all dependencies from the outside and report the capabilities they actually use. It would be even better if something like this could be integrated directly into the package management tool itself, so that you could run `npm update` and get back "New dangerous API usage in package X version a.b.c: filesystem access. Type package name to acknowledge and upgrade."
    • feross 5y ago
      Thanks. Glad you like our approach. > It would be even better if something like this could be integrated directly into the package management tool itself We're planning to build this. However right now, the primary way to consume Socket.dev data is through our GitHub app (https://socket.dev/integrations https://socket.dev/integrations).
• jamesfisher 5y ago
  I almost didn't read this because the title sounds like a whingeing rant. But it turned out to be a carefully considered proposed solution.
• jmull 5y ago
  I think making the capabilities the responsibility of the app (i.e., having the app handle tokens and pass them around) won't work, for a couple reasons: (A) it's tricky for the app to protect itself. The executing app code tells the system what the executing app code is allowed to do. It's tricky to get right -- you have to manage the trust boundaries internally yourself, and it will be easy to get wrong. (B) capabilities tend to depend on the environment, not just the app. E.g., production vs. staging vs. dev environments may well want different capabilities. (C) capabilities are probably best expressed declaratively, but putting tokens into into APIs makes the app handle them procedurally. Once an apps capabilities need to go beyond the trivial, this will really explode the complexity, making it brittle and very difficult to handle correctly. E.g. suppose you use express... how would it know how to pass tokens to its dependencies? It would sort of need to understand your app's security model, which it can't do. What would actually happen is express would presume a certain model and pass tokens it receives accordingly. App dev would adjust the tokens it passes to express, adjusting until it works. So that at least works even if responsibilities are mixed up. But now express can't make internal changes to the way it calls it dependencies without risk of breaking apps that take the update. Anyway, I'm thinking the capabilities need to be specified external to the app. Say, in a config file (or files) of some sort. The app could only be allowed at most read access to the capabilities file(s); (A). There are some pretty common and straightforward ways to customize config for different environments; (B). A config file lends itself naturally to a declarative form; (C).
• cryptonector 5y ago
  > A modest proposal > It seems .. frankly, solvable. I don't think TFA understands the reference. > Capabilities to the rescue And that's the summary of TFA: sandbox everything! Except there's always sandbox escapes. OK, those are harder, so sandboxing more is a start, for sure. There's no panacea here. Dependencies are costly. Open coding is costly. Curating external source and packages is costly. It's all costly. We need to recognize a lot of these costs.
• pojzon 5y ago
  Every time I read anything about JS, WAT comes to my mind and that explains why everything in web UI domain would require a complete bottom-up rewrite. And ofcourse that is why it will never happen. We just keep on digging deeper in mud same as with climate..
  • cryptonector 5y ago
    Gary Bernhardt's WAT video never fails to make me laugh.
• chha 5y ago
  No packages or repositiories deserves trust, at least not in the current state of affairs. There is no magic fix that will enable you to trust packages just by adding a new framework or anything similar, and what the linked article is addressing is only half the problem. We also need a way to make packages auditable. Package signing by the publisher and the repository needs to be mandatory. Having an actual link between the package and the commit it was built on and a way to reproduce the build[1] also needs to be possible. This would allow for proper code reviews, not just of your own code but also audits of whatever extra components you are using. Organizations need to define sensible thresholds for when you can use a package and when you implement the code yourself, to avoid adding a library only to use one function. They also need to define trust; what is needed to trust a package or a maintainer. And we need a system and proper best practice to guide us on what to trust; do we really want to add a package with 100 direct or transitive dependencies, where some hasn't been updated for the past two years, some are maintained by solo developers and some are just implementing already existing functionality? All of these are hard to implement and justify, when the current situation seem to work for a lot of people. [1] - https://wiki.debian.org/ReproducibleBuilds https://wiki.debian.org/ReproducibleBuilds
• tlhunter 5y ago
  I was once part of a startup named Intrinsic. We had built out a complex product that protected Node.js processes with extreme granularity. Not much information exists about the product after it was acquired but the blog posts are still up: https://medium.com/intrinsic-blog/securing-node-js-applications-with-intrinsic-db47ff86cff8 https://medium.com/intrinsic-blog/securing-node-js-applicati...
• newusertoday 5y ago
  what is the solution for people who want to use it for frontend only?just importing js libraries that will only need to work on browser so no access to system resources without nodejs or browser? any ideas?
• bugrasan 5y ago
• fabioferrari 5y ago
  We are distributing trust in a too thin way. Node packages should be grouped in superset packages with a concentrated trust on special maintainers. Makes no sense to upgrade a lot of small packages each time we do a "npm update".