25 ms·

Dependency Confusion: How I Hacked Into Apple, Microsoft and Other Companies

• jbverschoor 6y ago
  It won't be just companies. It'll be developers, sysops, etc who npm install a bazillion of packages, because the core language and libaries are not enough. Those people have keys, credentials and access to the internal networks.
  • randomsearch 6y ago
    Yep a big part of npm’s problems are actually just flaws with JS. Comparing with Java, it’s insane how many dependencies you have to manage, and that batteries are not included (esp weird when you consider front end apps involve downloading the code!).
• heipei 6y ago
  Previous discussion: https://news.ycombinator.com/item?id=26081149 https://news.ycombinator.com/item?id=26081149
  • database_lost 6y ago
    and the link to the original author's post: https://medium.com/@alex.birsan/dependency-confusion-4a5d60fec610 https://medium.com/@alex.birsan/dependency-confusion-4a5d60f...
• rachelbythebay 6y ago
  I’m cackling at how great this is. This is what happens when you trust the internet forever and just scarf down any old thing at build time. Of course it’ll get exploited! That’s what evil people do.
  • greggman3 6y ago
    There's more coming.... tons of github integrations ask for blanket access to your account vs Oauth, (https://github.com/marketplace https://github.com/marketplace). Tons of github users give that access, the access_tokens are only a password type breach away. If you have these access_tokens you can edit the repos they are for all you want.
    • PartiallyTyped 6y ago
      Different access tokens have different permissions, you can't just do whatever you want.
      • tomashubelbauer 6y ago
        Yeah, but at least with PATs (not sure about other token types), you can't scope them to a particular repo, so whenever you need to allow something to even see a private repo or write to a public repo, the token you supply to allow that can do that for all repos and that alone is potentially really destructive. I am not sure if there is a good reason for why PATs can't be scoped to a repository, because if they were allowed to be, it would do a lot for security I think.
        • RyJones 6y ago
          Worse: you can’t even scope it to an org, and some integration points are only available to org owners! GitHub is terrible.
          • seg_lol 6y ago
            That explains why at a place I worked we had multiple github orgs for different subteams, and/or they were really cheap ... yeah confused now.
            • RyJones 6y ago
              We also have multiple orgs, but we hit the requirement that you only have one bot account. It would be super nice if GitHub allowed much tighter scoping for PATs. I don’t know how many GitHub orgs the Linux Foundation has, but... hundreds? Having one bit account with wide permissions is a non starter
  • geek_at 6y ago
    I'm more amazed by the fact that they got bounties because the attack wouldn't be (easily) possible without insider knowledge on which dependencies their internal build system used
    • knowhy 6y ago
      > To test this hypothesis, Birsan began hunting for names of private internal packages that he could find in manifest files on GitHub repositories or in CDNs of prominent companies but did not exist in a public open-source repository. If I'm not mistaken insider knowledge wasn't necessary.
      • geek_at 6y ago
        my mistake
  • peteretep 6y ago
    You say this, but I feel like in the 20-odd years I’ve been using package managers I’ve seen very very few real world exploits?
    • hanselot 6y ago
      Why would the ones getting away with it bother publishing articles about it? Rather, 20 years is a plenty long enough time to cover your tracks.
    • Cthulhu_ 6y ago
      Consider yourself lucky I guess? I mean in the 20-odd years I've been driving I've never had an accident either.
    • dspillett 6y ago
      While the risk is low, the potential impact of a successful exploit is massive so the matter should be taken seriously. Remember how much was temporarily broken in the leftPad event? Imagine if all that had been silently back-doored instead?
  • CaptArmchair 6y ago
    There are a lot of expensive things you can outsource. Responsibility isn't among those. Free software / open source propels engineering as you can share and leverage the results of collective efforts. However, at no point did the concept come with inherent guarantees about concerns such as security. esr defined 19 points for "good" open source software development in his seminal essay "The Cathedral and the Bazaar". I feel some of those are sometimes easily thrown out of the window for the sake of "efficiency" or "cost-effectiveness". This issue resonates with bullet point 17 in particular: > A security system is only as secure as its secret. Beware of pseudo-secrets. I think this issue has less to do with package managers, and a lot with companies rushing into the convenience of public code platforms such as Github without properly vetting whether or not they might be inadvertently leaking internal information through packaging manifests. https://en.wikipedia.org/wiki/The_Cathedral_and_the_Bazaar https://en.wikipedia.org/wiki/The_Cathedral_and_the_Bazaar
    • iib 6y ago
      Offtopic, but I found nowhere to actually ask this question. Does anybody know if ESR is still alive? His blog [1] has not been updated in months--and looking at his post dates, this seems really out of character--, he hasn't posted anything on twitter, or his usual channels. [1] http://esr.ibiblio.org/ http://esr.ibiblio.org/
      • knowhy 6y ago
        This [0] is from Jan 2021. [0]: https://lists.nongnu.org/archive/html/gpsd-dev/2021-01/msg00059.html https://lists.nongnu.org/archive/html/gpsd-dev/2021-01/msg00...
      • tyingq 6y ago
        https://github.com/eric-s-raymond https://github.com/eric-s-raymond Scroll down to contribution activity. Forked a repo, created a pull request, etc, on Feb 1st 2021.
    • syshum 6y ago
      >>There are a lot of expensive things you can outsource. Responsibility isn't among those. That is not true at all, the industry both Development and even more so in Operations has been outsourcing responsibility for a long time, they is why we have support contracts, SLA's and other very expensive services we pay many many times more than the cost of hardware for... To outsource responsibility... Network down -- Call Cisco... Storage Down Call EMC or Nimble... etc
  • walrus01 6y ago
    Hey let's just sudo curl | bash what could possibly go wrong?
    • thombles 6y ago
      Not sure if serious, but I will point out this is significantly different. If I'm installing an application like homebrew or the Rust toolchain then I am explicitly giving them the right to code execution. It doesn't much matter whether they get it through the script on their website or the binaries downloaded from that website. Random libraries, possibly pulled in by a dependency of a dependency of a dependency... not so much.
      • hda2 6y ago
        I know this is besides the point, but rust can be installed - without piping the output of a public-facing server into a privileged shell - by using their offline installer: https://forge.rust-lang.org/infra/other-installation-methods.html#standalone https://forge.rust-lang.org/infra/other-installation-methods... They offer PGP signatures too!
    • da_big_ghey 6y ago
      That's curl | sudo bash smh what an amateur
    • beermonster 6y ago
      You even see Microsoft offering those types of install one-liners. e.g. curl -sSL https://dot.net/v1/dotnet-install.sh https://dot.net/v1/dotnet-install.sh | bash /dev/stdin <additional install-script args> See https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-install-script https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-in... There are other examples I've seen from time to time.
  • beermonster 6y ago
    And wonder how long it will take for similar repeats on other repos such as dockerhub, apt, nuget, homebrew etc etc.
  • christophilus 6y ago
    Honestly, it’s one of the things that makes me nervous about running Linux on all of my computers. At least with Windows (and probably OSX), my updates come from a single vendor who has strict internal code audits and security requirements. With Linux (I’m using Pop), my updates come from a package manager with a crapload of packages, each maintained by a different team / group with no central policy. There’s no way the small team at Pop can review and audit all of the things in the apt package system, and there have to be plenty of maintainers of popular packages who get sweet offers to sell out. Anyway. I’m sticking with Pop / Linux. But it does make me nervous!
    • jonny_eh 6y ago
      For Linux, it's about choosing a distro you can trust. Most manage their own repos, some more carefully than others.
    • danhor 6y ago
      At least with windows, the drivers aren't checked that much and accordingly, have had some serious issues. I'd guess distros are generally better off in that respect, but kernel space & user space aren't that different nowadays, when caring about your own security
  • tyingq 6y ago
    Also happy. I'm very happy to finally have a real world example to motivate all the folks that eye-rolled me every time I've raised it in the past. It just resonates better, especially with less technical leadership folks.
• seniorivn 6y ago
  solution: use nix as your package manager
  • ben_w 6y ago
    Helps in this specific case, but will not eliminate the broader issue. The broader issue is how can you trust 3rd party code to not do anything harmful, and it’s not like we can even perfectly trust our own fingers in that regard.
    • croon 6y ago
      But the broad issue will never be resolved. The entire foundation of society is built around some trust. What has proven to work is that we trust actors who are identifiable and have something to lose. This goes with everything from your bank, to government, to name brands, to your local restaurant. Nix package managers/repositories have a level of scrutiny to get into, and highly dedicated people in charge of. Random github repos (or npm packages) are extremely low effort/risk to set up. Of course the former can be abused, but the incentives are at least in its favor to likely be more trustworthy. And we have to make assumptions of trust everytime we sit down in our chair or turn on our computer, or plug in a space heater. We will never get around trust, but there are differences in levels of trust and trustworthiness.
      • Foxboron 6y ago
        >Nix package managers/repositories have a level of scrutiny to get into, and highly dedicated people in charge of. Random github repos (or npm packages) are extremely low effort/risk to set up. That's not really true though. Nix doesn't support signed sources, there are no signatures in the package repository and in theory "John Doe" with no information can add packages and send pull-requests. In practise nixpks is just a well moderated user repository and the level of scrutiny is less then the enterprise distros can offer. https://discourse.nixos.org/t/trust-model-for-nixpkgs/9450 https://discourse.nixos.org/t/trust-model-for-nixpkgs/9450 https://github.com/NixOS/nixpkgs/issues/20836 https://github.com/NixOS/nixpkgs/issues/20836
• ipsum2 6y ago
  Package management isn't what I initially think of when I hear "supply chain". Neat hack! It's like left-pad but malicious.
  • numbsafari 6y ago
    It should be. If you are a developer, your package manager, OS distributions, and any commercial software you use is all part of your supply chain. Your code is what it depends on.
• jtsiskin 6y ago
  That is insane that any company allowed this to happen. ""That said, we consider the root cause of this issue to be a design flaw (rather than a bug) in package managers that can be addressed only through reconfiguration," a Microsoft spokesperson said in the email." No, npm has scopes for a reason, why would that not fix this issue?
  • asiachick 6y ago
    Isn't it considered best practice to be secure by default? Wasn't that big fiasco with MongoDB? Why should PyPI, RubyGems, or npm be any different? I'm sure there is some reason but I'd expect them to all pull private repos before public. Maybe the bug wasn't explained correctly but if it prefers public over private that seems like a bug. OTOH, it certainly is an issue that if you forget and happen to test some code without being configured to have the private package server as your default then you'd get public repos. Maybe instead of named packages companies should be using private URLs for packages. That way you always get what you ask for?
    • joepie91_ 6y ago
      npm does not have any 'private package' functionality at all, instead you point it at a different registry server (using eg. Verdaccio or Artifactory) which then serves local packages and proxies public packages if they don't exist locally - or at least that's what they're supposed to do. Artifactory apparently didn't, and served up whichever was the highest version of public vs. private. Which is stupid. But the bottom line is that when using npm, the exact package selection policy is determined by whatever registry implementation you're talking to, and so it's the registry implementation which should prioritize private packages by default.
  • Cthulhu_ 6y ago
    Scopes were only introduced in NPM 2, and iirc it's still an optional feature. Companies that used NPM early on may have opted to never use those. But that's just NPM, it's an issue in all of the mentioned package managers.
• keyle 6y ago
  I'm flabbergasted by how silly this is. Bump the version and the package manager chooses yours online vs. the private one. Amazing. How silly and how expensive is this going to be as this blatant security issue is going ripple on for the next months to come.
  • prepend 6y ago
    This is why explicit pins are a good idea. Whenever you finish a project you should set the explicit versions in the lock and then tag it. The problem is with dependencies of your dependencies, but if they are public, then by their nature they won’t be using private packages that can be hijacked.
    • frakkingcylons 6y ago
      Even public packages have been hijacked. Pin all your dependencies (transitive included) and then use automation (e.g. dependabot) to update the pinned versions as needed.
  • beermonster 6y ago
    Debian has apt pinning for this kind of thing.
  • thw0rted 6y ago
    I don't think it actually works this way for NPM specifically, if you're using scoped packages correctly. I believe you can associate a scope with one (private) repo and it will not fall back on the public repo, or choose newer / higher-numbered versions on the public repo over a version from the private one.
• megous 6y ago
  But will anything change in the minds and hearts of developers?
  • sillysaurusx 6y ago
    Probably not. :)
• pabs3 6y ago
  For apt repositories you can do pinning by origin, which should prevent this issue.
  • ex3ndr 6y ago
    Does apt use tls by default today?
    • beermonster 6y ago
      Apt supports TLS via apt-transport-https (as you are probably already aware) but I don't think it's default in either Debian nor (X)Ubuntu derivatives. I'd like to know why TBH. The packages themselves are signed though, so I guess the risk is now on server authenticity as opposed to package integrity.
      • pabs3 6y ago
        The packages are not signed, but there is a hash chain from the signed Release files through to the packages themselves.
        • beermonster 6y ago
          Yes you are right, I stand corrected. Here is an explanation of the process under 'The current scheme for package signature checks' at the following url https://www.debian.org/doc/manuals/securing-debian-manual/deb-pack-sign.en.html https://www.debian.org/doc/manuals/securing-debian-manual/de....
• mbag 6y ago
  To mitigate this kind of supply chain attacks for python, we have created following tool [1], that will check python packages on Artifactory instance you specify and create packages with the same name on the PyPi. [1] https://github.com/pan-net-security/artifactory-pypi-scanner https://github.com/pan-net-security/artifactory-pypi-scanner
  • seg_lol 6y ago
    The thing that just happened is like a catastrophic chain-reaction collision in space. Now we will have to use guids for everything. Nothing has meaning.
  • joshlk 6y ago
    Uploading dummy packages to PyPi isn't the solution. It just pollutes PyPi and a nuisance to others. You have always been able to specify the `index-url` when installing packages using pip. This can also be added to `requirements.txt` files as well.
    • matusf 6y ago
      Specifying `index-url` is not a solution since `pip` will always choose a package with higher version regardless of the repository. Moreover, in case of same version, it will prioritize PyPI. This was discussed in following issues [0], [1]. [0]: https://github.com/pypa/pip/issues/5045 https://github.com/pypa/pip/issues/5045 [1]: https://github.com/pypa/pip/issues/8606 https://github.com/pypa/pip/issues/8606
    • mbag 6y ago
      Yes, if you have packages on the artifactory the `index-url` is always a way to go. However, if you forget to specify `no-index`, you might not get what you wanted, see [1] for how packages are found. And it's easy to make such mistake when using local resources (you forget to set proxy or internal DNS, new developer is not familiar with the setup and does plain `pip install`, internal server is temporarily unreachable). >It just pollutes PyPi and a nuisance to others. I agree, but so are the packages that are no longer maintained. You also reserve pakcage name if you decide to opensource it. Furthermore, by creating package you are leaking metadata about your organization, i.e. some functionality can be inferred from package names. And sure you can train and try to enforce security awareness, but your people need to be right 100% of the time, while attackers need them to make only one mistake. Similar with namesquatting of the popular packages. https://pip.pypa.io/en/stable/reference/pip_install/#finding-packages https://pip.pypa.io/en/stable/reference/pip_install/#finding...
• nightpool 6y ago
  The article mentions that RubyGems is vulnerable to this, and that Shopify in particular downloaded and ran a gem named "shopify-cloud", but I'm curious as to how this is possible given a "normal" bundler pure-lockfile setup, or more generally the source-block directives I've seen in most Gemfiles. That is, given a Gemfile.lock like, e.g. GIT remote: https://github.com/thoughtbot/appraisal revision: 5675d17a95cfe904cc4b19dfd3f1f4c6d54d3502 specs: appraisal (2.1.0) bundler rake thor (>= 0.14.0) How would Bundler ever try and download the `appraisal` gem from RubyGems? The Gemfile section is more explicable. While newer Gemfiles look like this: source "http://our.own.gem.repo.com/the/path/to/it" do gem 'gemfromourrepo' end # or gem 'gemfromourrepo', source: "http://our.own.gem.repo.com/the/path/to/it" Older Gemfiles apparently looked like the following: source 'https://rubygems.org' source 'http://our.own.gem.repo.com/the/path/to/it' gem 'gemfromrubygems1' gem 'gemfromrubygems2' gem 'gemfromourrepo' Which seems obviously vulnerable to the dependency confusion issue mentioned. So is the understanding that Shopify's CI systems were running `bundle upgrade` or another non-lockfile operation? (possibly as a greenkeeper-like cron job?) Or is `--pure-lockfile` itself more subtly vulerable?
  • withinboredom 6y ago
    Someone will eventually update deps, not necessarily CI. But now that devs machine is compromised. The attacker probably only has a small window of time after it gets in, but it should be long enough to exfiltrate dot-files and the source code of whatever it gets included in. Now they have ssh keys (mine are on a yubikey), and the GitHub url. They can further push malicious code into the repo.
    • nightpool 6y ago
      I would hope most SSH keys are password-encrypted if not protected by a hardware token like yours, but I agree that the "unscoped-source" Gemfile syntax is a huge vulnerable hole, and a bad one. I'm just confused about how what seems like a pretty uncommon operation led to such an immediate response and code execution from Shopify. (I also don't think it's true that the attacker has a "small window of time"—as soon as they get a single RCE, it's over, if they're running on a normal dev machine then they can daemonize into the background, add persistence, and snoop events over time. CI systems are obviously less vulnerable to this by nature.)
• CGamesPlay 6y ago
  I know that node has `package-lock.json` and `yarn.lock`, which include integrity checks. Are these checks decorative only? How could npm have been affected by this issue?
  • nstart 6y ago
    IIRC you need to use npm ci to ensure that package-lock.json is used. That said, when developing locally you are going to use npm install or npm update and update the package.json and package-lock.json files accordingly. I could be entirely off target here since I'm writing purely from memory. But there seems to be a few different ways one could trigger a pull from the malicious repo and end up with it inside the package-lock.json file
    • nickkell 6y ago
      I feel stupid for using npm for so long without knowing about this command. In hindsight it's obvious that install was updating the lock file each time, so why should it have been any different on the ci server?
    • K0nserv 6y ago
      No that's not correct, in fact this comment and the two sibling comments are both wrong. Quouting from NPMs documentation[0] for npm install > This command installs a package, and any packages that it depends on. If the package has a package-lock or shrinkwrap file, the installation of dependencies will be driven by that, with an npm-shrinkwrap.json taking precedence if both files exist. See package-lock.json and npm Consider an example where in package.json you have `"react": "^16.11"` and this has been resolved and fixed in package-lock.json as 16.12 at a previous point in time. Running npm install will not cause NPM to install 16.14 even though it matches the pattern specified in package.json, instead 16.12 will be installed because that's what package-lock.json says. What npm install does do, is detect if you've made changes in package.json and only then does it re-resolve the dependency. In the above example, if you changed `"react": "^16.11"` to `"react": "^16.14"` in package.json and then ran npm install the package-lock.json would be changed and version 16.14 would be installed. Bundler and CocoaPods also work this way.
• jl6 6y ago
  The real solution is to design and build software components that can be finished, so they can be ruthlessly vetted - rather than the endless churn of updates.
  • Daho0n 6y ago
    Gamers often complain how they become free QA testers if they buy a game in the first few months after release as most games are full of bugs (hi Bethesda!) but it is way worse in things like JavaScript libraries etc. It's as if finished have become a foreign word to most developers. Look at the resent story about Linux stable kernels that have had more than 255 minor releases and think how much of a shit show it would have been if they added features too like most developers do. The excellent small stable tools of Unix should have taught us something.
    • mschuster91 6y ago
      > Look at the resent story about Linux stable kernels that have had more than 255 minor releases and think how much of a shit show it would have been if they added features too like most developers do. At least some distribution kernels do new feature backports - mostly to support new hardware on LTS versions (like e.g. Ubuntu did for the Raspberry Pi, see https://github.com/raspberrypi/linux/issues/3464 https://github.com/raspberrypi/linux/issues/3464).
  • CuriousNinja 6y ago
    Not sure why parent is being down-voted as I believe this is an important point. In my opinion this would be applying the unix philosophy of having small tools that does one thing and does it well to code libraries.
    • hobofan 6y ago
      Because as long as the underlying hardware and technology overall keeps progressing there isn't much practicality in "finishing" software. Sure you could just "finish" Linux at 5.0 and then introduce e.g. io_uring via Linux-with-io_uring 1.0 instead of adding it to Linux 5.1. Same goes for all the libraries that add support for io_uring. Yes, you could "finish" some software on the feature level, but you would still need to maintain it if you want to add support for new platforms, etc., or it will become obsolete sooner or later. In the case of still maintaining libraries, this would solve nothing in the context of this attack vector.
• baybal2 6y ago
  This is why end-developer signing is essential. This is not as amendable to CI, but that's the point.
  • rmoriz 6y ago
    As you see with chrome extensions and android barcode apps, this is not a solution. Developers change or for whatever reasons can change their mind and ship bad things.
    • baybal2 6y ago
      Not a solution to that, but certainly a big impediment for platform attacks.
  • HereBeBeasties 6y ago
    How does it work in practice, though? For example, create-react-app in NPM has a bajillion deps. Do I trust 8,000 keys? Which ones are OK? I get that you could in principle namespace things (at least for package managers that support this) and insist on a small set of company-internal signing keys for those namespaces. But managing all that isn't easy and what about for package ecosystems that don't really have namespaces (e.g. PyPI, NuGet)?
    • baybal2 6y ago
      > Do I trust 8,000 keys? Which ones are OK? You can at least trust more 8000 developers whose keys are centrally signed, than 8000 packages thrown into signing CI tooling by who knows.
• andrejserafim 6y ago
  Why would you want your CI to depend on an external source. Say a legit upgrade happened, but it has a breaking change. Now your build is broken. Fixed versions for as many things as you can (including OS images, apt packages, Docker images, etc) lead to changes in your CI under your control. Sure, you have to upgrade manually or by a script. But isn't plain build stability worth it? Not even talking about security.
  • HereBeBeasties 6y ago
    It probably doesn't. But are you saying devs never updates their dependencies?
    • andrejserafim 6y ago
      When one updates internal dependency versions one usually has to find them. At least that was the story with my gigs. So there's a listing somewhere. So you wouldn't get a random version even considered. Version shadowing and overriding is a totally different concern of course.
• deleted 6y ago
  [deleted]
• deleted 6y ago
  [deleted]
• peteretep 6y ago
  > After spending an hour on taking down these packages, Ingram stressed that uploading illicit packages on PyPI puts an undue burden on the volunteers who maintain PyPI. I dunno, feels like fair game to me
• kasperni 6y ago
  I'm surprised the reverse fully-qualified domain name (FQDN) model used by Java isn't more widely adopted. If you want to upload artifacts to the main repository (Maven Central) you first need to show ownership of a particular domain. For example, via a DNS TXT record (example [1]). Would make these kind of attacks a lot more difficult. [1] https://issues.sonatype.org/browse/OSSRH-61509 https://issues.sonatype.org/browse/OSSRH-61509
  • numpad0 6y ago
    because without proper incentive mechanism people just use "com.mycompany.greatproduct12345" for everything
  • tarruda 6y ago
    It seems some of the new package systems such as node/npm fail to learn from years of maturity of existing ecosystem such as Java's
    • raverbashing 6y ago
      "Years of maturity" or, just thinking about the problem for a bit. How long did it take npm to have scoped packages. Sure, let me create a "paypal" project, they only need one js project no? If Java suffers from excessive bureaucracy, the newer package developers/repos suffer from too much eagerness to ship something without thinking Not to mention dependency and version craziness. If you want your software to be repeatable you need to be specific with the versions and code you're taking.
      • mstipetic 6y ago
        It drives me crazy that "official" sounding package names like yaml are seemingly given basically first-come first serve, with no oversight. Publish anything you want, but call it Mark's awesome yaml library, or companyName-yaml or something like that so that people are aware that's not an officially supported project
        • 6y ago
• walrus01 6y ago
  npm in particular has been problematic for a long time: https://naildrivin5.com/blog/2019/07/10/the-frightening-state-security-around-npm-package-management.html https://naildrivin5.com/blog/2019/07/10/the-frightening-stat... https://techbeacon.com/security/check-your-dependencies-githubs-npm-finds-nasty-trojan-packages https://techbeacon.com/security/check-your-dependencies-gith... https://thenewstack.io/npm-password-resets-show-developers-need-better-security-practices/ https://thenewstack.io/npm-password-resets-show-developers-n...
  • joepie91_ 6y ago
    Except this wasn't a problem with npm but rather with private registry implementations, and a setup with npm + Verdaccio is apparently actually one of the few configurations that isn't vulnerable to this problem. Not that I didn't expect someone to immediately take the opportunity to complain about npm, of course, despite it having nothing to do with the problem at hand... as has become tradition in tech circles.
• nstart 6y ago
  This post seems like a good time to note that by default, there's no direct way to verify that what you are downloading from dockerhub is the exact same thing that exists on dockerhub [1]. Discovered after seeing a comment on HN about a bill of materials for software, i.e., a list of "approved hashes" to ensure one can audit exactly what software is being installed, which in turn led me to this issue. [1] - https://github.com/docker/hub-feedback/issues/1925 https://github.com/docker/hub-feedback/issues/1925
  • guax 6y ago
    I remember when we used to sign binaries and packages and nobody checked the pgp files anyways. We could have something similar better today, just need to be automated enough.
  • beermonster 6y ago
    I think image signing support (or at least was) is not as good as it can be. It would be nice if more images were signed by publishers and verification performed by default. Even then, that only gives you a stronger indication that the image hasn't been altered since it was signed by the image author at any point after it being signed. However it is not a guarantee that the source produced the binary content. It's also not a guarantee that the image author knew what they were signing - though this is a different issue. Debian has a reproducible builds initiative[1] so people can compile packages themselves and them match byte for byte what Debian built. Not sure how far they've got with that. https://wiki.debian.org/ReproducibleBuilds https://wiki.debian.org/ReproducibleBuilds
    • iam-TJ 6y ago
      Approximately 25,000 of just over 30,000 source packages are now reproducible builds - generating over 80,000 binary packages. See the graphic on the page you linked to: https://tests.reproducible-builds.org/debian/unstable/amd64/stats_pkg_state.png https://tests.reproducible-builds.org/debian/unstable/amd64/...
      • beermonster 6y ago
• sitkack 6y ago
  We need a blockchain for source. It is obvious and we just haven't come to terms with it yet. Then anyone can run anything provided they have the right key.
• koolba 6y ago
  > The packages had preinstall scripts that automatically launched a script to exfiltrate identifying information from the machine as soon as the build process pulled the packages in. Pre and post install scripts in NPM packages are such a terrible idea. Even when it’s not malware, it usually just a nagging donation request with a deliberate “sleep 5” to slow down your build and keep the text displayed.
  • joepie91_ 6y ago
    I don't understand why people keep endlessly complaining about postinstall scripts. Such 'nagging donation requests' were banned by npm pretty much days after they first appeared, IIRC, and npm itself is literally a tool for installing code to execute later, so there's no security issue here. If someone wanted to embed malware into a package, they wouldn't need postinstall scripts for it. This is really a complete nothingburger.
    • minitech 6y ago
      “install” and “execute later” don’t always involve the same permissions. If you apply restrictive sandboxes your code, package managers that aren’t designed to be able to download untrusted code are annoying. Of course, this problem isn’t unique to npm. (It’s actually the opposite – all you need to do with npm is --ignore-scripts, whereas pretty much every other popular package manager I use just makes it impossible.) And yes, you want to sandbox the install too anyway, but it at least needs permissions enough to do its job, i.e. interact with the network somehow. (Although I’m working on a tool to make that fully deterministic so it can never exfiltrate anything.) There’s also the possibility that there’s no “execute” step at all, like installing a dependency tree just to inspect source, or in theory being able to skip auditing unused code paths.
      • seg_lol 6y ago
        Keep editing, we will wait. Eventually you containerize the container and realize that only a single if is enough for everyone.
• NicoJuicy 6y ago
  I used this version trick in nuget, but the other way around. To update existing non-maintained public packages, mostly because they were on. Net framework and a lot moved to .net core. In visual studio you can set the priority of where packages have to be checked. My own package repo has a higher priority. I never thought about using it as an attack vector though.
  • wozer 6y ago
    I believe you can work around this attack vector .NET by referencing strong-named assemblies.
• robertlagrant 6y ago
  I've often wondered about this, even in the accidental case of someone registering a package you use internally. And I know it's not perfect, but in Python if you use Poetry means you get a poetry.lock file with package hashes built in, so that's something.
• donaldihunter 6y ago
  This was inevitable from the moment we let build systems and runtime systems fetch things automatically and unsupervised from public repos. This is the simplest and most blatant approach yet, but taking ownership of existing projects and adding malicious code is an ongoing problem. Even deleting a public project can have the effect of a DOS attack. When I first used maven, I was appalled by how hard it was to prevent it from accessing maven central. And horrified to see karaf trying to resolve jars from maven central at run time. What a horrible set of defaults. This behaviour should be opt-in, disabled by default, not opt-out through hard to discover and harder to verify configuration settings.
  • brabel 6y ago
    Funny that you mention Maven, because Maven is not really vulnerable to this kind of attack simply because it requires a groupId in all dependencies, and to publish under a certain groupId you must prove control of the domain it refers to, which makes this attack nearly impossible (it's only possible if you use an internal groupId which is not controlled by you on Maven Central, AND an attacker could claim that groupId successfully with Sonatype, AND you configure Maven to first look at Maven Central, and only then at your internal repos which would be stupid to do as you normally do the exact opposite - and most enterprise setups won't even proxy to Maven Central at all). Also, Maven uses pinned versions, normally, and won't just download whatever newer minor version happens to be published when it builds, which again makes this attack quite hard to pull off.
    • donaldihunter 6y ago
      Back then it would have been maven 2 which supported version ranges in a similar way to OSGi manifests. But I really only mentioned maven as the first build tool I used which reached out to public repos uninvited and could break my builds as a consequence of that.
• rectang 6y ago
  PGP signing of packages should be table stakes for publishing to a public repository. If unsigned packages are accepted by a public repository to reduce friction for newbies, such packages should be hidden by default. Then, build tools should be configurable such that they only pull in dependencies signed by PGP keys drawn from a whitelist. Finally, companies need to maintain private repositories of vetted dependencies and avoid pulling from public repositories by default — and this requirement needs to be configurable from the project's build spec and captured in version control.
  • angry_octet 6y ago
    If you've seen the PGP/GPG code you'll know what a trash fire it is, and if you follow its development you'll see how unfriendly the maintainers are when bugs are pointed out. Adding dependencies on PGP just makes everything worse. X.509 PKI for code signing is also terrible and very very complicated and error prone. Also consider the community nature of development. You need to handle all sorts of painful crypto issues now.
• trishankkarthik 6y ago
  This is NOT a supply chain attack. Solarwinds was a supply chain attack. This is a typosquatting demonstration that happens every one or two years.
  • Abimelex 6y ago
    it's not and the article explains why
    • trishankkarthik 6y ago
      Are you able to read the title? "...novel supply chain attack..." But HN is the typical groupthink it always is.
• technics256 6y ago
  Can't there be a "package signature" of some sort that is specified and checked against in a package-lock.json or yarn.lock?
  • dininski 6y ago
    I'll try to answer this from a JS-specific perspective. As someone previously mentioned - you do get hash checks if you're using `npm ci` in your CI/CD setup. You get the resolution path as well. Which is all you need to reproducibly resolve dependencies, *if* you have set up npm correctly in your pipeline. It would be unlikely to be exposed to this particular attack, at least not automatically in your deployment pipelines. However this is still very, very dangerous, because of day-to-day engineering, really. Any engineer doing a simple `npm install` can inadvertently bring in and execute malicious code from their machine. From there on out it would be somewhat trivial to gain further access to the same network the code war run from.
  • fernandotakai 6y ago
    pip has hashing signatures and i don't know why people don't use it. it's quite easy too. https://pip.pypa.io/en/stable/reference/pip_hash/ https://pip.pypa.io/en/stable/reference/pip_hash/ https://pip.pypa.io/en/stable/reference/pip_install/#hash-checking-mode https://pip.pypa.io/en/stable/reference/pip_install/#hash-ch...
• xurukefi 6y ago
  I never understood why these package repositories don't include some (opt-in?) integrity checking option using digital signatures. If I download code that executes on my machine there should be at least the option to establish some level of trust. We have been doing that with linux distro package managers for decades. Seems like common sense to me.
  • BillinghamJ 6y ago
    They largely do in various forms. Both npm and yarn, by default, record hashes of the dependencies you're using and check them when redownloading. I think the issue tends to be more that there's just so many packages (often nested 10+ deep) and it's best practice to keep them as up to date as possible. When it's fairly typical for a JS project to have thousands of dependencies, there isn't really any practical way to both stay up to date and carefully review everything you pull in. I think the only viable solution for companies taking this issue really seriously is to keep their numbers of dependencies down and avoid having significant deep/indirect dependencies. Edit: as an example, in my company's Node stack (for 10 services) - there's >900 dependencies. In our React stack (for 2 sites), more than 1600. Contrary to what you might think, these are actually pretty small, lightweight systems. So really whatever you might have thought was the worst-case scenario on numbers of deps, the reality is more like 10x that in the modern JS ecosystem. In many ways, the vast number of tiny dependencies are one of the strongest points of the JS ecosystem. But it doesn't come without caveats.
  • magicalhippo 6y ago
    I was thinking the same thing. Surely PayPal's packages should be signed by a certificate only PayPal has, and they would want to verify that before using their packages?
    • angry_octet 6y ago
      If the signature reqt is attached to the package metadata, the new package just removed it. If it's part of their custom build system, what signs third party packages? Would it just sign the new one anyway, because how does it know which ones should have PayPal internal signing? Or are you proposing manual controls? shudder Channels and priorities embedded in the package tools are a better approach, combined with something like Artifactory. Some channels might require packages are signed, and possibly monotonically versioned.
• 3np 6y ago
  It surprises me a bit the way they refer to in-house dependencies purely by version number. When we have internal dependencies in e.g. package.json, it's always referred to by an explicit url and git ref.
• omega3 6y ago
  Less than $4k average bounty for this.
• SideburnsOfDoom 6y ago
  The upstream article was posted yesterday, here "Dependency Confusion: RCE via internal package name squatting " https://news.ycombinator.com/item?id=26081149 https://news.ycombinator.com/item?id=26081149 "Dependency Confusion: How I Hacked Into Apple, Microsoft and Dozens of Other Companies, The Story of a Novel Supply Chain Attack, Alex Birsan" https://medium.com/@alex.birsan/dependency-confusion-4a5d60fec610 https://medium.com/@alex.birsan/dependency-confusion-4a5d60f...
• ehnto 6y ago
  Pulling packages down at build time seems ludicrous to me, I can understand it in a development environment, but I don't understand how "Pull packages from the public internet and put them into our production codebase" past any kind of robustness scrutiny. I guess it's a case of the ease of use proving too great, so convenient in fact that we just kind of swept the implications under the rug.
  • uncledave 6y ago
    Yes. The way we develop software quite frankly scares the shit out of me on a daily basis.
    • alistairSH 6y ago
      Even worse - if you consider yourself average, half the software out there was built on top of even weaker foundations. The web really is held together by duct tape and bubble gum.
    • RGamma 6y ago
      Well it's mostly held together by trust and (in the commercial case) warranty. That said there's so many potential entry points for malicious actors it's not even funny anymore (esp. in desktop computing)... I try not to think about it too much and have faith in the powers that be
      • tremon 6y ago
        Can you point me to a documented instance where a warranty claim on a software product yielded a useful outcome for the claimant? The only one I can remember was against Microsoft for forcing an upgrade to Windows 10, which wasn't (IIRC) a warranty claim but a bait-and-switch issue.
        • RGamma 6y ago
          Now that you mention it, I can't remember anything off the top of my head, but it's gotta stand for something, right?
• angry_octet 6y ago
  I've been wishing npm/pypi/apt etc would improve for ages, but it seems like infrastructure improves one disaster at a time, software one hack at a time. I'm only annoyed I didn't do it myself. The pypi maintainer is being ridiculous, it is much better to have this guy poke MSFT than have the Russians do it, he's doing them a favour.
• konaraddi 6y ago
  For npm enterprise, it looks like setting the scope (e.g. @acmecorp/internal-pkg) would mitigate the public and private confusion. For Verdaccio, an open source light weight npm registry, it first checks if a private package is available before searching the public npm registry (however, their best practices say to use a prefix for private packages https://verdaccio.org/docs/en/best https://verdaccio.org/docs/en/best )
• ex_amazon_sde 6y ago
  Ex-Amazon SDE here. > a unique design flaw of the open-source ecosystems This is a big generalization. Inside Amazon, as well as in various Linux distributions, you cannot do network traffic at build time and you can only use dependencies from OS packages. Each library has its own package and the code and licensing is reviewed. The only open source distribution that I know to have similar strict requirements is Debian. [I'm referring to the internal build system, not Amazon Linux] [Disclaimer: things might have changed after I left the company]
  • trishankkarthik 6y ago
    I work in this area. This is not a supply chain attack. This is a typosquatting "attack" people keep rediscovering every year or two. I know, because I wrote an as yet unpublished paper on safely pulling packages from private and public repos.
    • itake 6y ago
      Can you share a link to the paper? My email is in my HN bio.
    • acdha 6y ago
      I think you’re getting downvoted because your point is obscured by the confrontational tone. Argument by authority is especially unconvincing when you aren’t using common terms correctly. In normal usage, “typosquatting” refers to someone registering common misspellings in a shared namespace. As clearly described in the post this is not that but rather exploiting non-obvious differences in the order in which different namespaces are checked. Using terms correctly is especially important in security: someone who read your comment might incorrectly believe that this did not affect them because they are using the correct names for all of their dependencies.
    • psanford 6y ago
      There's no typos in this attack.
    • ex_amazon_sde 6y ago
• adolph 6y ago
  I think JFrog and Azure won the prize for product placement on this one. When the article listed “Azure Artifactory” I wondered if Azure was “sherlocking” JFrog, but no, they have a partnership. Given the SolarWinds vector I expect more investment in tooling security.
• p0d 6y ago
  I teach and one of my students, with little IT experience, asked me last week about the security of package management. I found myself using the many eyeballs argument. It only takes one set of bad eyeballs. It seems to me that down through the years ease of deployment trumps security. npm, mongodb, redis, k8s. Or maybe sysadmin has just become outdated? Maybe front of house still needs a grumpy caretaker rather than your friendly devops with a foot in both camps. We can now even outsource our security to some impersonal third-party so they can 'not' monitor our logs. EOG # end of grump
• Communitivity 6y ago
  This doesn't surprise me. Horrify.. yes. I've noticed more dev teams succumbing to the temptation of easiness that many modern package managers provide (NPM, Cargo, Ivy, etc.) - especially as someone who has to work with offline systems on a regular basis. Because of that ease there are fewer tools and tutorials out there to support offline package management. There are more for using caches, though these are often along the lines of either 'the package manager will do this for you and it just works (but in case it doesn't, delete node_modules or cargo clean and re-try)', or stand up a dependency server on your own machine with these proxy settings (which has it's own security issues and is frequently disallowed by IT cybersecurity policies). As an example, many blog articles I found a while back suggest using yumdownloader from the yum-utils package. This is unfortunately not reliable, as there are some packages that get skipped. I have found I need to script reading a list of dependencies from a file; then for each dependency: create a directory for it, use repotrack to download its RPM and it's transitive dependency RPMs in the dependency's directory; then the script aggregates all the RPMs into one directory, removes the OS installed RPMs, uses createrepo to turn that directory into a RPM repository, and then makes an USF ISO image out of the directory for transfer onto offline system and installation.
  • Diggsey 6y ago
    I disagree: the problem is not that package managers make things easy, it's just that several of them are poorly designed. The fact that pip/npm/gem etc. look for packages in a fallback location if not found in the private repository is a terrible design flaw. One which not all package managers have. For example, when you add a cargo dependency from a private registry, you have to specify the registry that the dependency comes from, so cargo will never go looking in some other place for that crate. I'm sure many other package managers also have designs that are not vulnerable in this way. Similarly, many package managers do not support pinning of transitive dependencies (with hashes), or pinning does not happen by default, so that many people are still using floating dependencies.
    • specialp 6y ago
      In the case of RubyGems for some time now it has been throwing a warning if you do not use the `source` block to scope for gems coming from multiple gemservers.
• deleted 6y ago
  [deleted]
• jwr 6y ago
  I have to build some CSS libraries that sadly use npm for building. The way I approach this is through rubber gloves: I create custom docker containers with npm and a specific set of dependencies, frozen in time. This way I can at least get reproducible and reliable builds. This doesn't mean I'm not vulnerable to dependency attacks, but it at least limits the window, because I update these dependencies very, very rarely.
• PhineasRex 6y ago
  The only really shocking part of this is that Artifactory is vulnerable to this. I expect developers to be lazy about build security because I've seen it over and over again at multiple companies, but Artifactory's whole purpose is to provide secure build dependency management. I'll be rethinking using Artifactory in my infrastructure.
  • HereBeBeasties 6y ago
    A good look at their (public) bug tracker might change your mind about how surprising this is.
• snarfy 6y ago
  Sadly I've had to fix this at more than one company. It's a bit of cognitive dissonance having to explain why downloading random shit from the internet during the build is a bad idea, yet here we are.
• mavhc 6y ago
  https://security.googleblog.com/2021/02/know-prevent-fix-framework-for-shifting.html https://security.googleblog.com/2021/02/know-prevent-fix-fra... At Google, we have those resources and go to extraordinary lengths to manage the open source packages we use—including keeping a private repo of all open source packages we use internally
  • 0xbadcafebee 6y ago
    That's more for availability than security. Assuming you keep the crypto checksums / author signatures of all the source code and packages, you don't need to keep a copy of the source / packages. Just verify them at download time. Many Linux distros don't even have a copy of all those binaries, they rely on HTTP mirrors of random organizations. It's also useful for your organization to rebuild all of the source code from scratch (for reproducible packages anyway) and compare the new ones to the old ones, looking for things like compiler or hardware injection attacks. Secure build systems are definitely non-trivial.
    • actuator 6y ago
      One more advantage of keeping it together can be easier development cycle. IDE features like autocompletion and building would be faster if artifacts can be cached.
      • lrem 6y ago
        My Vim is indeed magic. I start typing in a name and it autocompletes, then adds includes for whatever the package the thing I just used is in. I also can't imagine going back to not having code search, with its turning every identifier into a link.
    • zaphar 6y ago
      It's not just that. Because it's all in the same repo and built with the same build tool it's also easier to run the same security checks you would use for your own code all automatically as part of your build process. All the tooling you use to secure your own code can be used to secure third party code as well with the same low level of friction.
• 0xbadcafebee 6y ago
  My favorite supply chain attack is still the chip vendors. Even if you come up with a hardware security module in your chip to verify the code that's running on it, that can be (and has been) hacked too. Sleeping dragons could be lying in wait in billions of devices and nobody would know unless they went out of their way to do a low-level analysis.
• vlovich123 6y ago
  Does this work with AOT compiled languages? Surely the fake packages that get uploaded don't know the structure of the internal libraries enough, so for something like Cargo this would just cause in your build suddenly failing mysteriously & easy to spot. A build.rs could probably do some damage to your build systems temporarily for the 1 or 2 days (if not hours) it takes for engineers to track down what's happening.
• ryukafalz 6y ago
  I see a lot of people saying things like "this is why package signing is important" and "we need to know who the developers are" and "we need to audit everything." Some of that is true to some degree, but let me ask you this: why do we consider it acceptable that code you install through a package manager implicitly gets to do anything to your system that you can do? That seems silly! Surely we can do better than that? This article from Agoric is extremely relevant here, from a previous such incident (re: the event-stream package on npm): https://medium.com/agoric/pola-would-have-prevented-the-event-stream-incident-45653ecbda99 https://medium.com/agoric/pola-would-have-prevented-the-even... Put simply: in many cases, the dependencies you install don't need nearly as much authority as we give them right now. Maybe some of these packages need network access (I see a few named "logger" which might be shipping logs remotely) but do they need unrestricted filesystem access? Probably not! (They don't necessarily even need unrestricted network access either; what they're communicating with is likely pretty well-known.)
  • detaro 6y ago
    Indeed, being able to apply capabilities on a package level would be great, but I don't know many languages/environments that implement this as a first-class feature.
    • ryukafalz 6y ago
      Yeah. JavaScript is probably the closest to being there (with things like SES[0], LavaMoat[1], etc.) but we're not quite there yet. It's just shocking that this sort of thing is as seemingly obscure as it is; it's like the whole industry has collectively thrown up their hands and said code execution is unavoidably radioactively dangerous. (While simultaneously using package managers that... well.) But it doesn't have to be! [0] https://github.com/Agoric/ses-shim https://github.com/Agoric/ses-shim [1] https://github.com/LavaMoat/LavaMoat https://github.com/LavaMoat/LavaMoat
    • eecc 6y ago
      Java does. Of course it’s never been used systematically and it has received precious little attention to DevOps ergonomics, but the infrastructure is there
• fortran77 6y ago
  I don't use npm much, but once I'm out of the initial development phase with any package manager and am "feature complete" we generally lock versions down so at least we're always pulling a specific version in. And, of course, on production build machines, all packages are local. This isn't just for "security" -- it's to ensure we can always build the same bits we shipped, and to avoid any surprises when something has a legitimate update that breaks something else.
• pinacarlos90 6y ago
  Question for you guys here: Is this kind of attack possible using Nuget-Package manager?
  • progre 6y ago
    Seems like it's a possibility at least. I'd be very very interested to know as well.
  • forecast10 6y ago
    Yes, it's possible. I've tried it myself.
  • maximcus 6y ago
    Fresh [PDF from MSFT mentioning nuget](https://azure.microsoft.com/mediahandler/files/resourcefiles/3-ways-to-mitigate-risk-using-private-package-feeds/3%20Ways%20to%20Mitigate%20Risk%20When%20Using%20Private%20Package%20Feeds%20-%20v1.0.pdf https://azure.microsoft.com/mediahandler/files/resourcefiles...)
• ddtaylor 6y ago
  Does anyone else skip reading articles on Medium because of their login policies?
• tantalor 6y ago
  Like some other commenters, I too initially balked at the apparent misuse of "supply chain attack" but the linked paper provides a good definition, A software supply chain attack is characterized by the injection of malicious code into a software package in order to compromise dependent systems further down the chain. Backstabber’s Knife Collection: A Review of Open Source Software Supply Chain Attacks https://link.springer.com/chapter/10.1007%2F978-3-030-52683-2_2 https://link.springer.com/chapter/10.1007%2F978-3-030-52683-... To be clear, just calling this a "supply chain attack" and omitting "software" is going to cause confusion with traditional supply chains. The analogy is not quite apt: in a software build system you have complete visibility into the dependency tree, so this attack is less useful, whereas with hardware suppliers you are relying on the security of your vendor.
  • Terretta 6y ago
    > The analogy is not quite apt: in a software build system you have complete visibility into the dependency tree, so this attack is less useful, whereas with hardware suppliers you are relying on the security of your vendor. Not necessarily — plenty software still ships with the third party supply chain bits incorporated as binaries, including commercial software. User is relying on security of one or more in a chain of upstream vendors. See Cyberpunk 2077 DLLs for instance. https://twitter.com/CDPRED_Support/status/1356604047671898112?s=20 https://twitter.com/CDPRED_Support/status/135660404767189811... Cyberpunk “builds” their game with a software build system, but not all of it is them building it.
    • tantalor 6y ago
      Yes, good point of course. Although I would be very suspicious of binary deps served through npm.
• unilynx 6y ago
  These install hooks... Why are they needed at all and why can't package (de)installation be without side effects ? I'm sure the hooks are needed for things NPM can't do by itself, but they shouldn't run by default. That puts pressure on developers to avoid them, and puts pressure on NPM to add whatever functionality is missing from package.json in a safe way. (and have npmjs.com search rank packages without scripts above those that do)
  • bagacrap 6y ago
    What would happen if the install hooks weren't there? You'd still have client code calling into the compromised package. Would it be possible to handle those calls without knowing the symbol names used by the internal package?
• DecoPerson 6y ago
  Imagine we navigated the web using a command line tool called “goto” which works exactly like a package manager. If I want to open my bank’s site, I type “goto mybank” . I could easily find myself in trouble, because: - There’s no autocomplete or bookmarks, so typos are easy. - If “mybank” is a name provided by my company’s name server, I could find myself redirected to the public “mybank” entry because Mr. Not-A-Hacker says his name entry is more up to date (or because I forgot to tell ‘goto’ to check the company name server.) - There’s no “green padlock” to check while I’m actively using the destination site. (Though at this point it’s too late because a few moments after I hit enter the destination site had the same access to my machine & network that I do from my current terminal.) - A trusted site may later become malicious, which is bad due to the level of unrestricted and unmonitored access to my PC the site can have. - Using scripting tricks, regular sandboxed browser websites can manipulate my clipboard so I paste something into ‘goto’ that I didn’t realize would be in my clipboard, making me navigate to some malicious site and giving it full access to my machine (if ‘sudo’ as added to the front). This is just a few cases off the top of my head. If ‘goto’ was a real thing, we’d laugh it into being replaced by something more trustable. How have current package managers not had these vulnerabilities fixed yet? I don’t understand.
• siraben 6y ago
  This attack demonstrates one of the problems outlined in the Nix thesis[0], that is the problem of nominal dependencies. That is, dependencies of the dependencies, build flags and so on are not taking into account, and in particular, the source of a package. Nix makes it possible to query the entire build time and runtime dependency graph of a package, and because network access during build time is disabled, such a substitution attack would be harder to pull off. The declarations for how the source is downloaded is specified declaratively and can be pinned to a specific commit of a specific Git repository, for instance. [0] https://edolstra.github.io/pubs/phd-thesis.pdf https://edolstra.github.io/pubs/phd-thesis.pdf
• umvi 6y ago
  > I have been fascinated by the level of trust we put in a simple command like this one sigh... am I the only one that likes environments where you can run simple commands to install stuff and you can generally trust your package managers? All the security folks love to act dumbfounded when people trust things, but post-trust environments have terrible UX in my experience. I hate 2FA, for example, because now I have to tote my phone around at all times in order to be able to access any of my accounts. If I lose my phone or my phone is stolen while travelling, I'm hosed until I can figure out how to get back in. > So can this blind trust be exploited by malicious actors? Yes, it can. Trust can always be exploited by malicious actors, and no amount of software can change that. And it creates a world that sucks over time. Show me a post-trust, highly secure environment that isn't a major PITA to use. And not just for computers. I'm sure you could use social engineering to abuse trust of customer service reps (or just people in general) and do bad things, and the end result will be a world where people are afraid do any favors for other people because of the risk of getting burned by a "malicious actor".
• nine_k 6y ago
  Popularized 3 years ago: https://medium.com/hackernoon/im-harvesting-credit-card-numbers-and-passwords-from-your-site-here-s-how-9a8cb347c5b5 https://medium.com/hackernoon/im-harvesting-credit-card-numb...
  • randompwd 6y ago
    Doesn't cover the same attack vector, and that link is a piece of fiction.
    • nine_k 6y ago
      It's a piece of fiction, of course, but explains a very similar attack vector in colorful details. Using DNS for exfiltration is a nice trick but does not change much in the general approach.
• stephenr 6y ago
  And still people won’t vendor their dependencies, so changes to dependencies are never reviewed. To paraphrase family guy: you’re making this harder than it needs to be.
• wyc 6y ago
  Here's the application called deptrust I submitted to the Mozilla Builders program (didn't get in :P) to address this problem space before I had to focus more on my current job. Please let me know if there are any collaborators who would like to work on this together someday! https://docs.google.com/document/d/1EW6uSZB0_D0qZuDSGuxujuVEBkCbpWznv7gRcbFvUZs/edit https://docs.google.com/document/d/1EW6uSZB0_D0qZuDSGuxujuVE...
  • Jefro118 6y ago
    Hey there, any chance you want to turn this doc into a dedicated webpage? Here's a demo: https://demos.writary.app/deptrust https://demos.writary.app/deptrust
• wepple 6y ago
  Does NPM offer cryptographic hash pinning of packages the way that PyPI does?* why is this not more widely used? * https://flawed.net.nz/2021/02/02/PyPI-Security-State/ https://flawed.net.nz/2021/02/02/PyPI-Security-State/
• cooervo 6y ago
  this shouldn't be a problem with golang right? because it uses an id when go mod is used. I'm rusty on go since I haven't used it in over 2 years but I believe this shouldn't affect it?
• Bannednad 6y ago
  Bullshit you would be next to julian
• malinens 6y ago
  luckily I reserved my company's namespace in packagist few months ago. each package manager works differently and it is hard to know inner workings of all package managers
• alkonaut 6y ago
  What I don’t get from the article is the reasoning behind the design that the central repository “wins” over the local/override repository. How was that design chosen, not just once but in all 3 of those large package ecosystems. Did pypi/gems/node borrow their design from each other given their similarity in other aspects? Are there any situations where this behavior is desired? Does any of the other ecosystems have flaws like this (nuget, cargo..)?
  • carols10cents 6y ago
    Cargo will not ever look on crates.io for a library if you specify a `registry` attribute for a dependency. https://doc.rust-lang.org/cargo/reference/registries.html https://doc.rust-lang.org/cargo/reference/registries.html
• tppiotrowski 6y ago
  Dev 1: “npm -g package_name” doesn’t work. Dev 2: try “sudo npm -g package_name”.
• forty 6y ago
  I don't understand why there is this issue. We publish our internal npm packages in the @company namespace and we own this namespace on the public npm registry. Problem solved, isn't it?
  • andrethegiant 6y ago
    Yes, I'm confused by this too. Scoped packages on npm solves this problem, yet it isn't mentioned in the article at all.
• fareesh 6y ago
  This seems to be tending towards the generic problem of permissions that we have seen previously elsewhere. For example in the case of Facebook, it used to be that users would accept permissions without considering them, and in-turn, various apps would access their data in bad faith. Likewise for mobile apps. Eventually Facebook removed many of the overtly powerful permissions entirely, likewise with the mobile operating systems. In the case of mobile, the concept of "runtime permissions" was also introduced that required explicit approval to be granted at the time of authorization. On Android, location access now prompts the user in the notification area informing the user of an app that accessed their location. Can some of these ideas be borrowed to the package/dependency management world? "The package you are about to install requires access to your hard drive including the following folders: x/y/z/all?
• soheil 6y ago
  This brings a whole new level of awareness to package files where a simple typo can mean your machine can be rooted. From now on I'll always be terrified whenever changing any of my package.json, Gemfile or requirements.txt files.
• jcims 6y ago
  Can anyone point me at a resource where I can download the full set of packages that are in the npm registry?
  • jcims 6y ago
    using https://replicate.npmjs.com/_all_docs https://replicate.npmjs.com/_all_docs
• claw_howitzer 6y ago
  Did this guy just make hundreds of thousands of dollars off this single bug (type)?
• furstenheim 6y ago
  Npm ecosystem already has the solution. Use namespaces @yelp/infra-js where @yelp is the npm user. It's not possible for an attacker to publish on that name in the public npm
• smilliken 6y ago
  This is both a security bug and a reproducibility bug. If anyone outside your network can break your build, your build is broken! It's mission critical to have a working build. The way Nix handles this is that every external resource is cached and hashed, and every reference to an external resource must have a hash integrity check. If someone swaps out a package on a web server somewhere, rebuilds keep working because they don't need to re-fetch (because the hash wasn't changed by an operator), and fresh builds fail with an error indicating the hash is invalid, which should trigger an investigation (in practice, this is exceedingly rare, and IMO always deserves attention). I dream for when build reproducibility is considered table stakes like version control.
• homakov 6y ago
  Why didn't npmjs/rubygems just check failed lookup requests for "shopify-cloud" etc and block those for a while to prevent damage, and notify the companies (doing their best)? Seems like low hanging solution.
• milo_im 6y ago
  Diffend allows you to manage the risks that come with using open-source third party dependencies by providing malware detecting security scanning and a risk management platform for your Ruby dependencies. https://diffend.io/ https://diffend.io/