8 ms·
Post Mortem: axios NPM supply chain compromise
- charcircuit 6mo agoDoes OIDC flow block this same issue of being able to use a RAT to publish a malicious package?
- hsbauauvhabzb 6mo agoNo, once the computer is compromised nothing really helps assuming the attacker is patient enough.
- fortuitous-frog 6mo agoNo. axios (v1 at least; not v0) were setup to publish via OIDC, but there's no option on npmjs for package maintainers to restrict their package to *only* using OIDC. The maintainer says his machine was infected via RAT, so if he was using software-based 2FA, nothing could have prevented this.
- dgellow 6mo agoActually there is an option to restrict to only OIDC publishing. It is a bit hidden and relies on a different form for reasons I really cannot understand. Npm UX is just so bad. Point 4 from https://npmdigest.com/guides/npm-trusted-publishing#ux-problem https://npmdigest.com/guides/npm-trusted-publishing#ux-probl... (I wrote that guide page for myself because I always get annoyed when dealing with npm OIDC)
- mcintyre1994 6mo agoNope, the most restrictive option available is to disallow tokens and require 2FA. I think that using exclusively hardware 2FA and not having the backup codes on the compromised machine probably would have prevented this attack though.
- pepve 6mo agoSomeone in the linked Github thread describes an attack where the attackers waited for the victim to use their Yubikey for an AWS login, giving the attackers access to AWS as well. I don't think hardware 2FA is safe against a RAT.
- the8472 6mo agoLogins are session-based. You could tie publishing of a package to a signature from the key, then 1 tap = 1 package hash. But yeah, if the system is compromised and the attacker is doing interactive attacks they can wait for something that requires using the key and then trigger the publishing and win a race against the real prompt. To the user it might just appear like having to tap twice.
- uticus 6mo ago> March 31, around 01:00 UTC: community members file issues reporting the compromise. The attacker deletes them using the compromised account. Interesting it got caught when it did.
- jamiemallers 6mo ago[dead]
- fraywing 6mo agoIncredible uptick in supply chain attacks over the last few weeks. I feel like npm specifically needs to up their game on SA of malicious code embedded in public projects.
- simulator5g 6mo agoThat's the reality of modern war. Many countries are likely planting malware on a wide scale. You can't even really prove where an attack originated from, so uninvolved countries would also be smart to take advantage of the current conflict. Like if you primarily wrote German, you would translate your malware to Chinese, Farsi, English, or Hebrew, and take other steps to make it appear to come from one of those warring countries. Any country who was making a long term plan involving malware would likely do it around this time.
- altmanaltman 6mo agoYou can write code in Chinese and Farsi?
- axitanull 6mo agoYou can deliberately put comments and descriptions using those language.
- simulator5g 6mo agoYes and there have been documented cases of translated malware. Sometimes its done a little sloppily and there is other evidence that points to the origin being in another country that doesn't speak the language its written in. But even then, you can't really prove they didn't just use a residential VPN or whatever.
- ipnon 6mo agoNPM is designed to let you run untrusted code on your machine. It will never work. There is no game to step up. It's like asking an ostrich to start flying.
- akersten 6mo agoAny good payload analysis been published yet? Really curious if this was just a one and done info stealer or if it potentially could have clawed its way deeper into affected systems.
- sheept 6mo agoThis article[0] investigated the payload. It's a RAT, so it's capable of executing whatever shell commands it receives, instead of just stealing credentials. [0]: https://safedep.io/axios-npm-supply-chain-compromise/ https://safedep.io/axios-npm-supply-chain-compromise/
- Zopieux 6mo agoNot much we didn't know (you're basically SOL since an owner was compromised), however we now have a small peek into the actual meat of the social engineering, which is the only interesting news imho: https://github.com/axios/axios/issues/10636#issuecomment-4180237789 https://github.com/axios/axios/issues/10636#issuecomment-418...
- hatmanstack 6mo agojasonsaayman and voxpelli had useful write ups from the "head on a swivel" perspective of what to watch out for. Jason mentioned "the meeting said something on my system was out of date." they were using Microsoft meeting and that's how they got RCE. Would love more color on that.
- pas 6mo agothey are cloning Zoom and MS Teams, and try to get people to either copy a script (which is in a textarea that's conveniently too small to show the whole script, and scrollbars are hidden by CSS, and there's a copy button, and when you paste it into the terminal you'll see last few lines, also look innocent, but there's a curl | zsh or `mshta` somewhere in there), download and run a binary/.dmg (and it might be even signed by GoogIe LLC. - the name chosen to look good in the usual typeface used on macOS). ... it seems the correct muscle memory response to train into people is that "if some meeting link someone sent you doesn't work, then you should create one and send them the link" (and of course never download and execute anything, don't copy scripts into terminals, but it seems even veteran maintainers do this, etc...) see Infection Chain here https://cloud.google.com/blog/topics/threat-intelligence/unc1069-targets-cryptocurrency-ai-social-engineering https://cloud.google.com/blog/topics/threat-intelligence/unc... textarea at the bottom of this comment: https://github.com/axios/axios/issues/10636#issuecomment-4182134203 https://github.com/axios/axios/issues/10636#issuecomment-418...
- ajross 6mo ago> it seems the correct muscle memory response [is something other than] never download and execute anything Arrgh. You're looking at the closest thing to a root cause and you're just waving over it. The culture of "just paste this script" is the problem here. People trained not to do this (or, like me, old enough to be horrified about it and refuse on principle) aren't vulnerable. But you just... give up on that and instead view this as a problem with "muscle memory" about chat etiquette? Good grief, folks. At best that's security theater. FWIW, there's also a root-er cause about where this culture came from. And that's 100% down to Apple Computer's congenital hatred of open source and refusal to provide or even bless a secure package management system for their OS. People do this because there's no feasible alternative on a mac, and people love macs more than they love security it seems.
- robshippr 6mo agoThe interesting detail from this thread is that every legitimate v1 release had OIDC provenance attestations and the malicious one didn't, but nobody checks. Even simpler, if you're diffing your lockfile between deploys, a brand new dependency appearing in a patch release is a pretty obvious red flag.
- clawfund 6mo ago[flagged]
- GCUMstlyHarmls 6mo agoTo be honest, I would have assumed the tooling would do attestation verification for me. The diffing the lockfile would be on me though.
- seanmarshall 6mo ago[flagged]
- anematode 6mo agoLooks like a very sophisticated operation, and I feel for the maintainer who had his machine compromised. The next incarnation of this, I worry, is that the malware hibernates somehow (e.g., if (Date.now() < 1776188434046) { exit(); }) to maximize the damage.
- ffsm8 6mo agoIsn't that already how it is? I mean the compromised machine registers itself on the command server and occasionally checks for workloads. The hacker then decides his next actions - depending on the machine they compromised they'll either try to spread (like this time) and make a broad attack or they may go more in-depth and try to exfiltrate data/spread internally if eg a build node has been compromised
- lexcamisa54 6mo ago[dead]
- lrvick 6mo agoI ask this on every supply chain security fail: Can we please mandate signing packages? Or at least commits? NPM rejected PRs to support optional signing multiple times more than a decade ago now, and this choice has not aged well. Anyone that cannot take 5 minutes to set up commit signing with a $40 usb smartcard to prevent impersonation has absolutely no business writing widely depended upon FOSS software. Normalized negligence is still negligence.
- 4ndrewl 6mo agoIs the onus really on people who write code here? It really should be on those who choose to use this unsigned code, surely?
- lorenzohess 6mo agoPerhaps, but if it's gotten to the point where millions of people download the unsigned code, signing should probably become required. Even reproducible builds.
- 4ndrewl 6mo agoRequired by who though? If your business etc depends upon some code, it's up to you to ensure its quality, surely? You copy some code onto your machine then it's your codebase, right?
- lrvick 6mo agoWhile I think anyone unwilling to sign their code is negligent, I also feel anyone unwilling to ensure credible review of code has been done before pushing it to production is equally negligent.
- lrvick 6mo agoAnyone that maintains code for others to consume has a basic obligation to do the bare minimum to make sure their reputations are not hijacked by bad actors. Just sign commits and reviews. It is so easy to stop these attacks that not doing so is like a doctor that refuses to wash their hands between patients. If you are not going to wash your hands do not be a doctor. If you are not going to sign your code do not be a FOSS maintainer.
- nurettin 6mo agoI never understood why all the CAS tutorials pushed axios. This was before vite and build-scripts was how you did react. After the compromise I reviewed some projects and converted them to pure JS fetch and vite.
- JackSmith_YC 6mo ago[dead]
- kanehorikawa 6mo ago[dead]
- pianopatrick 6mo agoSeems to me the root of the problem was that the guy was using the same device for all sorts of stuff. Seems to me that one drastic tactic NPM could employ to prevent attacks like this is to use hardware security. NPM could procure and configure laptops with identity rooted in the laptop TPM instead of 2FA. Configure the NPM servers so that for certain repos only updates signed with the private key in the laptop TPM can be pushed to NPM. Each high profile repo would have certain laptops that can upload for that repo. Set up the laptop with a minimal version of Linux with just the command line tools to upload to NPM, not even a browser or desktop environment. Give those laptops to maintainers of high profile repos for free to use for updates. Then at update time, the maintainer just transfers the code from their dev machine to the secure laptop via USB drive or CD and pushes to NPM from the special laptop.
- pas 6mo agothey can simply make an app that requires tapping a button, so people don't end up with TOTP seeds stored in their password manager on the same notebook where they run 'publish' from
- momo_dev 6mo agothis is why i pin every dependency hash in my python projects. pip install --require-hashes with a locked requirements file catches exactly this, if the package hash changes unexpectedly the install fails. surprised this isn't the default in the npm ecosystem
- minitech 6mo agoNpm and the other JavaScript package managers do generate and check lockfiles with hashes by default. This was a new release, not a republishing of an old version (which isn’t possible on the npm registry anyway).
- momo_dev 6mo agoi wasn't aware npm lockfiles check hashes by default now. my concern is more about the initial install before a lockfile exists, like in CI from a fresh clone without a committed lockfile. but you're right, once the lockfile is there the hash mismatch would be caught.
- arafeq 6mo ago[dead]
- aeneas_ory 6mo agoCheck if your machine was affected with this tool: https://github.com/aeneasr/was-i-axios-pwned https://github.com/aeneasr/was-i-axios-pwned
- zwarag 6mo agoHow do we know this is not the next tool in line to compromise a machine?
- aeneas_ory 6mo agoRead the source code
- scottburgess33 6mo ago[dead]
- eviks 6mo ago> something on my system was out of date. i installed the missing item Given the "extreme vigilance" of the primitive "don't install unknown something on your machine" level is unattainable, can there really be an effective project-level solutions? Mandatory involvement of more people to hope not everyone installs random stuff, at least not at same time? (though you might not even have more people...)
- robshippr 6mo agoThe interesting detail from the GitHub thread is shaanmajid's observation that every legitimate v1 release had OIDC provenance attestations and the malicious one didn't, but nobody checks. Even simpler, if you're diffing your lockfile between deploys, a brand new dependency appearing in a patch release is a pretty obvious red flag without needing any attestation infrastructure.
- ncr100 6mo agoDupe comment - double submitted? https://news.ycombinator.com/item?id=47622805 https://news.ycombinator.com/item?id=47622805
- redoh 6mo ago[flagged]
- cyberax 6mo agoAnother point: do NOT use the "~" or "^" versions for automatic updates. Just lock everything tight in your package files. Then have an alert on the lockfile changes.
- stingraycharles 6mo agoPlease no AI posts on HN.
- panstromek 6mo agoWell, the hack didn't survive more than 2-3 hours if I'm not mistaken. I don't think that counts as "nobody acted on it."
- panstromek 6mo agoActually, from the OP, the timeline is: > March 31, 00:21 UTC: axios@1.14.1 published with plain-crypto-js@4.2.1 injected > March 31, around 01:00 UTC: axios@0.30.4 published with the same payload > March 31, around 01:00 UTC: first external detections > March 31, around 01:00 UTC: community members file issues reporting the compromise. The attacker deletes them using the compromised account. So it was found out almost immediately.
- falkensmaize 6mo agoThe fetch api has been widely available in browsers for a decade now. And in node since 18. A competent developer could whip up a more axios-like library with fetch in a day easily. You can do all the cool things like interceptors with fetch too. Yet most developers I work with just use it reflexively. This seems like one of the biggest issues with the npm ecosystem - the complete lack of motivation to write even trivial things yourself.
- timcobb 6mo agoFetch can't do a lot of table stakes stuff...
- paustint 6mo agoOk, well have AI write some table stakes for you in 10 minutes with 100% test coverage and only provide exactly what "table stakes" you are missing without any bells and whistles.
- dkdbejwi383 6mo agoSuch as?
- Xenoamorphous 6mo agoOne I’ve noticed is download/upload progress.
- deleted 6mo ago[deleted]
- jabr 6mo agoYou’ve been able to do download/upload progress using the Streams API with fetch for more than seven years now. https://developer.mozilla.org/en-US/docs/Web/API/Streams_API https://developer.mozilla.org/en-US/docs/Web/API/Streams_API
- jeremie_strand 6mo ago[dead]
- Xentyon 6mo agoThis is why I've moved to native fetch for most projects. The fewer dependencies in the chain, the smaller the attack surface. For API clients especially, fetch + a thin wrapper is usually enough.
- Chyzwar 6mo agoNPM should fix this mess. Adding postinstall should require approval from NPM. NPM clients should not install freshly published packages. NPM packages should be scanned after publishing. High profile packages should verify upstream git hash signature. NPM install should run in sandbox and detect any attempt to install outside project directory. But npm being part of multi trillion company cannot be bothered to fix any of these. Instead they push for tighter integration with GitHub with UX that suck.
- cromka 6mo ago> NPM clients should not install freshly published packages. That would be a beautiful example of Cobra effect: what about updates that fix vulnerabilities? You're gonna force users to wait couple days or a week before they can get malware removed?
- Chyzwar 6mo agoThis could be controlled by npm. Client ask for available versions anyway. If package is security fix then it can be made available instantly. But this delay gives time for security scanners and time to notify maintainers that package was published.
- mcintyre1994 6mo agoThen the malicious packages would always be published as a security fix.
- mcintyre1994 6mo agoIn cases like this that isn’t an issue, NPM takes the malicious package down and you roll back to the previous version. The problem would be new versions that fix security issues though, and because this is all open source as soon as you publish the fix everyone knows the vulnerability. You wouldn’t want everyone to stay on the insecure version with a basically public vulnerability for a week.
- 6mo ago
- mt18 6mo ago[dead]
- jeremie_strand 6mo ago[dead]
- toniantunovi 6mo ago[dead]
- dfir-lab 6mo ago[flagged]