9 ms·
One thing that comes to mind is that “Jia Tan” might be more accurately seen as a “sleeper” of some sort: a foot soldier who infiltrates a juicy open source pro
by irdc 3y ago
One thing that comes to mind is that “Jia Tan” might be more accurately seen as a “sleeper” of some sort: a foot soldier who infiltrates a juicy open source project and waits for further instructions; backdooring sshd might not have been part of the original plan.
Which raises the concerning question of how much more sleeper maintainers there are.
- berkes 3y agoI once got a (probably scam) offer for adding a cryptominer to a library that I maintained at that time. And a more serious offer to add trackers to a popular >1M installs app. Both cases I obviously ignored it. But it made me aware of a nasty attack vector: someone who's thanklessly building a wordpress-plugin, pip, npm, or whatever software, thanklessly dealing with issues, PRs, support, maintainence, often for no pay, suddenly gets offered three figure sums to add a few lines of "affiliate stuff" or such. There are many places in the world, or people in situations where this amount of money really makes a compelling case.
- irdc 3y ago“Given enough underfunded maintainers, all security is shallow.”[0] 0. https://en.wikipedia.org/wiki/Linus%27s_law https://en.wikipedia.org/wiki/Linus%27s_law
- lenerdenator 3y agoI wouldn't say "funding" is necessarily the problem. Most maintainers do it because they like doing it. Their main limiting factor is time. I can drop a million dollars an hour into a maintainer's lap; that doesn't mean they can dedicate every waking moment to a project. They still have human needs that money can't buy like sleep, family obligations, and health concerns. And that's making the assumption that the maintainer uses that million/hr to quit their job. No, the problem is a lack of trustworthy candidates for maintainership and a lack of time. There are components of a GNU userland that are now too complex for a single human to both maintain and enhance at the same time. We now need to target multiple distros (really, more than are necessary, strictly speaking) and ISAs. Most are written in systems programming languages like C that are more complex than the average software engineer in 2024 works with. We need consolidation, simplification, maintainer redundancy, and a trust/governance framework for packages.
- mikrotikker 3y agoWe need to utilise a specialised AI to scan through the code looking for bugs and security holes. Imagine if openai donated server time to this.
- moritonal 3y agoIt's why I actually always encourage app devs to charge for their apps, even open source ones. It creates an exchange of value for the author to feel valued and detract from these vectors.
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- kijin 3y agoThis is what worries me more. It's easy to point a finger at a specific Bad Guy® and shout "He did it!" It's much harder to face the reality that any maintainer of any open-source project can slowly burn out to a point where they become accomplices in an attack, or at least turn a blind eye. The pool of open-source developers does not split cleanly into honest contributors and evil agents. The boundary is quite fluid -- more so in some circles than in others -- and there are always temptations to move from one side to the other and back again.
- david_allison 3y ago> suddenly gets offered three figure sums to add a few lines of "affiliate stuff" or such Back of the envelope calculation: you're looking at 2 orders of magnitude more money from "affiliate stuff" than you would be from generous user donations
- berkes 3y agoWell, yes. But it's also something you can do once. When (not if) it comes out, all credibility is lost. Whereas donations, regardless of how puny, are recurring and potentially forever.
- strogonoff 3y agoI believe the problem of thankless maintenance is best solved with two things: the thanks (yes, we are all human and want recognition and appreciation from fellow humans)[0], and a stable employment (work for a good large business while open-sourcing what’s possible)[1]. If you do OSS for profit, then it can become a question of where is more money; but if you work a reliable job with insurance, relationships and other implications then the stakes may be a bit different. Many of the biggest OSS projects today were started by people who had no money in mind whatsoever. Some had other jobs, others were students, etc. If we feel relatively secure, we are driven by our innate desire to tinker, create cool things and show it off. [0] Undermined by LLMs that are used to gobble up your code and suggest it to others commercially and without attribution. [1] Undermined by low employment protections (if you can expect to be fired at any time, you would be less loyal), and by LLMs (whatever you open-source now more directly benefits Microsoft or whatever).
- david_allison 3y ago3. More maintainers Days of 100+ notifications aren't easy. Things will slip through
- strogonoff 3y agoAgreed, but especially in light of recent events it’d be important to know who they are, and that’s not always easy.
- aleph_minus_one 3y ago> and a stable employment (work for a good large business while open-sourcing what’s possible) Even if it was hypothetically possible to open-source basically everything that the team in which I work produces: The software that I work on is very specialized software that is used by the company's employees and customers for specialized purposes. Imagine some nice LoB application that is actually somewhat comfortable to use. It basically does "what the users need" and is thus deeply ingrained in some parts of the company's workflows. The only use someone outside the industry might have for it is "cosplaying being employed in this industry". A lot of software that is developed (in particular in companies that don't sell or rent software) is of this kind. Thus: the open-source scene does in my opinion not have any use for a huge amount of software that is actually developed and actively used.
- acdha 3y ago> There are many places in the world, or people in situations where this amount of money really makes a compelling case. It’s especially easy to imagine that using the classic intelligence agency playbook: monitor high-impact maintainers and look for leverage before making the approach (“hey, saw your post about the divorce settlement and that $%#@ cleaning you out. My affiliate marketing pays in bitcoin…”) just as they’ve done for ages.
- xign 3y agoThat's definitely true. And a lot of times it could be a random open source project that is under the radar and rarely thought about. E.g. The Great Suspender Chrome extension which was sold to an unknown buyer which later turned it to malware: https://www.bleepingcomputer.com/news/security/the-great-suspender-chrome-extensions-fall-from-grace/ https://www.bleepingcomputer.com/news/security/the-great-sus...
- echelon 3y agoWe could all be Jia Tan. Someone could be bought, killed and replaced, or simply shadowed when they die or go to jail. Anonymity makes this even easier.
- craftkiller 3y ago> killed [...] die or go to jail All of my commits are signed with a PGP key that is on hardware security tokens and password-protected. In the event of my death, my digital identity could not be stolen without backdoors in my hardware security tokens. That being said, $5 wrenches and large sums of money are still possible attack vectors.
- ccccccc1 3y agoA cool tax-free no questions 500k can convince a lot of people
- TheCondor 3y agoOne thing I’ve learned, not from direct experience but from observation. These things are way cheaper than the more ethical and optimistic of us in society think. Your point is totally valid but the number is probably more like $5k-10k.
- saagarjha 3y agoTax free $500k? I don't want the IRS to come after me. Please mark all your bribes as regular income thanks
- acdha 3y agoAlso don’t forget that not everyone expects perfection and a canny attacker can exploit that. It’s really easy to focus on how you’d avoid trojans, keyloggers, etc. but I’d also ask how likely it is that if someone sent a message from your email address claiming you’d lost your token in a minor accident, etc. that they’d believe it - or simply accept it if commits started showing up with a new key (maybe with an upgraded crypto system) since 99% of Git users never check those.
- andrewinardeer 3y agoI believe this is a nation state actor and there are a a fleet of 'Jia Tans' working on other OSS projects to backdoor operating systems. And some have probably succeeded.
- irdc 3y agoAt this point, considering the apparent ease with which a project that is used pretty much everywhere was taken over, that seems like a reasonable position.
- arp242 3y agoI can walk out on the street and stab someone to death if I wanted to. This is surprisingly easy. Just because something is relatively easy to pull off doesn't mean it happens a lot. It's also not that easy to pull off because you need to have a project with relatively few eyes and a place to hide it. In this case: binary tests. But most projects don't have those. There is no evidence for any of this, including that it's a nation-state actor. There's also a case to be made that it's NOT a nation-state actor as nation states use Linux and want a secure Linux. The NSA and such have somewhat conflicting interests here. We just don't know. It's likely we will never know. All of this is starting to resemble the spy paranoia of the first world war. A few spies got caught and suddenly everyone was now a suspected German spy (including a general, if I recall correctly, who was detained for a while because he couldn't answer a question about baseball or some such). I suspect that very soon people will start demanding maintainers put some of their blood in a Petri dish to be tested with a hot needle. Just in case.
- acdha 3y ago> There's also a case to be made that it's NOT a nation-state actor as nation states use Linux and want a secure Linux. The NSA and such have somewhat conflicting interests here. We just don't know. I agree that we do not know that it’s a nation-state but this point seems to work in the opposite direction: this attack was very carefully constructed so only someone with a particular key pair could exploit it. That’s reminiscent of what the NSA did with the Dual EC constants, and they were confident enough about that to push it into the FIPS requirements for federal IT.
- brabel 3y agoEveryone working on important open source code should have a real identity associated with them. The fact that "Jia Tan" was able to become a maintainer without anyone ever trying to figure out their real identity shows a huge weakness in our trust model in OSS (everyone real would have something like a Linked In page, Facebook, Twitter, Instagram, or better, their own website with stuff that could be used to ensure they're a real person - that could be faked as well but the amount of effort would be high, and checking this would be much better than just allowing effectively anonymous users to be maintainers - there's just no need for anonymity in this scenario!).
- ninkendo 3y ago> everyone real would have something like a Linked In page, Facebook, Twitter, Instagram, or better, their own website with stuff that could be used to ensure they're a real person Oof, I guess I’m not real then, as I have none of those things.
- ed_elliott_asc 3y agoAlso this can all be faked
- SanitaryThinkin 3y agoOn top of what you mentioned I also dislike the TSA-like response the OSS community is taking with this happen stance. I have anonomously contributed to many projects because I enjoy my privacy. All of my founding projects have also been done with anonymity. Because someone wants their anonmity and privacy does not mean they're nefarious, and I find it funny the group that takes to these principals most is negging on those ideas.
- xign 3y agoPersonally, I do find it hard to trust an open source project maintained by an anonymous person. (I'm talking about maintainership, not regular contributions that need to be code reviewed by another maintainer) I may toy around with them but I will probably not use them in a manner where I need to trust them continuously. It's totally cool for you to do whatever you want, since it's a free world after all, but if you want other people to use your code, then it's a two-way street no? Your code has a direct effect on their computers, and so they are placing their trust on you. You may value your privacy, but you need to balance that with other people's valuing their own security, and it's likely that whatever project you maintain may have an alternative as well. If you just want to commit some code and not have people use them then that's another issue altogether. I guess what I'm saying is: it's a two-way street. You can do things anonymously, but big companies / projects also don't have an obligation to use your code.
- constantcrying 3y ago>Which raises the concerning question of how much more sleeper maintainers there are. Given how easy the infiltration is and how extremely hard to detect it is, likely a lot. For an intelligence operation it is also extremely cheap, you just need a few knowledgeable developers spending some time each week on the project. The upside being a backdoor into a significant portion of infrastructure, the downside being wasted time. I do not think it is unlikely that in many important open source software projects there are one or two people assign to keep an eye on things. They don't even need to be malicious, just being somewhat trusted contributors is enough. I would be extremely surprised if the NSA hasn't a couple of guys who keep watch on the Linux Kernel.
- moritonal 3y agoThe irony is that this would make a oddly effective way of having paid open source devs who for the most part just honestly improve projects. With the massive downside they undermine it at a critical moment.
- rightbyte 3y agoYe then maybe thanks to them finally we'll have The year of the air gapped Linux desktop. I dunno what to do if e.g. Debian gets compromized, as in, I can't trust the collective of maintainers. I assume any Windows machine is backdoored. Trivially proven by forced auto updates. Maybe air gapping some home computer for sensitive data might be a good idea.
- pants2 3y agoThe ideal situation is having multiple intelligence agencies all working on one project and spotting each others' backdoors, so at the end of the day we just have a really secure and well-maintained project.
- david_draco 3y ago> Given how easy the infiltration is and how extremely hard to detect it is, likely a lot. I read it exactly the other way around: the infiltration took years and detecting it was the default with a fuzzer, which had to be disabled for the exploit to succeed. It speaks to the hardening and that hardening should be required more. And of course the precarious roles of maintainers, which have been discussed elsewhere.
- mrkramer 3y agoMy assumption is that this was state sponsored mass surveillance campaign of some kind but God knows what exactly they were looking for. I think if backdoor was discovered 2 or 3 months later, we maybe could understand better what they wanted to do. My speculation is that they wanted to build a massive botnet and then snoop on machines' processes and traffic looking for something. It's hard to speculate because luckily they were captured soon enough.
- swed420 3y agoI find it intriguing that out of all the speculative comment threads I've read so far, none of them have suggested it was Microsoft attempting to make FOSS look bad/vulnerable.
- beardedwizard 3y agoHow would that benefit Microsoft, who owns GitHub, the home of OSS? It's not a secret that oss is vulnerable, the opportunity for MS is to sell the solution to a captive audience.
- swed420 3y agoMicrosoft making the decision to own GitHub in the first place also speaks to my suspicion. Embrace, Extend, Extinguish.
- epr 3y agoI've never been concerned about spies infiltrating open source projects compared to legitimate maintainers being hacked, even now after this whole xz incident. I'll put it this way. Let's say a bad guy had a decent budget to spend on paying agents/criminals to break into maintainer's homes on their behalf with a rubber ducky, etc. I'd expect a pretty high success rate compromising their hardware...
- dartos 3y agoYou’re ignoring scale. A single Jia Tian can be infiltrating 10s or more OSS projects each week without needing to travel around the world physically stealing hardware from various maintainers who they then need to impersonate. They can just impersonate some anons with no real lives or connections and just get the keys to OSS projects given time.
- epr 3y ago> A single Jia Tian can be infiltrating 10s or more OSS projects each week Single? 10s or more per week?! I can't help but think you are underestimating the cost of developer time. How many hours of work did it take JT to infiltrate to the point of finally implementing a backdoor? How much does that time cost? > just get the keys to OSS projects given time. This is not what JT did though, and for good reason. Trust of anons in open source is generally built through contributions of real developer work over time. That does not scale. > without needing to travel around the world physically stealing hardware from various maintainers I wasn't suggesting stealing hardware to impersonate someone. I'm talking about hiring petty criminals or using field agents to break into a house, using physical hardware access to install a backdoor, etc. into the legit maintainers hardware. The field guy's goal is to not get caught, so the maintainer is unaware they are compromised. I suppose the limitation with both approaches (maintainer plant vs compromising maintainers) is cost. My educated guess is that the cost of hiring skilled developers from a very limited pool for multiple years is more than it would cost to hire criminals that are already breaking into houses for low risk jobs where they don't even need to steal anything.
- berniedurfee 3y agoWhen you find one cockroach, you can be sure there are thousands more you haven’t found.