12 ms·

Very annoying - the apparent author of the backdoor was in communication with me over several weeks trying to get xz 5.6.x added to Fedora 40 & 41 because of it

• coding123 3y ago
  [flagged]
  • zb3 3y ago
    Not sure why are people downvoting you... it's pretty unlikely that various Chinese IoT companies would just decide it's cool to add a backdoor, which clearly implies that no matter how good their intentions are, they simply might have no other choice.
    • gpm 3y ago
      There are roughly speaking two possibilities here: 1. His machine was compromised, he wasn't at fault past having less than ideal security (a sin we are all guilty of). His country or origin/residence is of no importance and doxing him isn't fair to him. 2. This account was malicious. There's no reason we should believe that the identity behind wasn't fabricated. The country of origin/residence is likely falsified. In neither case is trying to investigate who he is on a public forum likely to be productive. In both cases there's risk of aiming an internet mob at some innocent person who was 'set up'.
      • zb3 3y ago
        Why do you exclude the possibility that this person was forced to add this at gunpoint?
        • encoderer 3y ago
          Yes exactly this. How do people think state actors have all those 0 day exploits. Excellent research? No! They are adding them themselves!
      • rjzzleep 3y ago
        The back door is in the upstream GitHub tarball. The most obvious way to get stuff there is by compromising an old style GitHub token. The new style GitHub tokens are much better but it’s somewhat intransparent what options you need. Most people also don’t use expiring tokens. The authors seems to have a lot of oss contributions, so probably an easy target to choose.
      • account42 3y ago
        I think the letters+numbers naming scheme for both the main account and the sockpuppets used to get him access to xz and the versions into distros is a strong hint at (2). Taking over xz maintainership without any history of open source contributions is also suspicious.
    • noncoml 3y ago
      Because it’s naive to think that the owner of the account used his real identity.
      • zb3 3y ago
        But my point is that people living in China might be "forced" to do such things, so we unfortunately can't ignore the country. Of course, practically this is problematic since the country can be faked
        • praut 3y ago
          [dead]
• gigatexal 3y ago
  Name and shame this author. They should never be allowed anywhere near any open projects ever again.
  • Lichtso 3y ago
    They might have burnt the reputation built for this particular pseudonym but what is stopping them from doing it again? They were clearly in it for the long run.
    • jethro_tell 3y ago
      You're assuming that it's even a single person, it's just a gmail address and an avatar with a j icon from a clip art thing.
      • Lichtso 3y ago
        I literally said "they", I know, I know, in English that can also be interpreted as a gender unspecific singular. Anyways, yes it is an interesting question whether he/she is alone or they are a group. Conway's law probably applies here as well. And my hunch in general is that these criminal mad minds operate individually / alone. Maybe they are hired by an agency but I don't count that as a group effort.
  • 0xbadcafebee 3y ago
    Please don't? 1. You don't actually know what has been done by whom or why. You don't know if the author intended all of this, or if their account was compromised. You don't know if someone is pretending to be someone else. You don't know if this person was being blackmailed, forced against their will, etc. You don't really know much of anything, except a backdoor was introduced by somebody. 2. Assuming the author did do something maliciously, relying on personal reputation is bad security practice. The majority of successful security attacks come from insiders. You have to trust insiders, because someone has to get work done, and you don't know who's an insider attacker until they are found out. It's therefore a best security practice to limit access, provide audit logs, sign artifacts, etc, so you can trace back where an incursion happened, identify poisoned artifacts, remove them, etc. Just saying "let's ostracize Phil and hope this never happens again" doesn't work. 3. A lot of today's famous and important security researchers were, at one time or another, absolute dirtbags who did bad things. Human beings are fallible. But human beings can also grow and change. Nobody wants to listen to reason or compassion when their blood is up, so nobody wants to hear this right now. But that's why it needs to be said now. If someone is found guilty beyond a reasonable doubt (that's really the important part...), then name and shame, sure, shame can work wonders. But at some point people need to be given another chance.
• yieldcrv 3y ago
  I wonder who the target was!
  • juliusdavies 3y ago
    Every Linux box inside AWS, Azure, and GCP and other cloud providers that retains the default admin sudo-able user (e.g., “ec2”) and is running ssh on port 22. I bet they intended for their back door to eventually be merged into the base Amazon Linux image.
    • Bulat_Ziganshin 3y ago
      my understanding is that any Debian/RPM-based Linux running sshd would become vulnerable in a year or two. The best equivalent of this exploit is the One Ring. So the really strange thing is why they put so little effort into making this undetectable. All they needed was to make it use less time to check each login attempt.
      • kevincox 3y ago
        In the other hand it was very hard to detect. The slow login time was the only thing that gave it away. It more seems like they were so close to being highly successful. In retrospect improving the performance would have been the smart play. But that is one part that went wrong compared to very many that went right.
    • throwaway7356 3y ago
      You don't need a "ec2" user. A backdoor can just allow root login even when that is disabled for people not using the backdoor. It just requires the SSH port to be reachable unless there is also a callout function (which is risky as people might see the traffic). And with Debian and Fedora covered and the change eventually making its way into Ubuntu and RHEL pretty much everything would have this backdoor.
  • swagmoney1606 3y ago
    Probably less of an individual and more of an exploit to sell.
• formerly_proven 3y ago
  I think this has been in the making for almost a year. The whole ifunc infrastructure was added in June 2023 by Hans Jansen and Jia Tan. The initial patch is "authored by" Lasse Collin in the git metadata, but the code actually came from Hans Jansen: https://github.com/tukaani-project/xz/commit/ee44863ae88e377a5df10db007ba9bfadde3d314 https://github.com/tukaani-project/xz/commit/ee44863ae88e377... > Thanks to Hans Jansen for the original patch. https://github.com/tukaani-project/xz/pull/53 https://github.com/tukaani-project/xz/pull/53 There were a ton of patches by these two subsequently because the ifunc code was breaking with all sorts of build options and obviously caused many problems with various sanitizers. Subsequently the configure script was modified multiple times to detect the use of sanitizers and abort the build unless either the sanitizer was disabled or the use of ifuncs was disabled. That would've masked the payload in many testing and debugging environments. The hansjans162 Github account was created in 2023 and the only thing it did was add this code to liblzma. The same name later applied to do a NMU at Debian for the vulnerable version. Another "<name><number>" account (which only appears here, once) then pops up and asks for the vulnerable version to be imported: https://www.mail-archive.com/search?l=debian-bugs-dist@lists.debian.org&q=from:%22krygorin4545%22 https://www.mail-archive.com/search?l=debian-bugs-dist@lists...
  • zb3 3y ago
    Also I saw this hans jansen user pushing for merging the 5.6.1 update in debian: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1067708 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1067708
    • hxelk1 3y ago
      From: krygorin4545 <krygorin4545@proton.me> To: "1067708@bugs.debian.org" <1067708@bugs.debian.org> Cc: "sebastian@breakpoint.cc" <sebastian@breakpoint.cc>, "bage@debian.org" <bage@debian.org> Subject: Re: RFS: xz-utils/5.6.1-0.1 [NMU] -- XZ-format compression utilities Date: Tue, 26 Mar 2024 19:27:47 +0000 Also seeing this bug. Extra valgrind output causes some failed tests for me. Looks like the new version will resolve it. Would like this new version so I can continue work. -- Wow. (Edited for clarity.)
  • 3y ago
• sorokod 3y ago
  [flagged]
  • matheusmoreira 3y ago
    Don't blame the guy. Could have happened to anyone. Even you.
    • sorokod 3y ago
      [flagged]
      • dang 3y ago
        It's uncharitable and comes across as a personal attack, which is not allowed in HN comments.
• jonathanspw 3y ago
  Yesterday sure was fun wasn't it :p Thanks for all your help/working with me on getting this cleaned up in Fedora.
  • w4ffl35 3y ago
    Is it normal that when I try to uninstall xz it is trying to install lzma?
    • inetknght 3y ago
      It means that `xz` was depended upon by something that depends on eg "xz OR lzma"
  • speleding 3y ago
    PSA: I just noticed homebrew installed the compromised version on my Mac as a dependency of some other package. You may want to check this to see what version you get: xz --version Homebrew has already taken action, a `brew upgrade` will downgrade back to the last known good version.
    • cozzyd 3y ago
      Is it actually compromised on homebrew though? I guess we can't be sure but it seemed to be checking if it was being packaged as .deb or .rpm?
    • mthoms 3y ago
      Thanks for this. I just ran brew upgrade and the result was as you described: xz 5.6.1 -> 5.4.6
    • jonahx 3y ago
      I also had a homebrew installed affected version. I understand it's unlikely, but is there anything I can do to check if the backdoor was used? Also any other steps I should take after "brew upgrade"?
      • 3y ago
• KingOfCoders 3y ago
  Sleeper.
• smeehee 3y ago
  Debian have reverted xz-utils (in unstable) to 5.4.5 – actual version string is “5.6.1+really5.4.5-1”. So presumably that version's safe; we shall see…
  • xorcist 3y ago
    Is that version truly vetted? "Jia Tan" has been the official maintainer since 5.4.3, could have pushed code under any other pseudonym, and controls the signing keys. I would have felt better about reverting farther back, xz hasn't had any breaking changes for a long time.
    • rnmkr 3y ago
      It's not only that account, other maintainer has been pushing the same promotion all over the place.
    • tobias2014 3y ago
      It looks like this is being discussed, with a complication of additional symbols that were introduced https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1068024 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1068024
      • binkHN 3y ago
        Thanks for this! I found this URL in the thread very interesting! https://www.nongnu.org/lzip/xz_inadequate.html https://www.nongnu.org/lzip/xz_inadequate.html
        • deleted 3y ago
          [deleted]
        • mehdix 3y ago
          It is an excellent technical write-up and yet again another testimonial to the importance of keeping things simple.
• Jommi 3y ago
  the account was either sold or stolen
  • account42 3y ago
    That's pure speculation and there are plenty of hints to the contrary.
• userbinator 3y ago
  because of it's "great new features" "great" for whom? I've seen enough of the industry to immediately feel suspicious when someone uses that sort of phrasing in an attempt to persuade me. It's no different from claiming a "better experience" or similar.
  • SilasX 3y ago
    You can find more examples of that kind of puffer if you go to a website's cookie consent pop-up and find the clause after "we use cookies to...".
    • transcriptase 3y ago
      I’ve long thought that those “this new version fixes bugs and improves user experience” patch notes that Meta et al copy and paste on every release shouldn’t be permitted.
      • nebula8804 3y ago
        Tell me about it. I look at all these random updates that get pushed to my mobile phone and they all pretty much have that kind of fluff in the description. Apple/Android should take some steps to improve this or outright ban this practice. In terms of importance to them though I imagine this is pretty low on the list. I have dreamed about an automated LLM system that can "diff" the changes out of the binary and provide some insight. You know give back a tiny bit of power to the user. I'll keep dreaming.
        • Nullabillity 3y ago
          It's worse, as someone who does try to privide release notes I'm often cut off by the max length of the field. And even then, Play only shows you the notes for the latest version of the app.
      • fl0ki 3y ago
        Slack's Mac app release notes [1] rotate a few copy pastes, here's the one that shits me the most. > We tuned up the engine and gave the interiors a thorough clean. Everything is now running smoothly again. Yeah nah mate, if every release is the first release where everything is running smoothly, I'm not going to believe it this time either. Makes me wonder if the team has some release quota to fill and will push a build even if nothing meaningful has actually changed. [1] https://slack.com/release-notes/mac https://slack.com/release-notes/mac
• kapouer 3y ago
  Github accounts of both xz maintainers have been suspended.
  • miduil 3y ago
    Not true, the original author wasn't suspended: https://github.com/Larhzu https://github.com/Larhzu https://github.com/JiaT75 https://github.com/JiaT75 was suspended for a moment, but isn't anymore?
    • boutique 3y ago
      Both are suspended for me. Check followers on both accounts, both have a suspended pill right next to their names.
      • miduil 3y ago
        Ah, thanks for correcting me there - really weird that this isn't visible from the profile itself. Not even from the organization. The following page for each other show both accounts suspended indeed. https://github.com/Larhzu?tab=following https://github.com/Larhzu?tab=following https://github.com/JiaT75?tab=following https://github.com/JiaT75?tab=following
        • miduil 3y ago
          Lasse's account was restored
      • fargle 3y ago
        github should add a badge for "inject backdoor into core open source infrastructure"
        • account42 3y ago
          Hey maybe it would get bad actors to come clean trying to get that badge.
    • 3y ago
• mongol 3y ago
  Interesting that one of the commits commented on update of the test file that it was for better reproducibility for having been generated by a fixed random seed (although how goes unmentioned). For the future, random test data better be generated as part of the build, rather than being committed as opaque blobs...
  • WhyNotHugo 3y ago
    I agree on principle, but sometimes programmatic generating test data is not so easy. E.g.: I have a specific JPEG committed into a repository because it triggers a specific issue when reading its metadata. It's not just _random_ data, but specific bogus data. But yeah, if the test blob is purely random, then you can just commit a seed and generate in during tests.
• api 3y ago
  I’m surprised there isn’t way more of this stuff. The supply chain is so huge and therefore represents so much surface area.
  • SoftTalker 3y ago
    There probably is. Way more than anyone knows. I bet every major project on github is riddled with state actors.
  • cozzyd 3y ago
    Imagine if sshd was distributed by PyPI or cargo or npm instead of by a distro.
    • darkelder 3y ago
      [dead]
• junon 3y ago
  GitHub has suspended @JiaT75's account. EDIT: Lasse Collin's account @Larhzu has also been suspended. EDIT: Github has disabled all Tukaani repositories, including downloads from the releases page. -- EDIT: Just did a bit of poking. xz-embedded was touched by Jia as well and it appears to be used in the linux kernel. I did quick look and it doesn't appear Jia touched anything of interest in there. I also checked the previous mirror at the tukaani project website, and nothing was out of place other than lagging a few commits behind: https://gist.github.com/Qix-/f1a1b9a933e8847f56103bc14783ab7b https://gist.github.com/Qix-/f1a1b9a933e8847f56103bc14783ab7... -- Here's a mailing list message from them ca. 2022. https://listor.tp-sv.se/pipermail/tp-sv_listor.tp-sv.se/2022-November/000802.html https://listor.tp-sv.se/pipermail/tp-sv_listor.tp-sv.se/2022... -- MinGW w64 on AUR was last published by Jia on Feb 29: https://aur.archlinux.org/cgit/aur.git/log/?h=mingw-w64-xz https://aur.archlinux.org/cgit/aur.git/log/?h=mingw-w64-xz (found by searching their public key: 22D465F2B4C173803B20C6DE59FCF207FEA7F445) -- pacman-static on AUR still lists their public key as a contributor, xz was last updated to 5.4.5 on 17-11-2023: https://aur.archlinux.org/cgit/aur.git/?h=pacman-static https://aur.archlinux.org/cgit/aur.git/?h=pacman-static EDIT: I've emailed the maintainer to have the key removed. -- Alpine was patched as of 6 hours ago. https://git.alpinelinux.org/aports/commit/?id=982d2c6bcbbb579e85bb27c40be84072ca0b1fd9 https://git.alpinelinux.org/aports/commit/?id=982d2c6bcbbb57... -- OpenSUSE is still listing Jia's public key: https://sources.suse.com/SUSE:SLE-15-SP6:GA/xz/576e550c49a3667a44b6375fb28916e5/xz.keyring https://sources.suse.com/SUSE:SLE-15-SP6:GA/xz/576e550c49a36... (cross-ref with https://web.archive.org/web/20240329235153/https://tukaani.org/misc/jia_tan_pubkey.txt https://web.archive.org/web/20240329235153/https://tukaani.o...) EDIT: Spoke with some folks in the package channel on libera, seems to be a non-issue. It is not used as attestation nor an ACL. -- Arch appears to still list Jia as an approved publisher, if I'm understanding this page correctly. https://gitlab.archlinux.org/archlinux/packaging/packages/xz/-/blob/main/keys/pgp/22D465F2B4C173803B20C6DE59FCF207FEA7F445.asc https://gitlab.archlinux.org/archlinux/packaging/packages/xz... EDIT: Just sent an email to the last committer to bring it to their attention. EDIT: It's been removed. -- jiatan's Libera info indicates they registered on Dec 12 13:43:12 2022 with no timezone information. -NickServ- Information on jiatan (account jiatan): -NickServ- Registered : Dec 12 13:43:12 2022 +0000 (1y 15w 3d ago) -NickServ- Last seen : (less than two weeks ago) -NickServ- User seen : (less than two weeks ago) -NickServ- Flags : HideMail, Private -NickServ- jiatan has enabled nick protection -NickServ- *** End of Info *** /whowas expired not too long ago, unfortunately. If anyone has it I'd love to know. They are not registered on freenode. EDIT: Libera has stated they have not received any requests for information from any agencies as of yet (30th Saturday March 2024 00:39:31 UTC). EDIT: Jia Tan was using a VPN to connect; that's all I'll be sharing here.
• drazk 3y ago
  After reading the original post by Andres Freund, https://www.openwall.com/lists/oss-security/2024/03/29/4 https://www.openwall.com/lists/oss-security/2024/03/29/4, his analysis indicates that the RSA_public_decrypt function is being redirected to the malware code. Since RSA_public_decrypt is only used in the context of RSA public key - private key authentication, can we reasonably conclude that the backdoor does not affect username-password authentication?
  • cbolton 3y ago
    Isn't it rather that the attacker can log in to the compromised server by exploiting the RSA code path?
• thayne 3y ago
  Do you know if it was actually the commit author, of if their commit access was compromised?
  • bpye 3y ago
    If it was a compromise it also included the signing keys as the release tarball was modified vs the source available on GitHub.
• LispSporks22 3y ago
  Nice. I worked on a Linux disto when I was a wee lad and all we did was compute a new md5 and ship it.
• KingLancelot 3y ago
  [dead]
• heresWaldo 3y ago
  [flagged]
• nodesocket 3y ago
  Can legal action be taken against the author if it's found he maliciously added the backdoor?
  • saltedtangerine 3y ago
    Good luck with that. We don't even know what country is he from. Probably from China but even if so. Good luck finding him among 1.5 Billions.
• menomatter 3y ago
  his account is active again on github https://github.com/JiaT75 https://github.com/JiaT75
• landownersubgrp 3y ago
  It is not good to take into consideration something with any unreadable text instead of the open text of the programme. It should be excluded.