4 ms·

Instead of needing the honeypot openssh.patch at compile-time https://github.com/amlweems/xzbot/blob/main/openssh.patch https://github.com/amlweems/xzbot/blob/m

Instead of needing the honeypot openssh.patch at compile-time https://github.com/amlweems/xzbot/blob/main/openssh.patch https://github.com/amlweems/xzbot/blob/main/openssh.patch How did the exploit do this at runtime? I know the chain was: opensshd -> systemd for notifications -> xz included as transient dependency How did liblzma.so.5.6.1 hook/patch all the way back to openssh_RSA_verify when it was loaded into memory?

jeffrallen 3y ago
ifunc
deleted 3y ago
[deleted]
tadfisher 3y ago
When loading liblzma, it patches the ELF GOT (global offset table) with the address of the malicious code. In case it's loaded before libcrypto, it registers a symbol audit handler (a glibc-specific feature, IIUC) to get notified when libcrypto's symbols are resolved so it can defer patching the GOT.
- MuffinFlavored 3y ago
  > When loading liblzma, it patches the ELF GOT (global offset table) with the address of the malicious code. How was this part obfuscated/undetected?
  - bewaretheirs 3y ago
    it was part of the binary malware payload hidden in a binary blob of "test data". In a compression/decompression test suite, a subtly broken allegedly compressed binary blob is not out of place. This suggests we need to audit information flow during builds - the shipping production binary package should be reproduceably buildable without reading test data or test code.
    - MuffinFlavored 3y ago
      How/why did the test data get bundled into the final library output?
      - matthew-wegner 3y ago
        "xz/liblzma: Bash-stage Obfuscation Explained" covers it well https://gynvael.coldwind.pl/?lang=en&id=782 https://gynvael.coldwind.pl/?lang=en&id=782
      - deleted 3y ago
        [deleted]
      - acdha 3y ago
        That’s what the compromised build stage did. It’s really interesting to read if you want to see the details of how a sophisticated attacker works: https://gist.github.com/thesamesam/223949d5a074ebc3dce9ee78baad9e27#design https://gist.github.com/thesamesam/223949d5a074ebc3dce9ee78b... https://gynvael.coldwind.pl/?lang=en&id=782 https://gynvael.coldwind.pl/?lang=en&id=782
        MuffinFlavored 3y ago
        > build-to-host.m4 I wasn't aware that the rogue maintainer was able to commit himself without any PR review (or he snuck it through PR review) rogue steps in the build process as well that went unnoticed so that he could bundle decompressed `xz` streams from test data, that also patched output .so files well enough to add hooking code to them. How many "process failures" are described in that process that exist in every OSS repo with volunteer unknown untrusted maintainers?
        belthesar 3y ago
        That's kind of the rub here. > volunteer That's the majority of OSS. Only a handful of the projects we use today as a part of the core set of systems in the OSS world actually have corporate sponsorship by virtue of maintainers/contributors on the payroll. > unknown The actor built up a positive reputation by assisting with maintaining the repo at a time when the lead dev was unable to take an active role. In this sense, although we did not have some kind of full chain of authentication that "Jia Tan" was a real human that existed, that's about as good as it gets, and there's plenty of real world examples of espionage in both the open and closed source software world that can tell us that identity verification may not have prevented anything. > untrusted The actor gained trust. The barrier to gaining trust may have been low due to the mental health of the lead maintainer, but trust was earned and received. The lead maintainer communicated to distros that they should be added. That's the rub here. It's _really easy_ to say this is a process problem. It's not. This was a social engineering attack first and foremost before anything else. It unlocked the way forward for the threat actor to take many actions unilaterally.
  - saagarjha 3y ago
    There were several parts to it. For one, the control flow was non-standard, as the initial code hid itself into an ifunc for CPU capability resolution. From there it patched its own return address and branched into functions with plausible but misleading names that continued on to patch the table prior to RELRO kicking in.
deleted 3y ago
[deleted]

jeffrallen 3y ago

ifunc

deleted 3y ago

[deleted]

tadfisher 3y ago

When loading liblzma, it patches the ELF GOT (global offset table) with the address of the malicious code. In case it's loaded before libcrypto, it registers a symbol audit handler (a glibc-specific feature, IIUC) to get notified when libcrypto's symbols are resolved so it can defer patching the GOT.

MuffinFlavored 3y ago
> When loading liblzma, it patches the ELF GOT (global offset table) with the address of the malicious code. How was this part obfuscated/undetected?
- bewaretheirs 3y ago
  it was part of the binary malware payload hidden in a binary blob of "test data". In a compression/decompression test suite, a subtly broken allegedly compressed binary blob is not out of place. This suggests we need to audit information flow during builds - the shipping production binary package should be reproduceably buildable without reading test data or test code.
  - MuffinFlavored 3y ago
    How/why did the test data get bundled into the final library output?
    - matthew-wegner 3y ago
      "xz/liblzma: Bash-stage Obfuscation Explained" covers it well https://gynvael.coldwind.pl/?lang=en&id=782 https://gynvael.coldwind.pl/?lang=en&id=782
    - deleted 3y ago
      [deleted]
    - acdha 3y ago
      That’s what the compromised build stage did. It’s really interesting to read if you want to see the details of how a sophisticated attacker works: https://gist.github.com/thesamesam/223949d5a074ebc3dce9ee78baad9e27#design https://gist.github.com/thesamesam/223949d5a074ebc3dce9ee78b... https://gynvael.coldwind.pl/?lang=en&id=782 https://gynvael.coldwind.pl/?lang=en&id=782
      - MuffinFlavored 3y ago
        > build-to-host.m4 I wasn't aware that the rogue maintainer was able to commit himself without any PR review (or he snuck it through PR review) rogue steps in the build process as well that went unnoticed so that he could bundle decompressed `xz` streams from test data, that also patched output .so files well enough to add hooking code to them. How many "process failures" are described in that process that exist in every OSS repo with volunteer unknown untrusted maintainers?
        belthesar 3y ago
        That's kind of the rub here. > volunteer That's the majority of OSS. Only a handful of the projects we use today as a part of the core set of systems in the OSS world actually have corporate sponsorship by virtue of maintainers/contributors on the payroll. > unknown The actor built up a positive reputation by assisting with maintaining the repo at a time when the lead dev was unable to take an active role. In this sense, although we did not have some kind of full chain of authentication that "Jia Tan" was a real human that existed, that's about as good as it gets, and there's plenty of real world examples of espionage in both the open and closed source software world that can tell us that identity verification may not have prevented anything. > untrusted The actor gained trust. The barrier to gaining trust may have been low due to the mental health of the lead maintainer, but trust was earned and received. The lead maintainer communicated to distros that they should be added. That's the rub here. It's _really easy_ to say this is a process problem. It's not. This was a social engineering attack first and foremost before anything else. It unlocked the way forward for the threat actor to take many actions unilaterally.
- saagarjha 3y ago
  There were several parts to it. For one, the control flow was non-standard, as the initial code hid itself into an ifunc for CPU capability resolution. From there it patched its own return address and branched into functions with plausible but misleading names that continued on to patch the table prior to RELRO kicking in.

deleted 3y ago

[deleted]