5 ms·

What I haven't seen discussed much is the linking mechanism that allowed the lib to hook into RSA_public_decrypt. Plenty of talk about what could or could not b

by usrusr 3y ago

What I haven't seen discussed much is the linking mechanism that allowed the lib to hook into RSA_public_decrypt. Plenty of talk about what could or could not be achieved by even more process separation and the like, but little about that function call redirect. Could it be possible to establish a way to link critical components like the code for incoming ssh with libraries in some tiered trust way? "I trust you when and where I call you, but I won't allow you to introduce yourself to other call sites"? This would surely fall into the category of "there would be ways around it, so why bother?" that triggers a "by obscurity" reflex in many, but I'd consider it reduced attack surface.

supriyo-biswas 3y ago
An Erlang actor like model where a program’s sub components call each other by message passing with each having its own security context may work. However, there are multiple security contexts at play in an operating system; with regards to the XZ backdoor it’s mostly about capability based security at the module level, but you also have capabilities at the program level, and isolation at the memory level (paging), isolation at the micro architectural level, and so on. Ensuring all of these elements work together while still delivering performance seems to be rather challenging, and it’s definitely not possible for the Unix-likes to make a move to such a model because of the replacement of the concept of processes.
kzrdude 3y ago
systemd merged a change to using dlopen for compression libraries recently https://github.com/systemd/systemd/pull/31550 https://github.com/systemd/systemd/pull/31550 which is a safer linking method in that sense.
- nextaccountic 3y ago
  Why is it safer?
  - jcranmer 3y ago
    It means the libraries are only loaded when they are needed, so if you never use the (e.g.) xz compression feature, the xz library will not be loaded, and a backdoor added in the xz library simply can't trigger. (Another side note is this may change the initialization order of libraries--so the initialization functions of an xz library don't run until xz is first used, and this may fail to let you intercept the ssh routines in time.)
    - account42 2y ago
      > this may fail to let you intercept the ssh routines in time It only makes it harder since you can always patch the code of the entire process at runtime (remapping things writable as needed).
  - kzrdude 3y ago
    I won't pretend I understand it all, but apart from linking only when needed, some explanations have come out how it would have prevented this exploit path. From https://research.swtch.com/xz-script https://research.swtch.com/xz-script > The effect of the scripts is to arrange for the nefarious object file’s _get_cpuid function to be called as part of a GNU indirect function (ifunc) resolver. In general these resolvers can be called lazily at any time during program execution, but for security reasons it has become popular to call all of them during dynamic linking (very early in program startup) and then map the global offset table (GOT) and procedure linkage table (PLT) read-only, to keep buffer overflows and the like from being able to edit it. But a nefarious ifunc resolver would run early enough to be able to edit those tables, and that’s exactly what the backdoor introduced. This early execution would not be possible if liblzma was dlopened later.
- edflsafoiewq 3y ago
  This hides the dependencies from ldd doesn't it?
  - kzrdude 3y ago
    Yes, but there was some way to put them back my elf metadata
    - edflsafoiewq 3y ago
      How?
      - kzrdude 3y ago
        I looked at this again and I think it implies that there /would/ be a way but it's not really standardized how to do it yet? https://github.com/systemd/systemd/pull/31131#issuecomment-1917618062 https://github.com/systemd/systemd/pull/31131#issuecomment-1...
cozzyd 3y ago
would it be sufficient to monitor the symbols called by ltrace, somehow?
- timattrn 2y ago
  I wonder about this too, at least in a testing/honeypot build
sliken 3y ago
From what I can tell the problem is the use of glibc's IFUNC. This broke the testing for XZ, suddenly accounts appeared to lobby for disabling that testing, which enabled the exploit.
- xign 3y ago
  IFUNC is arguably not the real issue. IFUNC was used to create a function that will be called on library load (since you need a resolver function to decide which function to map in). There are other ways to create "callback on library load" as well. I think ifunc was actually used for obfuscation instead. Jia Tan created a somewhat plausible scenario of using ifunc's to select which CRC function at load time to use for performance reasons (whether that actually increases performance is arguable but at least plausible). The malicious version swapped the resolver function to be a malicious one but either way it's just a way to create a function that can be called on library init. The actual hook for intercepting the call was done via audit hooks. So I guess it's really two things working together.
browser1 3y ago
[dead]
MonkeeSage 3y ago
Essentially the code patches ifunc resolvers to call into a supplied malicious object by surreptitiously modifying c files at build time (when ./configure && make is run) and links the compromised object files with a `now` linker flag that causes the ifuncs to be resolved immediately at library load time, which calls the patched resolvers while the process linkage table is still writable, which is the important part that allows them to just hijack the RSA_public_decrypt function in memory when the library is loaded. There's an excellent technical breakdown of the backdoor *injection process here: https://research.swtch.com/xz-script https://research.swtch.com/xz-script

by usrusr 3y ago

supriyo-biswas 3y ago

An Erlang actor like model where a program’s sub components call each other by message passing with each having its own security context may work. However, there are multiple security contexts at play in an operating system; with regards to the XZ backdoor it’s mostly about capability based security at the module level, but you also have capabilities at the program level, and isolation at the memory level (paging), isolation at the micro architectural level, and so on. Ensuring all of these elements work together while still delivering performance seems to be rather challenging, and it’s definitely not possible for the Unix-likes to make a move to such a model because of the replacement of the concept of processes.

kzrdude 3y ago

systemd merged a change to using dlopen for compression libraries recently https://github.com/systemd/systemd/pull/31550 https://github.com/systemd/systemd/pull/31550 which is a safer linking method in that sense.

nextaccountic 3y ago
Why is it safer?
- jcranmer 3y ago
  It means the libraries are only loaded when they are needed, so if you never use the (e.g.) xz compression feature, the xz library will not be loaded, and a backdoor added in the xz library simply can't trigger. (Another side note is this may change the initialization order of libraries--so the initialization functions of an xz library don't run until xz is first used, and this may fail to let you intercept the ssh routines in time.)
  - account42 2y ago
    > this may fail to let you intercept the ssh routines in time It only makes it harder since you can always patch the code of the entire process at runtime (remapping things writable as needed).
- kzrdude 3y ago
  I won't pretend I understand it all, but apart from linking only when needed, some explanations have come out how it would have prevented this exploit path. From https://research.swtch.com/xz-script https://research.swtch.com/xz-script > The effect of the scripts is to arrange for the nefarious object file’s _get_cpuid function to be called as part of a GNU indirect function (ifunc) resolver. In general these resolvers can be called lazily at any time during program execution, but for security reasons it has become popular to call all of them during dynamic linking (very early in program startup) and then map the global offset table (GOT) and procedure linkage table (PLT) read-only, to keep buffer overflows and the like from being able to edit it. But a nefarious ifunc resolver would run early enough to be able to edit those tables, and that’s exactly what the backdoor introduced. This early execution would not be possible if liblzma was dlopened later.
edflsafoiewq 3y ago
This hides the dependencies from ldd doesn't it?
- kzrdude 3y ago
  Yes, but there was some way to put them back my elf metadata
  - edflsafoiewq 3y ago
    How?
    - kzrdude 3y ago
      I looked at this again and I think it implies that there /would/ be a way but it's not really standardized how to do it yet? https://github.com/systemd/systemd/pull/31131#issuecomment-1917618062 https://github.com/systemd/systemd/pull/31131#issuecomment-1...

cozzyd 3y ago

would it be sufficient to monitor the symbols called by ltrace, somehow?

timattrn 2y ago
I wonder about this too, at least in a testing/honeypot build

sliken 3y ago

From what I can tell the problem is the use of glibc's IFUNC. This broke the testing for XZ, suddenly accounts appeared to lobby for disabling that testing, which enabled the exploit.

xign 3y ago
IFUNC is arguably not the real issue. IFUNC was used to create a function that will be called on library load (since you need a resolver function to decide which function to map in). There are other ways to create "callback on library load" as well. I think ifunc was actually used for obfuscation instead. Jia Tan created a somewhat plausible scenario of using ifunc's to select which CRC function at load time to use for performance reasons (whether that actually increases performance is arguable but at least plausible). The malicious version swapped the resolver function to be a malicious one but either way it's just a way to create a function that can be called on library init. The actual hook for intercepting the call was done via audit hooks. So I guess it's really two things working together.

browser1 3y ago

[dead]

MonkeeSage 3y ago

Essentially the code patches ifunc resolvers to call into a supplied malicious object by surreptitiously modifying c files at build time (when ./configure && make is run) and links the compromised object files with a `now` linker flag that causes the ifuncs to be resolved immediately at library load time, which calls the patched resolvers while the process linkage table is still writable, which is the important part that allows them to just hijack the RSA_public_decrypt function in memory when the library is loaded. There's an excellent technical breakdown of the backdoor *injection process here: https://research.swtch.com/xz-script https://research.swtch.com/xz-script