5 ms·

> libsystemd is linked to all systemd servies, which opens a rich attack surface of third party services to backdoor. Many people have warned that systemd does

> libsystemd is linked to all systemd servies, which opens a rich attack surface of third party services to backdoor. Many people have warned that systemd does way too much for an init system, and presents way too much attack surface. Cluck, cluck (chickens coming home to roost).

himinlomax 3y ago
The backdoored library was linked to libsystemd, not systemd itself.
- blkhawk 3y ago
  "the car bomb was build into the spare wheel not the car itself...."
  - nottorp 3y ago
    "... and although the spare wheel is included by default, the car is modular and you can always remove it yourself" [By dismantling the car completely and reassembling it, since you'd have to rebuild from source...]
    - chgs 3y ago
      Most cars no longer come with a spare wheel, certainly by default. Not sure if that’s because cars don’t get punctures any more due to the great state of the roads here in the U.K, or because people just can’t even change a wheel.
      - nottorp 3y ago
        Not in most systemd distros though. Those include all kinds of spare wheels that automatically take over your actual wheels.
      - JonChesterfield 3y ago
        It's to help meet emissions targets according to the garage I bought from. The spare wheel weighs more than not having one.
- deleted 3y ago
  [deleted]
- inopinatus 3y ago
  The dependency is attributable, in the largest part, to systemd's neoplastic aggrandizement of userland infrastructure and associated plumbing, making this a distinction without much of a difference.
  - XorNot 3y ago
    "everyone should just reimplement LZMA!" What could possibly go wrong. I'm sure there's no history of compression tools having serious vulnerabilities due to implementation errors...
    - inopinatus 3y ago
      This is another furphy, because OpenSSH proper neither requires nor uses xz/lzma. It's made clear in Andres Freund's original report¹ that the libsystemd dependency dragging it along arises from distros patching openssh to support systemd notifications. The sad part is that systemd notifications are just a datagram on a socket, so using libsystemd for this is reminiscent of Joe Armstrong's banana. [1] https://seclists.org/oss-sec/2024/q1/268 https://seclists.org/oss-sec/2024/q1/268
      - LtWorf 3y ago
        As many have already pointed out, the library can also be linked to sshd via selinux.
        inopinatus 3y ago
        I've seen that ambit claim too, but I'm not even sure what distro(s) it is referring to since I'm unable to confirm it on any host where I have ldd casually to hand. Ref however https://seclists.org/oss-sec/2024/q1/356 https://seclists.org/oss-sec/2024/q1/356
        deleted 3y ago
        [deleted]
raverbashing 3y ago
Really I get the idea of systemd and I do think that the Unix principles were good for the 70s but maybe not so great today Still in the true LP fashion, a lot of design decisions are not being thought with stability or longevity in mind
- medstrom 3y ago
  LP?
  - yunohn 3y ago
    I assume Lennart Poettering, creator of systemd et all. https://en.m.wikipedia.org/wiki/Lennart_Poettering https://en.m.wikipedia.org/wiki/Lennart_Poettering
  - syslog 3y ago
    My guess is „Lennart Poettering“.
  - deleted 3y ago
    [deleted]
- blueflow 3y ago
  Its acceptable for avahi and pulseaudio because you don't need to use them. With systemd, its a big problem because systemd is widely distributed. Systemd failed on its promise to create a new standard for defining services. Right now a ton of projects ship their own supervisor (runit, supervisord) or docker compose file to reduce their contact surface to systemd. Look what GitLab Omnibus does. But with everything connected to systemd (udev, dbus), anything without systemd is sort of second class in terms of being tested. Ideally i would have the stability of Debian without the surprises from systemd. I tell people to "Press Ctrl+Alt+Del 7 times within two seconds" way too often.
- LtWorf 3y ago
  You know that if it hadn't been a library linked by systemd, it would have just been another library right? Your hastiness to hate on systemd made you forget what we're even talking about…
skywhopper 3y ago
While I agree generally about systemd, this line in particular is not even correct. You don’t have to link libsystemd to run as a systemd service. Using the notify feature is optional anyway, but you can do it without linking to that library.
LtWorf 3y ago
You know they just ran ldd on sshd and picked one of the results as target right? If it wasn't that it'd have been a different one…
kemotep 3y ago
Would an OpenBSD-style minimum functionality Unix philosophy designed init system stop a developer from taking over as maintainer of a different upstream project, allowing them to submit malicious patches?
- jbverschoor 3y ago
  How would you handle dependencies, port, socket, timed triggers etc? 10000s lines of boiler plate code? Very easy to skip something in there. Why do we need to repeat everything? The problem is exactly that.. preprocessing, build scripts etc. Everything is a script and is therefore executable instead of a configuration file/statement which only increases the attack surface
  - kemotep 3y ago
    The grandparent comment was about how the issue was actually systemd. My comment is pushing back against that. As you point out, that isn’t a simple swap out. The replacement to systemd can be as complicated and intertwined as systemd is today if not more. But what init system you use is irrelevant to the issue of what do you do when an upstream project is taken over by a malicious developer?
    - denton-scratch 3y ago
      > The grandparent comment was about how the issue was actually systemd. It's not my view that "the issue was actually systemd". The issue was a complex, sophisticated attack by someone taking a long view. But the attacker injected his code into the system by manipulating part of systemd into loading a fairly obscure compression library that he had hacked. Systemd is of course more-or-less omnipresent these days. Why does an init system need to load libraries at all? Well, it doesn't, unless it has arrogated to itself much more functionality than PID1 should ever have.
      - kemotep 3y ago
        The init system was utilized in a supply chain attack. Fixing the init system helps but is only a bandaid to the issue of supply chain attacks.
jchw 3y ago
That statement is stupid anyway. Firstly, it's not true: most systemd services don't link to libsystemd. It's useful for supporting the sd_notify feature, but plenty of things support it without using libsystemd. Rich attack surface: there was like... four dependencies that aren't glibc, and irrespective of this backdoor attempt they were almost all already being made dlopen dependencies, which would've prevented this backdoor from working since it would've never been triggered for sshd. (Note, also, that linking libsystemd into sshd was a downstream patch.) Seriously: $ libtree $(nix build --print-out-paths nixpkgs\#systemdLibs)/lib/libsystemd.so libsystemd.so.0 └── libcap.so.2 [runpath] That's what you get today in NixOS unstable. The truth is that libsystemd and systemd in general don't have ridiculous dependency trees or attack surfaces. Most likely the big reason why this backdoor was being pushed heavily to try to make it into Debian and Fedora releases was because the clock was ticking on the very small in they had found in the first place. There are a lot of criticisms of systemd that I think are reasonable, but this really isn't one.
- JeremyNT 3y ago
  > (Note, also, that linking libsystemd into sshd was a downstream patch.) I feel like a lot of people are really glossing over this point. What's happened here is that Red Hat and Debian have made a choice to patch OpenSSH in a way that opened it up to this attack. It's a little ironic that e.g. Arch, which actually shipped the malicious code to end users since it publishes changes so much faster, as shipped never would have executed the payload (because they didn't patch OpenSSH).
  - LtWorf 3y ago
    But it's nice when your init knows when a service is ready and can start the other stuff that depends on that one… Otherwise you're in retry hell.
    - jchw 3y ago
      Honestly, I'm a bit curious why sshd startup would ever be significant. Not sure. The most intensive operation I can think of that might happen at sshd startup would be host key generation, but at least in NixOS it looks like this is actually handled in an ExecStartPre= script, which I believe means it will happen before After= units execute. Of course I sincerely doubt that distributions decided to add sd_notify support just for the hell of it, so I am sure there is a reason, it's just not overtly obvious, especially considering plenty of distros don't do this and sshd (and dependent units presumably) certainly seems to work absolutely fine.

himinlomax 3y ago

The backdoored library was linked to libsystemd, not systemd itself.

blkhawk 3y ago
"the car bomb was build into the spare wheel not the car itself...."
- nottorp 3y ago
  "... and although the spare wheel is included by default, the car is modular and you can always remove it yourself" [By dismantling the car completely and reassembling it, since you'd have to rebuild from source...]
  - chgs 3y ago
    Most cars no longer come with a spare wheel, certainly by default. Not sure if that’s because cars don’t get punctures any more due to the great state of the roads here in the U.K, or because people just can’t even change a wheel.
    - nottorp 3y ago
      Not in most systemd distros though. Those include all kinds of spare wheels that automatically take over your actual wheels.
    - JonChesterfield 3y ago
      It's to help meet emissions targets according to the garage I bought from. The spare wheel weighs more than not having one.
deleted 3y ago
[deleted]
inopinatus 3y ago
The dependency is attributable, in the largest part, to systemd's neoplastic aggrandizement of userland infrastructure and associated plumbing, making this a distinction without much of a difference.
- XorNot 3y ago
  "everyone should just reimplement LZMA!" What could possibly go wrong. I'm sure there's no history of compression tools having serious vulnerabilities due to implementation errors...
  - inopinatus 3y ago
    This is another furphy, because OpenSSH proper neither requires nor uses xz/lzma. It's made clear in Andres Freund's original report¹ that the libsystemd dependency dragging it along arises from distros patching openssh to support systemd notifications. The sad part is that systemd notifications are just a datagram on a socket, so using libsystemd for this is reminiscent of Joe Armstrong's banana. [1] https://seclists.org/oss-sec/2024/q1/268 https://seclists.org/oss-sec/2024/q1/268
    - LtWorf 3y ago
      As many have already pointed out, the library can also be linked to sshd via selinux.
      - inopinatus 3y ago
        I've seen that ambit claim too, but I'm not even sure what distro(s) it is referring to since I'm unable to confirm it on any host where I have ldd casually to hand. Ref however https://seclists.org/oss-sec/2024/q1/356 https://seclists.org/oss-sec/2024/q1/356
        deleted 3y ago
        [deleted]

raverbashing 3y ago

Really I get the idea of systemd and I do think that the Unix principles were good for the 70s but maybe not so great today Still in the true LP fashion, a lot of design decisions are not being thought with stability or longevity in mind

medstrom 3y ago
LP?
- yunohn 3y ago
  I assume Lennart Poettering, creator of systemd et all. https://en.m.wikipedia.org/wiki/Lennart_Poettering https://en.m.wikipedia.org/wiki/Lennart_Poettering
- syslog 3y ago
  My guess is „Lennart Poettering“.
- deleted 3y ago
  [deleted]
blueflow 3y ago
Its acceptable for avahi and pulseaudio because you don't need to use them. With systemd, its a big problem because systemd is widely distributed. Systemd failed on its promise to create a new standard for defining services. Right now a ton of projects ship their own supervisor (runit, supervisord) or docker compose file to reduce their contact surface to systemd. Look what GitLab Omnibus does. But with everything connected to systemd (udev, dbus), anything without systemd is sort of second class in terms of being tested. Ideally i would have the stability of Debian without the surprises from systemd. I tell people to "Press Ctrl+Alt+Del 7 times within two seconds" way too often.
LtWorf 3y ago
You know that if it hadn't been a library linked by systemd, it would have just been another library right? Your hastiness to hate on systemd made you forget what we're even talking about…

skywhopper 3y ago

While I agree generally about systemd, this line in particular is not even correct. You don’t have to link libsystemd to run as a systemd service. Using the notify feature is optional anyway, but you can do it without linking to that library.

LtWorf 3y ago

You know they just ran ldd on sshd and picked one of the results as target right? If it wasn't that it'd have been a different one…

kemotep 3y ago

Would an OpenBSD-style minimum functionality Unix philosophy designed init system stop a developer from taking over as maintainer of a different upstream project, allowing them to submit malicious patches?

jbverschoor 3y ago
How would you handle dependencies, port, socket, timed triggers etc? 10000s lines of boiler plate code? Very easy to skip something in there. Why do we need to repeat everything? The problem is exactly that.. preprocessing, build scripts etc. Everything is a script and is therefore executable instead of a configuration file/statement which only increases the attack surface
- kemotep 3y ago
  The grandparent comment was about how the issue was actually systemd. My comment is pushing back against that. As you point out, that isn’t a simple swap out. The replacement to systemd can be as complicated and intertwined as systemd is today if not more. But what init system you use is irrelevant to the issue of what do you do when an upstream project is taken over by a malicious developer?
  - denton-scratch 3y ago
    > The grandparent comment was about how the issue was actually systemd. It's not my view that "the issue was actually systemd". The issue was a complex, sophisticated attack by someone taking a long view. But the attacker injected his code into the system by manipulating part of systemd into loading a fairly obscure compression library that he had hacked. Systemd is of course more-or-less omnipresent these days. Why does an init system need to load libraries at all? Well, it doesn't, unless it has arrogated to itself much more functionality than PID1 should ever have.
    - kemotep 3y ago
      The init system was utilized in a supply chain attack. Fixing the init system helps but is only a bandaid to the issue of supply chain attacks.

jchw 3y ago

That statement is stupid anyway. Firstly, it's not true: most systemd services don't link to libsystemd. It's useful for supporting the sd_notify feature, but plenty of things support it without using libsystemd. Rich attack surface: there was like... four dependencies that aren't glibc, and irrespective of this backdoor attempt they were almost all already being made dlopen dependencies, which would've prevented this backdoor from working since it would've never been triggered for sshd. (Note, also, that linking libsystemd into sshd was a downstream patch.) Seriously: $ libtree $(nix build --print-out-paths nixpkgs\#systemdLibs)/lib/libsystemd.so libsystemd.so.0 └── libcap.so.2 [runpath] That's what you get today in NixOS unstable. The truth is that libsystemd and systemd in general don't have ridiculous dependency trees or attack surfaces. Most likely the big reason why this backdoor was being pushed heavily to try to make it into Debian and Fedora releases was because the clock was ticking on the very small in they had found in the first place. There are a lot of criticisms of systemd that I think are reasonable, but this really isn't one.

JeremyNT 3y ago
> (Note, also, that linking libsystemd into sshd was a downstream patch.) I feel like a lot of people are really glossing over this point. What's happened here is that Red Hat and Debian have made a choice to patch OpenSSH in a way that opened it up to this attack. It's a little ironic that e.g. Arch, which actually shipped the malicious code to end users since it publishes changes so much faster, as shipped never would have executed the payload (because they didn't patch OpenSSH).
- LtWorf 3y ago
  But it's nice when your init knows when a service is ready and can start the other stuff that depends on that one… Otherwise you're in retry hell.
  - jchw 3y ago
    Honestly, I'm a bit curious why sshd startup would ever be significant. Not sure. The most intensive operation I can think of that might happen at sshd startup would be host key generation, but at least in NixOS it looks like this is actually handled in an ExecStartPre= script, which I believe means it will happen before After= units execute. Of course I sincerely doubt that distributions decided to add sd_notify support just for the hell of it, so I am sure there is a reason, it's just not overtly obvious, especially considering plenty of distros don't do this and sshd (and dependent units presumably) certainly seems to work absolutely fine.