7 ms·
Systemd replacing ELF dependencies with dlopen
- rwmj 3y agoThe interesting part is: https://github.com/systemd/systemd/pull/32234 https://github.com/systemd/systemd/pull/32234
- AlotOfReading 3y agoIt's nice to see a proposal that fits so neatly within existing infrastructure and norms. I just hope they either standardize or drop the extensible type field idea instead of letting it turn into a semi-documented mess other projects can't easily adopt.
- stefan_ 3y agoIf they are "optional", why do initrd generators need to know them? This truly has all the hallmarks of a systemd "solution"..
- Denvercoder9 3y agoA dependency being optional on a technical level doesn't mean that it's optional on a functional level for everyone. `libfido2` mentioned in the pull request is a great example: if your rootdisk is encrypted using a FIDO token, you need that library in the initrd; but otherwise you don't. By making the dependency optional and initrd generators aware of that optional dependency, people that need it will have things just work, while others don't need to install the library.
- deleted 3y ago[deleted]
- kelnos 3y agoYou know that you use a token to unlock your boot device. You don't necessarily know the specific set of libraries needed to enable the feature, though.
- Spivak 3y agoBecause the optional dependencies actually do things. By default an initrd-builder would probably copy all deps including optional ones but it now enables you to choose not to do that as well, an avenue that was previously not available.
- kragen 3y ago[flagged]
- naasking 3y agoWhat's the conflict? Microsoft now ships a Linux subsystem that's used by Docker for Windows. There's convergent interests here.
- kragen 3y agoi'm not suggesting we try to stop microsoft from using linux, but they certainly shouldn't be making key architectural decisions
- naasking 3y agoIs there evidence that any Microsoft execs are pressuring Poettering? Wouldn't other people that work on systemd and linux object to poor architectural decisions? This is a key feature of open source development, it's just not so easily subverted.
- sounds 3y agoI mean, Microsoft pays him a regular paycheck. I think you're moving the goalpost quite a bit to ask that the community accept anyone's contributions until solid proof of malicious intent is shown. Are you prepared to accept my contribution? I sign my name "JiaT57notThatJiaTan"
- naasking 3y agoI'm not moving any goalposts, you're saying someone with decades of open source kernel development and a solid reputation is no longer trustworthy because of who now signs that paycheque. That's just nonsense, and has nothing to do with the scenario you posed.
- 3y ago
- turminal 3y agoI don't understand the initial motivation for converting regular dynamic library dependencies to dlopen dependencies. How does that help with reducing the footprint?
- ongy 3y agodynloading libraries can have some side effects. Which the recent xz attack used to mess with `sshd`, even if it never actually used functions from the library. dlopen loading only loads the library *when it's actually used*, and doesn't include everything and the kitchen sink on startup. If the libraries are used seldomly enough, it might also allow to make more dependencies optional in package management.
- marcosdumay 3y agoSo... I repeat the GP. What you gain is that the vulnerabilities will be harder to track down?
- ongy 3y agoThe specific way sshd was infected would not have happened with libxz as dlopen library. Debian's sshd only uses libsystemd for the notify api. I.e. it doesn't need any feature that uses libxz. If it's dlopen()ed, it does not need to be loaded into the process context to use an unrelated feature. FWIW, IMO upstream systemd should split their monolithic library and allow users to pick better that way, but this has other implications on DX.
- newpavlov 3y agosshd should not have used libsystemd in the first place for the trivial notification. And the ifunc stuff is its own security nightmare. Papering over this by dlopen-ing some libs in libsystemd does not address the deeper issues.
- NewJazz 3y agoFWIW, IMO upstream systemd should split their monolithic library and allow users to pick better that way, but this has other implications on DX. They've done the exact opposite afaict. Libsysd used to be split up, now it is monolithic.
- blueflow 3y agoAt least Poettering acknowledged that Systemd's dependency creep is a problem. [edited out]
- Starlevel004 3y agoThe PR for changing compression libraries to use dlopen() was opened several weeks before the xz-utils backdoor was revealed. https://github.com/systemd/systemd/pull/31550 https://github.com/systemd/systemd/pull/31550
- blueflow 3y agoThen its good the systemd team changed their minds even if its a decade late.
- cratermoon 3y ago"Jia Tan" was pushing hard to get systemd updated to use the new xz because he saw this change in progress and wanted to get it in a release before this went through.
- Denvercoder9 3y agoDo you have a source for that? The backdoor was added to a minor release without a SONAME change, and generally such updates are done by distributions and not something upstreams concern themselves with.
- panick21_ 3y agoIt was mentioned in the recent Oxide and Friends podcast with the guy who, discovered the backdoor. Maybe there is a link there somewhere.
- Denvercoder9 3y agoI'm not going to listen to a 1.5 hour podcast to find the exact quote, but someone must've misspoke or misunderstood. I did some searching in the meantime, but there's no evidence at all of "Jia Tan" interacting with the systemd developers. They pressured distributions to update xz-utils, but not systemd upstream.
- fsniper 3y agoWouldn't dlopen make you more vulnerable to runtime library attacks? Instead of having dependencies defined at the build/start up time, they will be loaded at runtime on demand. So the dependency library could be changed at any time and you wouldn't have any idea what code you are loading at this time? So upgrading XZ could even attack already running sshd too? I would be more inclined to have static linked libraries instead of having lazy loading of them for preventing this kind of attacks..
- rcxdude 3y agoThis seems a bit nitpicky: you just move the threshold for where you might load a backdoored library from first exec to first use which dlopens that library (which might even just be at exec anyway, I haven't checked the source to see how systemd handles it). I think the focus on the specific means that the xz backdoor used to get into sshd is misdirected: it's code that is run in all kinds of priviledged contexts and has thousands of different ways to turn that into RCE. If sshd didn't load it directly the developers of the backdoor would have simply taken a different route.
- wakawaka28 3y agoLoading at runtime is a feature. You're supposed to lock down the environment for execution so the right libraries get used. The trouble with statically linking is that you have to relink everything to update. You also can't easily audit which versions are in a binary. It seems appropriate for some small userspace programs but not for complex software like services with many dependencies.
- jcranmer 3y ago> So upgrading XZ could even attack already running sshd too? With the specifics of how the xz injection attack worked, actually it wouldn't. libsystemd would have been loaded to implement the service-started notification, and then there is no reason to use anything further and therefore nothing to trigger the loading of the xz library. Which is kind of the point of this change: the goal is to load libraries on-demand instead of eagerly, so if there is nothing demanding the library load, the library is never loaded.
- arp242 3y agoTo be honest, liblzma using systemd seems like an unimportant detail more than anything else. On my laptop there's 141 binaries in /bin/ that link against lzma; an abbreviated list with similar entries omitted below. And that's a fairly minimal Void Linux system – the list for an average Ubuntu system is probably a lot longer. Sneaking in via kmod, grub-install, udevd, or xbps-install seems just as viable as sneaking in via libsystemd. Perhaps a little bit harder, but not much. It will remain problem as long as you have a single command that runs your malicious code as root. Being directly linked against sshd was convenient, but there's all sorts of devious shit you can pull once you've got root, and it doesn't take that much more effort. % for f in /bin/*(*); ldd $f 2>&1 | grep -q lzma && print $f /bin/amdgpu-arch /bin/bsdcat /bin/bsdcpio /bin/bsdtar /bin/bsdunzip /bin/c-index-test /bin/clang-* /bin/clangd /bin/diagtool /bin/ffmpeg /bin/ffplay /bin/ffprobe /bin/find-all-symbols /bin/grub-editenv /bin/grub-install /bin/grub-mkimage /bin/grub-mknetdir /bin/grub-mkrescue /bin/grub-mkstandalone /bin/kmod /bin/lspci /bin/modularize /bin/mpv /bin/nvptx-arch /bin/pp-trace /bin/pskctool /bin/qemu-* /bin/rtorrent /bin/tiffcp /bin/tiffdump /bin/tiffinfo /bin/tiffset /bin/tiffsplit /bin/udevadm /bin/udevd /bin/update-mime-database /bin/xbps-alternatives /bin/xbps-checkvers /bin/xbps-create /bin/xbps-dgraph /bin/xbps-digest /bin/xbps-fbulk /bin/xbps-fetch /bin/xbps-install /bin/xbps-pkgdb /bin/xbps-query /bin/xbps-reconfigure /bin/xbps-remove /bin/xbps-rindex /bin/xbps-uchroot /bin/xbps-uhelper /bin/xbps-uunshare /bin/xmlcatalog /bin/xmllint /bin/xsltproc /bin/zstd
- jwilk 3y agoThey are doing dlopen() wrong. Instead of dlopen()ing libfoo.so.7, you should build a private wrapper library, normally linked with -lfoo, and then dlopen() that private wrapper. Then you get dependency tracking for free.
- deleted 3y ago[deleted]
- Denvercoder9 3y agoFor some definition of free, that includes writing and maintaining a private wrapper library. This seems a lot like a workaround for our tools not supporting optional dependencies. Let's fix that instead.
- edflsafoiewq 3y agoUhh, how does that get you dependency tracking?
- KerrAvon 3y agoThis seems like kind of a janky workaround when what you really want is linker support for weak/on-demand linking.
- mike_hearn 3y agoRight, as Lennart points out macOS has supported this for years. And adding a DT_WEAK_NEEDED to ELF wouldn't be that hard. The glibc maintainers even work for Red Hat. Feels like systemd is the wrong place to solve this.
- gumby 3y agoI thought mastodon supported long posts. I thought breaking these things into a chain of tiny links was a twitter pathology.
- tristan957 3y agoIt's configurable by the instance maintainers.
- gumby 3y agoThanks. That awful thread would have made a fine blog post. I can’t imagine why anyone would switch that misfeature on.
- amluto 3y agoSigh. On the one hand, this seems like a slight improvement. On the other hand, while it’s a bit of a slog, it’s possible to actually sandbox dependencies. libxz, for example, has well defined inputs and outputs. It’s possible to load it in such a way that all it can do is consume inputs and produce outputs that depend on those inputs. (And allocate memory and waste CPU, but that’s DoS at worst.)
- 10000truths 3y agoA shared object that is loaded in the same thread/process as the rest of your application is not sandboxed from your application.
- amluto 3y agoThat's true. But both in-process and out-of-process sandboxing are doable, as is writing code in a manner that is provably (for various definitions of provability) free of various classes of side effects.
- ptx 3y agoIn can be with RLBox[0], which compiles the code to WebAssembly and then back to C. It's used in Firefox to sandbox various format parsing libraries. [0] https://rlbox.dev/ https://rlbox.dev/
- quotemstr 3y agoWe could just implement a DT_NEEDED_WEAK already instead of using these awful hacks.
- thesnide 3y agowhat i do miss is a libsystemd-notify. Doing dlopen() is just fixing the wrong prob. I agree that a simple self-contained .h header file would solve it in a better way. But I stopped trying to push for minimalism. Sadly.