6 ms·
Solo – a .so loader for static Linux binaries
- j16sdiz 2mo ago> backed by its own ELF loader (x86-64 and aarch64) and a glibc ABI bridge Yacks
- fabiensanglard 2mo agoPlease elaborate and explain to people with less knowledge why this is bad.
- arjvik 2mo agoMapping parts of files into executable memory, and then executing them, had better be bulletproof! Exploiting this seems like a direct path to RCE, and it's likely that this sort of library is used by privileged code. Purely academically, this is a very cool piece of code! Just hoping that it gets a thorough vetting before used by privileged/security-critical software :)
- pg83 2mo agoWell, ld.so already does this, and it's no big deal. The Python interpreter also does this when executing a .py script (code is code, whether it's machine-readable or human-readable). In any case, we take testing very seriously—every glibc shim we've written is covered with tests, and we run our loader against 1000 of the most popular Debian packages. The project has 100% code coverage. Perhaps, if I have the time, I'll also do some fuzzing on this thing.
- lunixbochs 2mo agosimilar energy to my https://github.com/lunixbochs/crossldso https://github.com/lunixbochs/crossldso
- pg83 2mo agoOh, cool, another prior art I didn't know :)
- mananaysiempre 2mo agoIf you want to load the OpenGL/Vulkan vendor driver then unfortunately you don’t have much of a choice: those are linked against glibc, and I believe generally also against libwayland so screw off if you want a different protocol library (I might be wrong about the latter part). If you instead want to load plugins or whatnot into your statically linked executable, then personally I’d argue that you shouldn’t be emulating Linux dynamic linking semantics at all, because the whole late-bound global namespace thing is silly and wrong. (Solaris, which is where Glibc took this model from, moved away from it[1] as much as compatibility allowed, and so did Darwin[2], whereas Windows never made the mistake to begin with, but Glibc persisted and Musl copied it.) [1] https://www.linker-aliens.org/blogs/rie/entry/direct_binding_now_the_default/ https://www.linker-aliens.org/blogs/rie/entry/direct_binding... [2] https://web.archive.org/web/20011004090044/http://developer.apple.com/techpubs/macosx/ReleaseNotes/TwoLevelNamespaces.html https://web.archive.org/web/20011004090044/http://developer....
- catlifeonmars 2mo agoSo not completely static, since it must link against a libc :P
- pg83 2mo agoThe binary itself is completely static; the link even provides commands on how to check this!
- catlifeonmars 1mo agoAh yeah my mistake static != no external dependencies. I was thinking of golang static binaries (built without CGO) which do not have a dependency on glibc at all.
- nomel 2mo agoI don't know much about musl. > GPU: Vulkan and OpenGL drivers are supplied by the host as shared objects, usually built against glibc, and a fully static musl binary cannot normally dlopen() them. Why? Have people managed to break the ancient concept of shared libraries, and this is a fix for that?
- ranger_danger 2mo agomusl does not perfectly emulate all aspects of glibc, so trying to use libraries that assume glibc can sometimes lead to problems.
- pg83 2mo agoOn the one hand, this is technically true, but on the other, what serious issues do you know that will cause problems in practice? I run tests on 1,000 of the most popular Debian packages.
- okanat 2mo agoBecause glibc and GNU set a terrible precedent. On GNU/Linux systems the shared binary interpreter / loader, GCC compiler, the C library and the system C/C++ ABI all depend into each other. You cannot change any of them independently. All shared libraries depend on the specific glibc version to load them into memory to be able to use that specific glibc version as their C library and make calls like dlopen. Shared libraries have always been broken in Linux. Unfortunately many things like GPU drivers, graphics libraries and NSS need shared libraries to dynamically load certain runtimes (because you don't want to load all possible GPU drivers in existence to your RAM). So an ecosystem has been developed on top of terrible ABI and architecture GNU/glibc provided.
- jeffbee 2mo agoHow are we supposed to take this stuff seriously if the author (sic) isn't even willing to write the readme? Claude exists! If I want some slop I can push the button myself.
- pg83 2mo agoIsn't this a README? https://github.com/pg83/solo/blob/main/README.md https://github.com/pg83/solo/blob/main/README.md
- WD-42 2mo agoParents point is the readme was written by Claude. Signals low effort.
- pg83 2mo agoREADME.md was written by me, and I, of course, used claude/codex for it. In general, I do everything through claude/codex, the reasons are described in https://github.com/pg83/solo/blob/main/CONTRIBUTING.md https://github.com/pg83/solo/blob/main/CONTRIBUTING.md . And no, it's not low effort, and no, I don't see the point in wasting time de-claude-ifying the text just to avoid it looking like I didn't spend enough time on it.
- Barbing 2mo agoI believe usually when someone complains about text written by a language model they are hoping to read human-written text instead of human-laundered LLM output.
- WD-42 2mo agoYou make it clear why you write all your code through a llm. But a README is not code. Presumably you would like people to read it. A machine authored readme reflects poorly on a project.
- 2mo ago
- simonask 2mo agoIt is a testament to the complete failure of the GNU/Linux userland that something like this seems at all attractive to spend time on (or, it seems, LLM tokens). Actually, scratch that, because Windows and macOS have historically struggled with ABI compatibility as well (macOS less so, due to not caring about backward compatibility in the first place). How did we get to the point where people feel they need to go to the length of embedding an ELF loader in their binary (!!) rather than just linking with glibc?
- pg83 2mo agoGlibc has a terrible history of binary incompatibility. If that's so hard to believe, try running binaries built on one distribution on other distributions. Linux has two stable ABIs: the kernel ABI for static programs, and, ironically, WINE.
- vlovich123 2mo agoI haven’t heard of this and I don’t think you’re right. Glibc, for all its faults, as a general rule does backward compatibility well. The problem is if you compile against a newer glibc (common in CI by default) and try to run on a distro with an older (common in the wild). If your CI uses an older glibc you should be fine AFAIK.
- pg83 2mo agohttps://bugzilla.redhat.com/show_bug.cgi?id=638477 https://bugzilla.redhat.com/show_bug.cgi?id=638477 is the most "famous" example. There are also much less well-known "little things" that regularly pop up here and there. > If your CI uses an older glibc you should be fine AFAIK. In any case, my binaries work not only under glibc, but also under Alpine, and (work in progress) under android/bionic.
- vlovich123 2mo agoNot sure what you’re trying to show with that bug report but it’s not a case of cross distro glibc issues. If I read correctly it’s a vanilla behavioral change that exposed preexisting UB in flash. Not sure how the comments about alpine or bionic relate either to my claim that cross distro glibc is fine.
- pg83 2mo agoHow this differs (is better!) from prior art - https://github.com/pg83/solo#how-this-differs-from-prior-work https://github.com/pg83/solo#how-this-differs-from-prior-wor...
- pamcake 2mo agoNote: That md file is LLM spew (like most of rest of codebase). https://github.com/pg83/solo/commits/main/README.md https://github.com/pg83/solo/commits/main/README.md https://github.com/pg83/solo/commits/main/ https://github.com/pg83/solo/commits/main/
- pg83 2mo ago> spew Rude. It only gets better from there - https://github.com/pg83/solo/blob/main/CONTRIBUTING.md https://github.com/pg83/solo/blob/main/CONTRIBUTING.md!
- pamcake 2mo agoDo you belive the machine is taking offence? It can not. Telling me, a coder you never met or interacted with before, that I would introduce more bugs than The Product(tm) is pretty rude and prejudiced. > The project author believes that, with capable human direction, modern LLMs write code faster than people and introduce fewer bugs.
- pg83 2mo ago> Do you belive the machine is taking offence? It can not. Obviously, this is an offense to me. > Telling me, a coder you never met or interacted with before, that I would introduce more bugs than The Product(tm) is pretty rude and prejudiced. Don't exaggerate. I didn't say that you personally introduce more bugs (as you rightly pointed out, I don't know you and don't know how often you introduce bugs), I said that the average developer introduces more bugs than an SOTA LLM.
- 2mo ago
- nubinetwork 2mo agoWhy not just pass the GPU to a docker container?
- pg83 2mo agoAvailable in README - https://github.com/pg83/solo#how-this-differents-from-prior-work https://github.com/pg83/solo#how-this-differents-from-prior-...
- setheron 2mo agoIf you dynamically sold an SO are you still static even if you did it "custom" ? At that point it's a dynamic loader in another name?
- pg83 2mo agoTechnically, you're right, it's a dynamic loader. Technically, it's pure dynamic loading. If we look at the issue at its core, we're still a statically linked program in a hostile environment, forced to dynamically load device drivers from the system. It's similar to Golang; on MacOS, it has to use libSystem, even though otherwise, these are the statically linked Go binaries we're used to and love. Let me add a little more detail: if I use vdso with gettimeofday in a statically linked program on Linux, am I still a statically linked program, or not? :)
- socceroos 2mo agoDo people say "so", "ess-oh" or "dot-ess-oh"? The title "a .so" is clunky to the "ess-oh" gang.
- notpushkin 2mo agoI don’t think I’ve said it out loud more than a couple times in my life. But in general I think I spell out / pronounce the “dot” in file extensions unless it’s completely obvious from context.
- account42 1mo agoWell .so is short for shared object so I think "a .so" is correct.
- valleyer 1mo agoThat's not generally how the English language works. We say "an SoC" but "a system-on-chip".
- eqvinox 2mo agoIf you can figure out your own ELF loader, you can figure out how to build a partially static executable that doesn't need this. You can mix static and dynamic linking. Build tooling around that is just shit.
- pg83 1mo agoIn this scenario, I'll have to choose which libc I want to run. These won't be portable Linux binaries in the true sense of the word; I'll have to leave Alpine out, and possibly Android, which I don't want.
- dwattttt 1mo agoYou don't strictly need to; you can write freestanding C, and use the Linux kernel directly.
- eqvinox 1mo agoFair point - while it's certainly possible to make a decision per individual library elsewhere, libc is the one thing that really will need to be linked dynamically for glibc, and it'll freeze a minimum version into the binary (i.e. it should be built against an old version of glibc). I don't think Android is a target in most cases of this, so this would boil down to shipping 2* binaries, one glibc and one musl. I'd need to check how musl behaves for compatibility, I'm assuming it'll be a minimum like glibc. All of this said - you'll need to ship multiple binaries anyway, these days: x86_64 + aarch64. Possibly more…
- shevy-java 1mo agoCan you really mix it? When I break or remove a shared lib, some binaries no longer work. With static libs or even better, e. g. statically compiled busybox, I don't have that issue, so I disagree on the claim that mixing solves everything as such. I keep the basic toolchain I use as statically compiled variant. The whole system works better if I can break it less easily.
- dwattttt 1mo ago
- comex 1mo agoThis is a big forwards-compatibility risk. Suppose glibc adds a new symbol, and then a GPU driver adds a dependency on that symbol. The user wants to run an old executable with the updated GPU driver (maybe the old GPU driver doesn’t support their GPU). Normally, this would work fine: the user has to use a new copy of glibc, which will be compatible with both the new GPU driver and the old executable. But with your approach, the GPU driver is forced to use the glibc reimplementation which has been statically linked into the executable. Which, since the executable is old, can’t possibly implement the new symbol. The same issue would occur if glibc adds a new version of an existing symbol and then the GPU driver is recompiled. (Or, for that matter, if a GPU driver adds a dependency on a symbol which glibc has always supported but which isn’t in the subset that you reimplemented, though in theory that could be solved if you reimplemented 100% of the symbols.)
- account42 1mo agoYes this is just asking for trouble - and all it does is solve a problem that doesn't really exist. Just dynamically link against the oldest glibc you want to support. Its annoying that Linux toolchains don't have built in easy mode support for that but its much easier to deal with than this thing will be when it breaks. It's also not just new symbols, the loader semantics also aren't static and new enough libraries may not support older semantics - e.g. the loader used to use DT_HASH entries for symbol resolution but now they are no longer present on all distributions.
- yxhuvud 1mo agoThis is not a desirable solution on musl based systems. Whatever you do in that situation ends up horrible, so it is about finding the least bad solution. Which this seems like a workable variant of.
- egorfine 1mo ago> Just dynamically link against the oldest glibc you want to support I wish it would that simple for practical use cases. I ship professional software for colorists for Hollywood studios and they absolutely love to never upgrade. We have to ship for RockyLinux 8. Sad.
- pjmlp 1mo agoSo we are re-inventing patched a.out files, back when UNIX systems started to introduce dynamic loading, before ELF was invented? Advocates of static linking keep forgetting once upon a time UNIX only had static linking, then we had overlays, and eventually dynamic linking came to be.
- pg83 1mo agoOf course, I remember those times very well. And I also remember very well that dynamic linking appeared ONLY because we were catastrophically short on memory; everything else was added much later. Now we have plenty of memory, and we can very well return to our blessed roots!
- pjmlp 1mo agoNot at all, the pain of doing plugins with UNIX IPC was another one. Ah, you have lots of memory, we're very wealthy. /s
- mohamedkoubaa 1mo agoThere is no RAM shortage in Ba-Sing-Se
- rfgplk 1mo agoI've implemented the same thing for micron (more or less). One advice I'd give you is to _really_ take care regarding SysV/ELF ABI conventions, there's tons of undocumented stuff in there and it's really easy to mess something up or cause a security defect (see AT_SECURE). That being said the way you're doing is also tricky(ish) because if I understood your implementation correctly you're hooking this into an already running musl which could cause backwards compatibility issues if musl changes under you. Doing this is safer if you control the entire runtime.
- pg83 1mo ago> One advice I'd give you is to _really_ take care regarding SysV/ELF ABI conventions, there's tons of undocumented stuff in there and it's really easy to mess something up or cause a security defect (see AT_SECURE) Yes, it's not trivial, but I hope that over time everything will settle down. > if I understood your implementation correctly you're hooking this into an already running musl which could cause backwards compatibility issues if musl changes under you No, I hook this to musl, which is statically linked into my binary, and I have complete control over it.
- sieve 1mo agoEvery couple of years, I revisit my PL dev hobby and this time I decided to create a language/runtime with pre-emptive scheduling using instruction fuel. While I always do freestanding builds, this time I decided that I also wanted to support native FFI. That is when I realized the true horror of (g)libc. It wants to inject itself at the root of the library/program and everything from threading to dlopen/dlsym is impacted. I tried a lot of workarounds including trying to implement a loader myself, but the complexity (and fragility) grew so much that I felt it was not worth it. Finally, I retreated into the safe world of a freestanding runtime + syscalls. FFI, if it has to happen, will occur via IPC of some kind. A second process linked against glibc that will manage calls on behalf of the clean first one.
- torginus 1mo agoI have faced a similar issue in the past, and I don't understand how static binaries from the host are supposed to solve this. From what I remember, GPU access on Linux 'works' by accessing specific FDs under /dev, which are vendor specific - this is what these libs do under the hood. The libraries don't have any magic powers - if the FD is inaccessible, you won't be able to do anything. So there's some vendor specific access needed in containers anyway (or a blanket allow, which is a BAD idea). Also not sure why dynamic linking isn't good enough for this - the issue lies with the permissions, not how you load/link libraries.
- snarfy 1mo agoYou normally do this sort of thing with plan old `dlopen` and `dlsym` and make your own "plugin" system.
- ok123456 1mo agoIs the 'hello.jpg' reference intentional?
- colinsane 1mo agoi'm not sure how much people realize that the modern "graphics driver" is actually just "the kernel multiplexes userspace messages to/from the GPU and we've taught mesa to understand each family of GPU you'd ever care to support." i helped somebody get Doom running on an old embedded system running some 4.x kernel. we just built everything, including mesa, statically and deployed that. imagine building everything static but loading the system's libgl.so instead (which was probably just a symlink or abstraction over mesa's own implementation). what's the benefit: we'd get older, less optimized graphics routines from 5 years ago? if you're statically linking, then just bring your own graphics "driver". the kernel interfaces are stable enough. it's not conceptually different than embedding `syscall`s directly into your application the same way you do when statically linking libc.
- pg83 1mo ago> i'm not sure how much people realize that the modern "graphics driver" is actually just "the kernel multiplexes userspace messages to/from the GPU and we've taught mesa to understand each family of GPU you'd ever care to support." In the case of OpenGL, it has a fairly sophisticated state tracker, which you can carry around with you. For example, I can statically compile ANGLE this way. But even OpenGL/Vulkan have a shader compiler that's unique to the hardware (it's expensive to carry it around with every piece of hardware in the universe). And I'm not even talking about libdrm, which is quite tightly tied to the kernel version. I know exactly what I'm talking about, because I can also statically compile Mesa into my application.
- mochaa 1mo agoif you really want a single elf that could link to system libraries with a foreign libc (you shouldn't), the somehow correct-ish way to do it is map your preferred ldso manually , setup stack, call DT_ENTRY etc etc like how kernel does it, and just yank the libdl symbols. see cosmo_dlopen for an example implementation.
- pg83 1mo agoThe README.md link describes cosmopolitan's approach, among other things, and it's worse than mine.
- syngrog66 1mo agoClaude-ware
- stlahxm 1mo ago[dead]