6 ms·
PatchELF: Simple utility for modifying existing ELF executables and libraries
- phissenschaft 6y agoSuper useful for updating rpath!
- kelvie 6y agohttps://nixos.org/guides/nix-pills/automatic-runtime-dependencies.html#idm140737320199808 https://nixos.org/guides/nix-pills/automatic-runtime-depende... Part of the excellent "nix pills" guide on why things are the way they are in Nix, and why this tool was created. The whole series is a pretty easy evening read if you have experience in functional languages (like haskell), and really helps you appreciate what nix is doing. One of my main reservations about Nix and introducing it at work is basically requiring that other developers wrap their minds around functional concepts like partial application. This might just be an imagined problem, but not one I want to spend cycles explaining to the higher-ups
- mikepurvis 6y agoI'm working on it now at my work, partially following some advocacy in the last Nix thread on here a month or so back. I think the biggest barrier for me is that you can't really be only partially in— like, you sort of can, but you lose a lot of the benefits if you have impure builds linking to a bunch of filesystem stuff. So yeah, you need sufficient buy-in that you can spend the effort required to basically port your entire system to a new operating system and packaging scheme. And depending how big your system is, that might be a lot of work that has to happen upfront before any real value is delivered.
- rsa25519 6y agoYou can be partially in :-) I highly recommend using the Nix package manager alongside whatever you're comfortable with. That way you can `nix shell -p foobar` when you need a package quickly or fallback to brew/apt/etc if you're not yet comfortable addressing the situation in Nix.
- NieDzejkob 6y agoApart from adapting proprietary or hard-to-compile software to non-FHS distros like NixOS or Guix, what are the usecases of patchelf?
- dannyz 6y agoLots of Python packages containing native code are optionally distributed with pre-compiled libraries that have been modified with patchelf. I believe both auditwheel (https://pypi.org/project/auditwheel/ https://pypi.org/project/auditwheel/) and conda-build do this.
- dimator 6y agoYes. Auditwheel inspects the native python parts and shoves any external .so dependencies into the wheel, making it hermetic.
- eikenberry 6y agoSounds like something that would make binary patching easier. Lack of simple binary patching is one of the key bits missing that would make switching to static binaries more viable. Currently one of the big upsides to dynamic libraries is the bandwidth they save in distribution where binary patches would go a long way to addressing this.
- dathinab 6y agoNeeding a LD_PRELOAD on a system where LD_PRELOAD is blocked. Sure it still requires you to have rights to create+run a executable. But it's a much better model then LD_PRELOAD as it can only be used on executable you can write to, which normally means you your user/group but not other user/group owned executables. Which is especially relevant with suid/sgid. You can also use this, to patch a library which is compiled in a way where LD_PRELOAD doesn't work. Like suid binaries. Through you have to potentially make a copy and set the suid bit again.
- 3v1n0 6y agoYeah, I've been using it to set rpath on some files that needed to be packaged in a particular way...
- mkhnews 6y agoThank you for this great tool.
- stabbles 6y agoPatchelf is almost always broken, and when they merge fixes they don't release a new version right away. Between 2014-2019 running patchelf+strip would corrupt binaries, after the fix running patchelf twice on the same binary would corrupt it. And now patchelf doesn't work on all binaries, there's a bugfix merged https://github.com/NixOS/patchelf/pull/230 https://github.com/NixOS/patchelf/pull/230 merged Nov 19, 2020, but no release since then. Also patchelf exposed a bug in ldconfig noted 11 years ago and only fixed in glibc 2.31: https://nix-dev.science.uu.narkive.com/q6Ww5fyO/ldconfig-problem-with-patchelf-and-64-bit-libs https://nix-dev.science.uu.narkive.com/q6Ww5fyO/ldconfig-pro... And currently patchelf still has a bug: https://github.com/NixOS/patchelf/pull/275 https://github.com/NixOS/patchelf/pull/275 Apparently elf files are hard.
- jchw 6y agoTo be fair, patching existing binaries isn’t all that easy to begin with. Once you need to start moving around structures, a lot of bets are off. In case of NixOS, I’m sure the long nix store paths added often overflow structures and require workarounds to avoid breaking images. Hacking around Win32 PE images, I’ve run into a lot of tricky subtle issues over time...
- stabbles 6y agoYeah, it wasn't meant to criticize the project as a whole; I'm happy that at least someone took on the project of "growing" the rpath section... My only point of critique is they should immediately tag a new version after a bug fix, cause many distro's have buggy versions of patchelf.
- ornxka 6y agoI'm not really sure why it has to be that hard, like, why don't we just use base64-encoded JSON or something?
- KMag 6y ago> why don't we just use base64-encoded JSON or something? Base64 would be counterproductive, increasing both space and parsing time... presumably out of fear that JSON would contain a naughty byte for a greenfield file format. It would be much better to just design tho format to not have any naughty bytes. Parsing time for exacutables and libraries is definitely on the critical startup path. You really want a length-delimited format, or better yet, one where offsets to various structures are stored at fixed offsets so you can find everything in O(1) time with a tiny constant factor. Compiler writers and tool authors are perfectly comfortable working with binary file formats. There's nothing more inherently future-compatible about JSON than a forward-compatible binary format like flatbuffers. Having to escape and then unescape naughty bytes is a huge downside for text-based formats that are hardly ever read by humans. On a side note, Zlib DEFLATE / gzip / LZMA etc. aren't magic for getting rid of space overheads. Try gzip -9'ing your system's wordlist, now convert it to UTF-16 and gzip -9'ing it. You'll see a several percentage increase in size, despite an entropy change of at most a constant and small number of bits (-log2(P(UTF-16)/P(UTF-8)). I've frequently seen huge JSON proponents use hand-wavy arguments that gzip will reduce any size differences to zero. It's also nice if the file format is very close to being able to just be mmap()ed into the process's address space and only require minimal patching to a minimum number of pages in order to be an optimized in-memory representation. Also, there's a huge amount of momentum behind executable formats. Incremental improvements by adding new features in new segment types or appending new fields to old data structures (where there's no ambiguity) is much preferred to wholly new formats. So, creating a new debugging symbol section that's just flatbuffers is workable. Replacing the whole ecosystem with base64-'d JSON would have way more downsides than upsides. On another side note, you need to be very careful with JSONifying floating point values. Many libraries don't give you bit-perfect round-tripping of IEEE-754 double precision values.
- jbangert 6y agoI built a (research) library a few years ago to rewrite ELF binaries; our research projects ran into a lot of limitations with doing incremental patches to a binary (ELF has a lot of redundant representations of the same data). For us, parsing the binary into a normalized representation, modifying that, and re-serializing worked — we could make more intrusive changes to the binary, and (almost? I don’t recall anything breaking) everything in the Debian repos still ran after the binaries has been rewritten. I expect the library is now woefully out of date, and documentation is mostly in the form of conference talk slides: https://github.com/jbangert/mithril https://github.com/jbangert/mithril there’s also https://github.com/aclements/libelfin https://github.com/aclements/libelfin (parsing only, supports dwarf); https://github.com/bx/elf-bf-tools https://github.com/bx/elf-bf-tools (Turing machine inside elf relocations) and of course the “olg guard” of ELF reversing tools ERESI/elfsh (website seems down; GitHub mirror on https://github.com/thorkill/eresi https://github.com/thorkill/eresi).
- chazeon 6y agoProbably one or two years ago I randomly met this wonderful tool amidst dealing with a VSCode Remote SSH problem via search. The story begins with using VSCode remote SSH on a HPC that is running CentOS 6. VSCode Remote SSH ship with a Node binary that is dynamically linked to a glibc that is not supported on older OSes such as CentOS 6 https://code.visualstudio.com/docs/remote/linux https://code.visualstudio.com/docs/remote/linux . I am not the system manager on the HPC and could not update the system myself, and VSCode team is too arrogant to support those old OSes. Alhough I have a homebrew environment and has a new glibc built there, the Node VSCode does not recognize that from environment variable. Manually update string in the node binary’s strings does provide me with solution at least: https://github.com/microsoft/vscode-remote-release/issues/103#issuecomment-552593089 https://github.com/microsoft/vscode-remote-release/issues/10... As a researcher I am really thankful of the developer who created this wonderful tool.
- chazeon 6y agoAlso another reason I have to come up with this is though VSCode is open-sourced, VSCode’s remote SSH extension is private. I had to look into the extension module’s uglified source code to figure out what is happening out there… Alternative solution would involve edit the uglified js file https://github.com/microsoft/vscode-remote-release/issues/103#issuecomment-552167999 https://github.com/microsoft/vscode-remote-release/issues/10...
- c0deR3D 6y agoBTW, is there any recommended open source dynamic binary patching tool? Have been using Intel Pin, but fine-grained patches would incur noticeable performance regression permanently. Thanks!
- gnufx 6y agoI don't know exactly what Pin does, but probably dyninst, which also supports Arm8 and POWER. It won't avoid slowdown, though.