4 ms·

I tried to learn this arcane art and use it in order to do some special things. I failed miserably. Long story short, I wanted to append some null program heade

I tried to learn this arcane art and use it in order to do some special things. I failed miserably. Long story short, I wanted to append some null program header entries to the otherwise standard linker output so that my tool could easily target them for patching. Discussed this a bit on GNU mailing lists but this approach just didn't seem to be going anywhere. https://sourceware.org/pipermail/binutils/2023-November/130532.html https://sourceware.org/pipermail/binutils/2023-November/1305... In the end, I just bypassed the linker script. I wrote a dedicated ELF patching tool that does what I want, and suggested a linker feature to let users add spare program headers via command line option. The maintainer of the mold linker quickly accepted the feature request. The maintainers of GNU ld apparently didn't. The maintainers of LLVM lld probably never even saw it. https://sourceware.org/pipermail/binutils/2023-November/130640.html https://sourceware.org/pipermail/binutils/2023-November/1306... To this day I wonder if it's possible to do this via linker script. I wonder if the author of this article would know.

ChickeNES 2y ago
Honestly for your use case, what if you made the interpreter executable a ZIP archive instead? Then any zip archiver could add new files/programs.
- matheusmoreira 2y ago
  Many people suggested that I do that. I believe it's the standard way to embed files into executables. Open the executable and read backwards from the end until the zip archive metadata is found. My goal was to do it in such a way that the Linux kernel memory mapped the embedded file section before the process had even started running. I explicitly wanted to avoid doing things like opening /proc/self/exe and reading its contents into memory. My interpreter is freestanding and has zero dependencies. This solution fits the minimalist spirit of the thing. I wrote an article detailing how it works: https://www.matheusmoreira.com/articles/self-contained-lone-lisp-applications https://www.matheusmoreira.com/articles/self-contained-lone-... Long story short: Linux passes a pointer to the ELF program header table to the process via the auxiliary vector which means the program segments are already reachable from inside the program, and I took advantage of that by adding custom load segments and making the interpreter search for them at runtime. This little mechanism allows embedding arbitrary data, including zip files. I made a little lisp S-expression file format which the interpreter simply parses but it could have been anything. Zip archivers should be able to edit the data in place, it should work fine provided the segments are updated with the new size of the zip archive. The practical application of this is to enable the creation of a freestanding, self-executing application by making a copy of an existing interpreter and adding the desired code and data to it, even though the interpreter has already been compiled and linked: cp lone my-app lone-embed my-app my-app.ln ./my-app The interpreter introspects, discovers there's an embedded segment inside itself and evaluates it. Unfortunately I had to write a tool to append those segments. Never figured out how to do it with linker script.

ChickeNES 2y ago

Honestly for your use case, what if you made the interpreter executable a ZIP archive instead? Then any zip archiver could add new files/programs.

matheusmoreira 2y ago
Many people suggested that I do that. I believe it's the standard way to embed files into executables. Open the executable and read backwards from the end until the zip archive metadata is found. My goal was to do it in such a way that the Linux kernel memory mapped the embedded file section before the process had even started running. I explicitly wanted to avoid doing things like opening /proc/self/exe and reading its contents into memory. My interpreter is freestanding and has zero dependencies. This solution fits the minimalist spirit of the thing. I wrote an article detailing how it works: https://www.matheusmoreira.com/articles/self-contained-lone-lisp-applications https://www.matheusmoreira.com/articles/self-contained-lone-... Long story short: Linux passes a pointer to the ELF program header table to the process via the auxiliary vector which means the program segments are already reachable from inside the program, and I took advantage of that by adding custom load segments and making the interpreter search for them at runtime. This little mechanism allows embedding arbitrary data, including zip files. I made a little lisp S-expression file format which the interpreter simply parses but it could have been anything. Zip archivers should be able to edit the data in place, it should work fine provided the segments are updated with the new size of the zip archive. The practical application of this is to enable the creation of a freestanding, self-executing application by making a copy of an existing interpreter and adding the desired code and data to it, even though the interpreter has already been compiled and linked: cp lone my-app lone-embed my-app my-app.ln ./my-app The interpreter introspects, discovers there's an embedded segment inside itself and evaluates it. Unfortunately I had to write a tool to append those segments. Never figured out how to do it with linker script.