3 ms·
I wouldn't do it this way, within the toolchain. Better to just append the data to the finished executable, with a header that contains an identifying marker. T
by kazinator 7y ago
I wouldn't do it this way, within the toolchain. Better to just append the data to the finished executable, with a header that contains an identifying marker. The program can scan its own image and look for that marker to get at the data. That will work on any OS with any executable and linker format and can be done post-production (users or downstream distributors can receive the binary and add customized data to it without requiring a dev environment with a toolchain, just tiny utility you can bundle with the program).
Scanning for the marker can be avoided, if we do the following:
/* inside the program, at file scope */
struct {
char marker[16] = "dW5pcXVlbWFyawo="
uint32 offset;
} binary_stuff;
Then your tiny utility programs opens the executable and looks for the sequence "dW5pcXVlbWFyawo=" inside it. Having found it, it puts the offset of the data into the 32 bit offset field which follows, and writes the data at that offset.
When the program runs, if it sees a nonzero value in binary_stuff.offset, it can open its image and directly proceed to that offset to read the stuff or map it: no searching.
- tantalor 7y agoSounds like a great vector for malware.
- kazinator 7y agoAny unsigned native executable is a vector for malware. Images with stuff appended to them can be signed. For instance, think of signed Linux kernel images on secure embedded systems. The signature includes the embedded initramfs and device tree blob.
- mwcampbell 7y agoThe advantage of the approach described in the article is that there's less work to do at startup time. For applications that run on end-users' machines, we should do whatever we can to minimze work done at startup, getting it as close as possible to just mapping a big blob into memory and jumping to the code that does the real work of the program.
- kazinator 7y ago> For applications that run on end-users' machines, we should do whatever we can to minimze work done at startup. That is not an absolute given. If this is some tool that is launched a large number of times from frequently running shell scripts, or from a busy web server's CGI, I'd tend to agree. The principle you are championing is not widely followed anyway. All sorts of applications that people use on their machines on a daily basis have atrocious startup times. Though "everyone's startup time sucks" is no excuse, it does remove the motivation to worry about single digit millisecond differences. With my technique we don't actually have to do anything at all until the blob is required. If there are ways of executing the program that don't require the data, that initialization need not even take place. With lazy initialization we can move the cost out of the startup time. Even if you work at minimizing startup time, there is a lot of activity in a modern application, like attaching shared libraries and opening various files. Contribution from this approach is a like rounding error in the least significant digit of startup time compared to the default stuff that happens before your program even gains control.
- marcan_42 7y agoThis isn't portable. Executable files aren't binary blobs, they're a structured format. If you append data to an executable there is no guarantee it's actually going to end up mapped in memory for you. You'd have to put it into an actual ELF segment, and at that point you're back to using the linker. It's a lot more sensible to ask the linker to do this as in OP than to hack together something like you've described.
- kazinator 7y ago> If you append data to an executable there is no guarantee it's actually going to end up mapped in memory for you. I didn't state it clearly enough, but I didn't say anything about it being mapped. It almost certainly isn't mapped. Loaders do not blindly map the whole thing to memory; then you would end up with debug info unconditionally mapped. Even if it were mapped, the program wouldn't easily find it with the latter approach I described: the offset given in the structure is measured from the start of the file, not from some address in memory. > ask the linker It is not portable either. For instance, it doesn't work with Microsoft's linker which is called link.exe.
- asveikau 7y agoNo need to scan the image if the last few bytes has a "footer" describing the length that precedes it. I still think I like adding it as a section better, though.
- kazinator 7y agoThe footer could work. If you round up the file to some reasonable power-of-two block size and put the footer right at the end of the last block, that would be highly portable; it would work on systems/devices that only write binary files in multiples of a block size (even if you write a short block at the end).
- gmueckl 7y agoYou should check out how self-extracting archives work. All of those that I am aware of have a first part that is a regular executable and the archive to ve extracted is appended to that. File formats like zip have an advantage here as they keep their index at the end of the file after the compressed data. So there is no need to scan for the start of the archive.
- claytongulick 7y agoIIRC this is how nwjs works too.