3 ms·

It is definitely not going to be easy to do for every CPU architecture, but it's not as ridiculous as it seems. Basically, the difference between an "object fil

by jchw 2y ago

It is definitely not going to be easy to do for every CPU architecture, but it's not as ridiculous as it seems. Basically, the difference between an "object file" and an executable file or shared library is not very large. Often times, except for Microsoft platforms, they are in the same actual file format, e.g. ELF. The big thing is relocations. Object files have granular relocations that executable files don't; at least on Windows, executable images just have minimal relocations that point to addresses of code and data that will need to be fixed up if the executable is relocated, but the executable image itself is only able to be relocated with regards to its image base address as a whole unit. In contrast, object files contain symbol-level relocations. To be able to accurately reconstruct this information, you need to annotate the disassembly with somewhat accurate information about symbols. The other big difference with an object file is well, it is not linked. None of the symbols are "resolved". This is particularly easy to fix actually: during delinking if the symbol is outside of the current scope then it just needs to be replaced with an unresolved symbol, pretty much. Then when relinking, another object file or library needs to provide the symbol so that it can be linked back up. (There are a few smaller differences too, like the lack of an entrypoint, but it really isn't a whole lot of significance.) This means that the boundaries for which you carve object files out of an executable image or shared object is actually completely arbitrary. It obviously was segmented into translation units when it was originally compiled, but nothing really cares about those boundaries at the linking stage. (Of course, you probably want to try to figure it out if possible, since it will probably be very hard to do a matching decompilation if your object boundaries are incorrect.) Take this with a grain of salt as despite having literally worked on this problem I feel like I might be messing up some of the details a bit. I've been meaning to write a blog post about object files, though there are actually a couple of good ones already floating around.

fngjdflmdflg 2y ago
>It obviously was segmented into translation units when it was originally compiled, but nothing really cares about those boundaries at the linking stage. (Of course, you probably want to try to figure it out if possible, since it will probably be very hard to do a matching decompilation if your object boundaries are incorrect.) Can you expound on this? Why are the boundaries not important at the linking stage - aren't you linking the wrong code then? Or did I not understand the point here?
- boricj 2y ago
  The boundaries aren't important during delinking, as long as you do not cut across a variable or function. To the linker, sections are just arrays of bytes. It doesn't care what's inside those, all it has to do is lay them out in memory and apply relocations. How you decide to slice up the original program is up to you. It doesn't have to follow the boundaries from the original object files.
skissane 2y ago
> Often times, except for Microsoft platforms, they are in the same actual file format, e.g. ELF. On 32-bit and 64-bit Windows (both NT and 9x), EXE, DLL and OBJ files are all COFF. EXE/DLL files stick an MS-DOS stub and the "PE\0\0" header before the COFF header, OBJ files start with the COFF header directly. This is different from DOS, 16-bit Windows and OS/2, where the standard object file format (Intel OMF) was completely unrelated to the executable formats (MZ for DOS; NE for 16-bit Windows and OS/2 1.x)
- jchw 2y ago
  Interestingly, I recently tried to dive in and figure out why exactly Microsoft went with COFF. It seems like Intel OMF was adapted all the way up through OMF386, and Microsoft switched to COFF around 1989, near the beginning of the NT OS/2 project. Fascinatingly, it seems like a big criteria that Microsoft had factored in was that tools for COFF already existed for the Intel i860 RISC processor, so early on in NT's life they seemed to think that it was going to be a lot more important. That said, I do not consider PE to be similar enough to COFF to warrant calling them the same format. Aside from having completely different headers, PE has many additional structures that don't exist in COFF, and even the structures that are shared are fairly different between PE and COFF. PE barely uses anything from COFF, and it doesn't even use it the way that COFF does. Of course ELF objects also have some differences between object file and shared object/executable, but it's at least very clearly the same format the whole time, just utilized a bit differently.

by jchw 2y ago

fngjdflmdflg 2y ago

>It obviously was segmented into translation units when it was originally compiled, but nothing really cares about those boundaries at the linking stage. (Of course, you probably want to try to figure it out if possible, since it will probably be very hard to do a matching decompilation if your object boundaries are incorrect.) Can you expound on this? Why are the boundaries not important at the linking stage - aren't you linking the wrong code then? Or did I not understand the point here?

boricj 2y ago
The boundaries aren't important during delinking, as long as you do not cut across a variable or function. To the linker, sections are just arrays of bytes. It doesn't care what's inside those, all it has to do is lay them out in memory and apply relocations. How you decide to slice up the original program is up to you. It doesn't have to follow the boundaries from the original object files.

skissane 2y ago

> Often times, except for Microsoft platforms, they are in the same actual file format, e.g. ELF. On 32-bit and 64-bit Windows (both NT and 9x), EXE, DLL and OBJ files are all COFF. EXE/DLL files stick an MS-DOS stub and the "PE\0\0" header before the COFF header, OBJ files start with the COFF header directly. This is different from DOS, 16-bit Windows and OS/2, where the standard object file format (Intel OMF) was completely unrelated to the executable formats (MZ for DOS; NE for 16-bit Windows and OS/2 1.x)

jchw 2y ago
Interestingly, I recently tried to dive in and figure out why exactly Microsoft went with COFF. It seems like Intel OMF was adapted all the way up through OMF386, and Microsoft switched to COFF around 1989, near the beginning of the NT OS/2 project. Fascinatingly, it seems like a big criteria that Microsoft had factored in was that tools for COFF already existed for the Intel i860 RISC processor, so early on in NT's life they seemed to think that it was going to be a lot more important. That said, I do not consider PE to be similar enough to COFF to warrant calling them the same format. Aside from having completely different headers, PE has many additional structures that don't exist in COFF, and even the structures that are shared are fairly different between PE and COFF. PE barely uses anything from COFF, and it doesn't even use it the way that COFF does. Of course ELF objects also have some differences between object file and shared object/executable, but it's at least very clearly the same format the whole time, just utilized a bit differently.