3 ms·

I work on a matching decomp project that has tooling to recompile C into binaries matching a 28 year old game. In the final binaries created by compiled with g

I work on a matching decomp project that has tooling to recompile C into binaries matching a 28 year old game. In the final binaries created by compiled with gcc 2.6.3 and assembled with a custom assembler there appear to be unused, uninitialized data that is whatever was in RAM when whoever compiled the game created the release build. Since the goal is a matching (reproducible) binary, we have tools to restore that random data at specific offsets. Fortunately our targets are fixed

fngjdflmdflg 2y ago
What even causes this to happen? ie. what dev tool would add random data from RAM to a binary? Is this likely a bug or is there some reason for it like needing to reach a specific file size somewhere?
- dezgeg 2y ago
  By accidentally writing out uninitialized memory contents to the file with the game still working. It's even worse in DOS era where there is no memory protection so uninitialized memory can contain data used by other processes, so for example parts of source code can get leaked that way. There's a big list of those in https://tcrf.net/Category:Games_with_uncompiled_source_code https://tcrf.net/Category:Games_with_uncompiled_source_code
  - jonhohle 2y ago
    Yeah, this was originally all DOS and Windows 3.1 utilities for writing programs that would run on MIPS. The data is small enough that it isn’t relevant, just not reproducible through standard build tools because it was never meant to be bitwise reproducible.
- aidenn0 2y ago
  Simply calling write() on a C struct can do that, if there is any padding in the struct. Then, of course, there are bugs.
tuananh 2y ago
please do write more about it.
- jonhohle 2y ago
  We use a tool named dirt-patcher[0], which was written for the project. It lets you write arbitrary bytes at specified offsets[1]. As far as we know at this time, they’re just uninitialized bytes that would have been padding for alignment or other reasons anyway. Maybe if we move to an official build tool chain we’ll find they are deterministic, but for now, we believe they are garbage that happened to make it into the final binary. 0 - https://github.com/Xeeynamo/sotn-decomp/blob/master/tools/dirt_patcher.py https://github.com/Xeeynamo/sotn-decomp/blob/master/tools/di... 1 - https://github.com/Xeeynamo/sotn-decomp/blob/master/config/dirt.us.json https://github.com/Xeeynamo/sotn-decomp/blob/master/config/d...

fngjdflmdflg 2y ago

What even causes this to happen? ie. what dev tool would add random data from RAM to a binary? Is this likely a bug or is there some reason for it like needing to reach a specific file size somewhere?

dezgeg 2y ago
By accidentally writing out uninitialized memory contents to the file with the game still working. It's even worse in DOS era where there is no memory protection so uninitialized memory can contain data used by other processes, so for example parts of source code can get leaked that way. There's a big list of those in https://tcrf.net/Category:Games_with_uncompiled_source_code https://tcrf.net/Category:Games_with_uncompiled_source_code
- jonhohle 2y ago
  Yeah, this was originally all DOS and Windows 3.1 utilities for writing programs that would run on MIPS. The data is small enough that it isn’t relevant, just not reproducible through standard build tools because it was never meant to be bitwise reproducible.
aidenn0 2y ago
Simply calling write() on a C struct can do that, if there is any padding in the struct. Then, of course, there are bugs.

tuananh 2y ago

please do write more about it.

jonhohle 2y ago
We use a tool named dirt-patcher[0], which was written for the project. It lets you write arbitrary bytes at specified offsets[1]. As far as we know at this time, they’re just uninitialized bytes that would have been padding for alignment or other reasons anyway. Maybe if we move to an official build tool chain we’ll find they are deterministic, but for now, we believe they are garbage that happened to make it into the final binary. 0 - https://github.com/Xeeynamo/sotn-decomp/blob/master/tools/dirt_patcher.py https://github.com/Xeeynamo/sotn-decomp/blob/master/tools/di... 1 - https://github.com/Xeeynamo/sotn-decomp/blob/master/config/dirt.us.json https://github.com/Xeeynamo/sotn-decomp/blob/master/config/d...