9 ms·
Resident Evil 4 (GameCube) – complete byte-identical decompilation to C/C++
- deleted 8d ago[deleted]
- davikr 8d agoHow many tokens and which models were used?
- wk_end 8d agoFrom a preservation perspective, this is one of the least useful decomps ever, given how committed Capcom is to making sure RE4 is ported absolutely everywhere (kidding!) Made using the leaked debug build and its symbols, which shows how meaningful the work the game preservation community that acquires and distributes these things is. OTOH this is pretty off-putting: > Where the compiler needed a particular source shape to reproduce a register choice or a schedule and no natural spelling was found, the construct is marked with a // COMPILER-DIFF: comment (644 of them: dead tests, empty asm("") launders and anchors, register T x asm("rN") pins, padding statements). To me the value of a decomp isn't reproducing the original bytes per se (we already have the original bytes after all) - it's about reconstructing the understanding of the original game as represented by human-readable source code; getting byte-for-byte is just an indication that you've gotten it right. Needing to add a bunch of slop to force the compiler to match the original output is actually just an indication that you've gotten it wrong - and it's a demonstration of the danger of Goodhart's law, especially as it applies to AI.
- Pannoniae 8d agoCorrect, this is just hacks for stuff you didn't manage to match exactly. Either that or your build environment isn't the same. Sadly, there are things which aren't really possible to reproduce in a byte-identical manner, things like exact file layout or variable declaration order, compilation order and that kinda stuff. And they might cause small but equivalent changes like different inlining/optimisation decisions, so it's really tricky to get it byte-exact.
- mitxela 8d agoWhy wouldn't those be possible to reproduce?
- Pannoniae 8d ago"assign all these pointers to the correct types, a wrong guess leads to different COMDAT folding" "get the order of local variables in this function right, otherwise the register allocation doesn't match. Oh and there's 150 local variables just in this function, good luck trying them all" "Find out the translation unit boundaries exactly (assume there's no pdb otherwise this is trivial) and after doing so, figure out the order they were compiled in, otherwise it won't match" "brute force the compilation flags for the project and if you're done, also bruteforce it for the CRT or any other middleware which usually came prebuilt so it doesn't match the main game" Should I continue;)
- mitxela 8d agoIndeed it's a shitload of work. It's also possible and has been done. You don't have to only use global brute-force - you could also reverse engineer the compiler. The OOT/MM decomps achieved completeness without //COMPILERDIFF.
- Pannoniae 8d agoCorrect me if I'm wrong (I'm not very well-versed in game decomp scenes) but aren't all those bytematched decomps from 90s or at the latest early 2000s games? They didn't have global optimisation (MSVC introduced it in VS .NET or 2003 I think and many games didn't use it until later) So these are mostly problems with more advanced compilers yk
- dezgeg 7d agoCorrect, N64 and PS1 era projects can essentially always be strictly tackled one function at a time. There are very minor things (like string literal sharing) where some earlier function can affect later function, but the workarounds are comparatively simple. On the other hand, I have heard even old MSVC is a nightmare. Various things are affected by hash table ordering (so the names of variables matter in some situations), stuff like order of #includes mattering, etc.
- toast0 8d agoCertainly, it's better if you don't have these. But if I was working on something like this, I think you release when you have something that covers most of the territory, and then refine it over time. Maybe someone else comes and helps out here and there.
- brandonpelfrey 8d agoYou might find it interesting that I am working on AI-driven decomp of a PS1 game not by matching bytes but by having agents produce C code and test code. Agents submit a C proposal to the harness, the harness compiles their C proposal for the original function, then the test suite provided by the implementer runs against the original machine code and the compiled C code version. The line and branch coverage of both must be 100% and given identical inputs and starting RAM, the function return value and RAM state and RAM/MMIO read write sequence must be identical. This is done with a small MIPS simulator which can run all the tests extremely fast. The reason I’m finding this is much faster than a traditional decomp is that while it’d be nice for the bytes to match, finding the perfect blend of compiler version, compiler args, permitting variables etc to try and find the perfect register assignments, etc is all very time consuming. My ultimate goal is not a byte for byte match, that’s just one way to ensure correctness. I’ve found agents are much faster and effective at reading the original assembly and understanding what’s going on then writing semantically equivalent C.
- Tiberium 8d agoThat's a great approach, I think byte matching is just popular because it's extremely easy to test in the end: are the bytes the same? While your approach requires putting far more trust into the tests.
- ErroneousBosh 8d ago> given how committed Capcom is to making sure RE4 is ported absolutely everywhere Heh. I was just looking at the RE family on Steam. Nothing over a fiver. I guess they really did make it more convenient than piracy.
- jchw 8d agoMaking a function fully matching by virtue of hacks like this is mostly not harmful to the ability to understand the code, but is a useful tool in ensuring that the code as a whole really is matching and identical to the original. Otherwise, it's difficult to prove. There is also value in decomps beyond just understanding and general interest. They can also be used to make more advanced mods, better translation patches, etc. The fidelity here matters, like having asm code accessing structures makes it hard to modify structures, but any fidelity improvement beyond pure asm is very welcome. Of course I still prefer to try to recover the original code that caused the compiler to do what it did, but it's a really challenging problem sometimes. I've been working on decompiling code from old versions of MSVC for literally years now and you accumulate some knowledge of what things impact register allocation or the order of symbols but some of it comes from things that get fully erased from the source. Like for example, debug builds generally seem to retain symbols that aren't actually referenced anywhere, but those symbols only actually make it into an object file if they are. For functions that were only ever inlined and not actually referenced anywhere... They still wind up in the object files and thus in debug builds, despite nothing referencing them. They are also COMDAT any'd because they can appear in multiple objects legally, which means the exact object that winds up retaining it in the final linked executable is arbitrary (and the compilation flags of the object containing it, too - I bet that was fun for developers to debug.) This is incredibly useful but very challenging, needless to say. It may even be feasible to construct examples that would be legitimately infeasible to simply guess back to equivalent source, which I suspect is a major reason why until it was finally shown to be possible in larger scale projects many people wrote fully matching decomps off as a fool's errand..
- Pannoniae 8d agoWith /LTCG /GL I don't think it's possible to get matching (or at least it's a very tall order), the codegen is wayy too volatile for an exact match and since inlining and reg alloc work on heuristics with thresholds it really cascades. Even stuff like what order you declare your locals in or the exact frontend syntax can mess things up...
- jchw 8d ago
- mitxela 8d agoThe OOT and MM decomp had a bruteforcer tool that would reorder lines until the register allocation matched.
- Sesse__ 7d agoYou're probably thinking of https://github.com/simonlindholm/decomp-permuter https://github.com/simonlindholm/decomp-permuter, which is used across many more decomp projects (and can do a lot more than swap lines). It's a huge help, but it's by no means enough for all regalloc differences; there's lots of stuff it cannot do.
- ricardobeat 8d ago“byte-identical” is one of Claude’s favorite expressions
- Tiberium 8d agoThe whole readme is extremely Claude-written, its dense Claudish style leaks from every sentence ;)
- ronnier 8d agoAnd what’s wrong with that? Nothing. AI is an amazing tool that has made me and many other more productive. It’ll only get better and stronger.
- mikeweiss 8d agoYes and at some point it won't even need YOU anymore.
- deleted 8d ago[deleted]
- moduspwnens14 8d agoDefinitely one of Claude's load-bearing terms.
- sick_of_slop 8d ago[dead]
- mikae1 8d agoI'm so emotionally torn about all the awesome decompilation work done. Too bad all the cool decompilation is happening for games on early 3D centric consoles. I say this a Quake fanatic. Low res textures combined with that blurry bilinear style filtering is a look that's not easy to love. It's just a generational thing I guess.
- toast0 8d agoI'm pretty curmudgeony on 3D games. The gamecube and friends were the 2nd generation of 3D first game systems and most games that sold well were not simply am existing genre game but with 3D objects that will poke your eyes out. Doesn't mean I liked them, or that they wouldn't have been better as a 2D game. But IMHO the graphics stopped distracting from fun games... OTOH, I played plenty of games on the Atari 2600 were the player character wasn't much more than a chonky pixel :p
- saturn8601 8d agoThose games like for N64 never really emulated great. I can see the motivation for those games to finally get nice gameplay on other platforms.
- chocochunks 8d agoRE4 isn't really an early 3D game. It was a late GameCube title almost a decade after Quake. There has been some 2D decomps too, Pokemon Red/Blue being the biggest I can think of. I can also imagine they make more sense for the 3D consoles where games tended to be written in higher level languages than raw assembly.
- mikae1 8d agoTrue, not the earliest gen. But it's a 2001 console (not 2004) despite this being a 2004 title.
- chocochunks 8d agoIt's a 2005 game. Yeah it's running on 2001 tech but it still has all the lessons learned from the previous years both in terms of 3D game design and getting the most out of the hardware. Would you call Doom 3 an early 3D game? It can run on 2001 hardware too. Or Half-Life 2?
- LaurensBER 8d agoDespite all "hate" against AI in the (retro)-gaming scene I'm genuinely excited to see these kind of projects (not sure if this one involved any kind of AI) and emulation. From a technical perspective emulation is absolutely amazing, it requires deep technical knowledge, an excellent understanding of the source system and the optimisations are on another level.
- ronnier 8d agoI really think the hate is just masked jealousy, now project managers, tpms and line managers can submit rather substantial code. That wasn’t possible before and it took away some value from devs (I say this as a dev of more than 25 years). I watch PM fix issues now instead of waiting for a dev. I understand the anger but these tools are not going away
- 12397 8d agoOf course it was possible before. You could put an Einstein paper on a photocopier, mask the author, put your name in the place and publish. Except everyone would have thought you are an idiot. You could clone the Linux kernel, erase all copyrights and release under Idiux. Except everyone would have thought you are an idiot. Now that there are laundering machines financed with unlimited printed and previously stolen money a large number of developers affirms and praises the idiots.
- bigyabai 8d agoIn the early days of emulation, there were tons of low-quality and imprecise emulator backends. If you wanted to play a dozen SNES games, you usually needed a half-dozen emulators to get them all to boot to the main menu. They were disparate efforts usually led by 1 or 2 people that wanted to get a Super Famicom game to run, and then gave up on implementing or fixing support for other games. We live in a golden age of emulation because that attitude died out. With more powerful computers (and less hack-oriented development) it became possible to properly emulate a complete console with a high level of accuracy. For preservationist purposes, accurate emulation is much more important than being the first to emulate a game, or even being able to decompile it. The only reason that we can point and laugh at low-quality efforts like Nintendo Switch Online is because the community cares even more than Nintendo did. The reason Wine/DXVK/Proton works so well is because it didn't fracture itself into a thousand downstream forks to fix one specific game. It's a holistic effort. A lot of the hand-wringing comes from a justified place that doesn't want to see emulation backslide into a bazaar-like community. You're free to disagree with their logic and vibe-code a thousand game decomps, but in all likelihood emulation will be a more popular choice for the foreseeable future. Lots of people emulate stuff on their iPhone or Steam Deck, almost nobody is playing a game decomp on it though. I can't imagine AI tipping the scales on decomp popularity, especially among average Joes.
- sharktheone 8d agoI hope that the CC0 license doesn't make any problems. It might though since decompilation does not remove the copyright of Capcom. Maybe just hope that they just don't care anymore
- sick_of_slop 8d ago[dead]
- groundzeros2015 8d agoI opened a file at random. This looks more like emulating the behavior in compilable C syntax rather than recovering the game's programming void Em1eWeaponSet(cEm10* em) { Em10Work* w = EM10_WK(em); w->mot[41] = 0; w->mot[42] = 0; … w->mot[62] = 0; w->mot[63] = ARC(0x26C); w->mot[64] = ARC(0x26D); w->mot[65] = ARC(0x276); w->mot[66] = ARC(0x277); w->mot[67] = 0; w->mot[68] = 0; w->mot[69] = ARC(0x26A); w->mot[70] = ARC(0x26B); w->mot[71] = ARC(0x270); w->mot[72] = ARC(0x271); w->mot[73] = ARC(0x272); w->mot[74] = ARC(0x273); w->mot[75] = ARC(0x274); w->mot[76] = ARC(0x275); w->mot[77] = 0; w->mot[78] = 0; }
- deleted 8d ago[deleted]
- dezgeg 8d agoThere is more normal looking code in other files. It sounds like the original game had animation data (or similar) embedded in the C code.
- groundzeros2015 8d agoI just checked a few more. It looks it can sometimes tell the intent of a stack variable and give it a name. But anything working with data looks like above.
- corysama 8d agoNot a surprise. Now the fun works starts: Deriving semantics from psuedo-assembly.
- Dwedit 8d agoThe real question is if it's relocatable or not (still works correctly if you add or remove bytes)
- shakna 8d agoCC0, on the decompilation of a copyrighted work? That's... Not the way it works. You inherit the copyright from the original, as this is derivative. There's a reason decompilation has always been a grey area of seeing whether or not the copyright holder cares.
- mitxela 8d agoYou also get to add your own copyright since decompilation is also a creative enough process. There can be more than one copyright holder in a work. The main reason you'd avoid this is to stop the first copyright holder from wanting to sue you - not because it's actually invalid.
- shakna 7d agoIf what you committed was piracy, then no. It doesn't matter how creative you were. You only get rights, were first you had a right to use. If you are "significantly similar", you are probably infringing. Feel free to explore the myriad of precedent setting cases in the music remix community.
- throwaway888777 7d agoThere are some absolutely insane ideas in the decomp space about how copyright works. You took a copyrighted work, did some lossless mathematical transformations on it, and have created an artifact that reproduces the original copyrighted work with perfect fidelity every time. From a legal perspective, this is effectively identical to zipping and unzipping a copyrighted file. You won't have much luck claiming that the intermediate Zip file is somehow free of copyright. The legal problem is the original taint of using a copyrighted work (expressly contrary to its license), and the fact that nothing transformative is occurring here sufficient to enliven the defense of fair use. (Not even close, and no, this is nowhere near the realm of being arguable. The fact that this spits out a copyrighted work 'byte for byte' is a stake through the heart of any such argument.) Decomp projects are cool, but if you work on them, please be aware that you're taking a risk, and some video game companies are famously litigious, even for old games.
- mw888 8d agoTangential: I have no issue with it, but this seems by README writing style to be AI-assisted. As time goes on, whether AI was used or not will become far less a point of interest - if the project is good it is good. The unceremonious use of (what I speculate is) AI-assistance is becoming more normal. I think a lot of people who were die-hard skeptics will just quietly protest less and less. Also, for people who use it well and poorly alike, the urge to not advertise any AI-use at all is common today. People who know how to use it well don't feel the need to explain themselves to unreasonable absolutist skeptics.
- devinprater 8d agoCool, could maybe eventually have a version blind people like me can play through this. A lot of blind vibe coders have been working on making decompiled games accessible.
- vivzkestrel 7d ago- special and maybe even a stupid request - can someone skilled and passionate look into doing some of the older ubi titles? - like maybe rainbow six vegas 1/2 or - splinter cell games?