6 ms·
Paper Mario PC ports beckon as coder completes decompilation of the N64 classic
- VHRanger 4y agoHow do people come up with variable and function names in a decompilation like this? Do they guess it based on what they end up thinking the function/variable does in the code?
- soulofmischief 4y agoIn the case of Super Mario 64, some of the source code was leaked in 2020.
- m-p-3 4y agoWhich you can absolutely not base yourself if you want to ensure it's considered a clean-room reverse engineering and not have Nintendo's armies of lawyers sweeping down on you.
- stefan_ 4y agoThey are going through and mechanically translating functions 1:1 from the original binary, so there aren't exactly any concerns with "clean-room reverse engineering" here.
- m-p-3 4y agoYou can still achieve that without having access to the original source code, so the implementation would be considered clean from copyright claims as long as it doesn't contain any copyrighted assets (which are necessary to achieve a perfect clone), but those can be injected during compilation from the original ROM that the end-user must acquire themselves.
- AshamedCaptain 4y agoIn which country is this interpretation of copyright law valid ? Soon: Freedows 12. "Your Honor, I just looked at the compiled code, so it must be considered clean from copyright claims!".
- m-p-3 4y agoCopyright laws mostly covers the assets (images, texts, etc) and the actual implementation. If you happen to make your own implementation that leads to the same output when compiled, that would be considered fair use in most situations. As long as you didn't copy from the original source code as long as you're not distributing the assets or the compiled version, the copyright holder can't stop you from distributing your own work. ReactOS and Wine exists and are reimplementations of copyrighted code.
- AshamedCaptain 4y ago> If you happen to make your own implementation that leads to the same output when compiled, that would be considered fair use in most situations as long as you didn't copy from the original source code as long as you're not distributing the assets or the compiled version. Good luck trying to prove to any judge in the world that you came up with an implementation that happens to be exactly the same as the original game code, down to the same checksum, by pure chance. What is this, "the ChatGPT defense"? Do you really think that code has intrinsically less right to copyright than art, or what? >ReactOS and Wine exists and are reimplementations of copyrighted code. Clean room reimplementations that don't even dare to look at the copyrighted code.
- m-p-3 4y agoCopyright, specifically code will only let you protect the IMPLEMENTATION, the code the copyright owner specifically made, not the IDEA of the code. > Good luck trying to prove to any judge in the world that you came up with an implementation that happens to be exactly the same as the original game code, down to the same checksum, by pure chance. Not impossible if you reverse engineer the software and look how the binary code is behaving, emulators does this all the time. A reimplementation of code that leads to the same binary output isn't covered by the copyright, as long as you don't share copyrighted assets or the actual binaries. What people do with the alternative source code is their business, you can't hold responsible that person for others' action.
- dtx1 4y agoAnd Nintendo is well-known to be hyper litigious fucks that will try to destroy their fans in court, even when they are within the legal bounds of copyright.
- zamadatix 4y agoSuper Mario 64 was fully decompiled in 2019. It has went through several cleanup refreshes since then but they intentionally avoid the leaked source in doing so.
- AshamedCaptain 4y ago> It has went through several cleanup refreshes since then but they intentionally avoid the leaked source in doing so. As if that would make a difference in how big of a copyright violation all of this is.
- zamadatix 4y agoAssets are not included in the process, which helps a lot. It still requires the original ROM to compile a ROM. Nintendo has heavily went against PC ports, even as far as having videos about them taken down on YouTube, but never even attempted to touch the root decomp projects for various titles. Later games you can get into more trouble because EULAs or on screen warnings started restricting reverse engineering, as well as DRM mechanisms which are impossible to reverse engineer without being implicated as also advertising a way to work around them.
- soulofmischief 4y agoIs this clean-room development process the cover story or honest-to-god truth? How does anyone verify that outside of the individuals who worked on it directly?
- circuit10 4y agoYou could download the leak and compare them I’m sure people have done this, I have the leaked code somewhere so I could do it if you want
- zamadatix 4y agoThe decompilation process and toolset made available would take more work to fake than to just do, not to diminish how much work that last bit is. Particularly since parts of the toolset work successfully across many titles (e.g. Paper Mario, Super Mario 64, and Ocarina of Time).
- chongli 4y agoJust judging by the title of the article, the process was a long one. If you start at the entry point of the game and trace every memory access, and you run the game in a profiling and debugging emulator, you can figure out what every variable does. It’s a long and painstaking process. Definitely one you probably don’t embark on unless you really care about the result (or you’re well paid).
- rhn_mk1 4y agoThat's right. You can look for signs of what the piece of code does, or what the memory is used for. Functions will betray they purpose by doing calls to external libraries. Those libraries have functions with certain names and their arguments have names too. Then you have counters which you can recognize by looping and incrementing by a certain amount. Some variables are easily guessed because they refer to a string that's stored in plain text. "%d:%s" is likely to be a printf format string, so you can follow that lead and find out which arguments correspond to %d and %s. Sometimes you get lucky and the debug message tells you directly something like "Invalid value of the coin counter: %d". This is a dead giveaway for what the argument corresponding to %d is. Other kinds of variables can be guessed by using a debugger and looking at their values, or patching code to modify them and watching what happens.
- crazygringo 4y agoI can't help but think that might be something a ChatGPT-type tool could be extremely helpful at. Trained on enough code, couldn't it figure out reasonable enough names from the structure of the code? On the other hand, it might not work at all if short functions tend to be incredibly generic, and all the real meaning is tied up in the nonlocal organization of it all. And then of course you need context to know what the nouns and verbs are at all. I can imagine similar structures involving Luigis and Koopas, or invoices and payments.
- circuit10 4y agoI want to try this but the context problem seems a bit hard to solve, you probably need to have a mental model of the whole program at once
- nicolas_17 4y agoGee I'm sure nobody thought of using AI for that before /s
- klondike_ 4y agoDebug symbols were left in the released binary for SM64
- circuit10 4y agoI think they didn’t turn optimisations all the way up but I didn’t hear anything about debug symbols. Are you sure you’re not getting it mixed up with that? I see how you could because a build with less optimisations is often a debug build
- AshamedCaptain 4y ago> As draconian and litigious as Nintendo is about these things, previous projects of this type have managed to avoid the gaze of its lawyers so far. [...] The reason is probably because projects like these don't really infringe on any of Nintendo's intellectual property. Coders like Roseman aren't just ripping Nintendo's assets (like textures) out of their games and slapping them onto the PC, instead, they're remaking the code themselves, which Nintendo has yet to find a way to crack down on. How is this is even remotely legal. They have literally decompiled Nintendo's binary -- up to the point the decompiled sources build to a binary with the same checksum -- and are now redistributing it. There's absolutely no interoperability motivation or any other type of DMCA exemption that would apply here. It's like claiming that the art assets are subject to copyright but not the program.
- Waterluvian 4y agoA very common theme I see among groups who commit time to these game remake/remaster/port passion projects is that they seem quite willfully ignorant of what the law is and shocked when the obvious outcome comes to pass. And then some site like Kotaku picks it up and frames them sympathetically, so Nintendo becomes “draconian” when it is doing exactly what it needs to do: protect its IP and business. What I’m fascinated by is just how much free work people throw into obviously doomed projects. And now they’re doing all kinds of acrobatics hoping to find some loophole. I think this suggests just how strongly emotionally attached people can become to games they grew up with. I can definitely sympathize. But also… if you’re this talented… make the world a new game. Make A Hat in Time if you love Banjo Kazooie so much. Make Stardew Valley if you love Harvest Moon so much!
- AshamedCaptain 4y ago> A very common theme I see among groups who commit time to these game remake/remaster/port passion projects is that they seem quite willfully ignorant of what the law is and shocked when the obvious outcome comes to pass. [..]I think this suggests just how strongly emotionally attached people can become to games they grew up with. I have also seen it in non-game projects, e.g. retro OS reimplementations. I am not even sure where this misconception comes: that you can just decompile the original software, rearrange the source files a bit and that's it! . Even otherwise smart people actually believe this. It's incredibly damaging and pervasive thinking; by now I have to mistrust all new "miracle" contributors that come up with significant amounts of code.
- Waterluvian 4y agoIf they’ve written their own code and need you to supply the assets from a ROM, have they not basically just written an emulator that runs only one game? Yes, it’s not really “emulation,” but the UX is basically the same. I also imagine emulating the game was never really a big problem. So this was more just for fun/passion?
- circuit10 4y agoIt makes mods way easier since you can modify the source instead of struggling with assembly patches
- Waterluvian 4y agoOh that would be kinda fun. I’d be amazed if Nintendo tolerates that. I’m thinking about how SMW romhacks basically inspired (but competed with) Mario Maker.
- geraldwhen 4y agoShip of Harkinian is the ocarina of time decompilation. It fixes so much of the base game issues and adds extra fun stuff like a randomizer.
- circuit10 4y agoThey’ve tolerated it so far with Super Mario 64 but they might not forever
- ur-whale 4y agoAutomated and smart decompilation is the ONE thing I would love to apply AI to. Especially given the fact that we can generate and infinite training set: High level code -> compiler -> assembly code All the AI's got to do is learn the inverse map. All you have to do is grab all of github, feed it to random compilers, grab the assembly output. Boom : instant training set for a decompiling AI (and one that can recreate original code from assembly, hopefully complete with comments and meaningful variable and function names)>
- circuit10 4y agoA problem with that is that you sometimes need to consider the context of the whole program at once and things in different files so limited context length is an issue
- ur-whale 4y ago>limited context length is an issue Fresh from HN: https://arxiv.org/abs/2304.11062 https://arxiv.org/abs/2304.11062
- nicolas_17 4y agoGee I'm sure nobody thought of using AI for that before /s
- ur-whale 4y ago> Gee I'm sure nobody thought of using AI for that before /s Instead of behaving like an unproductive ass, how about you do something useful and provide a link to folks working on the problem?
- TeaDude 4y agoThis is really cool. I'm personally most interested in the event scripting system as there's not a lot of documentation for developing those for systems with constrained ram (where you want a lot of stuff to be stored and read directly from ROM and not a lot of dynamic allocation ..if any!) Most embedded scripting projects reimplement languages like javascript which aren't very well suited for these conditions. Mostly intended to be run standalone instead of enhancing a running application. Actually, I might look at those Zelda decompilations as well now that people are bringing them up.
- b409ba0801cd21 4y agoWhat surprises me the most about these decompilation projects is that they post the whole decompiled code instead of just a list of directives to feed into a given (hopefully open source) decompiler. I used to be somewhat active on a ROM hacking message board, and there were rules about how users could share their modifications. Users had to share a patch. Uploading or linking to prepatched or original binaries was prohibited, and moderators would routinely intervene and remove them. Having been exposed to such measures to avoid action from copyright holders in the past, I just don't understand why it is so popular to just post the code openly.