30 ms·
GTA III and Vice City fully reverse engineered: re3
- eimrine 6y agoAny new technics revealed for speedrun community? AFAIK, WR for GTA VC is 8 minutes and this run cannot be possible w/o reverse engineering.
- eimrine 6y agoI have just realised that this is not a leak of source code but a new implementation. This version definitely needs a new speedrun, I have contacted Marushko.
- james_woods 6y agoNice! Btw: Is there a list with projects that got done due to the pandemic? :)
- MaxBarraclough 6y agoVery impressive. Neat that they're even intending to get a PS2 build working. Wonder if they plan to do further work on graphical improvements.
- aap_ 6y agoPS2 build is a lot of work unfortunately and I'm afraid it is me who'll have to do it. Guess we'll see how far we'll get... I've done some graphical improvements but we want to keep it mostly true to the original spirit. But yes, there are some things i would still like to improve upon a bit.
- mcjkrw 6y agoWhat is so different about the PS2 version? Now that you have the source code for PC, why is it a lot of work to port it to PS2? By the way, this project is so cool. Thanks for making this!
- aap_ 6y agoHi, (co-)author here. Cool to see this here :)
- ChuckNorris89 6y agoThanks for your efforts.
- ayoubElk 6y agoThis is fascinating, thanks! Can you speak about what it took to achieve this? (required knowledge, process, etc...)
- aap_ 6y agoWe were lucky that we had symbols from PS2 gta3 and the android games. other than that it was a lot of reading code in IDA and massaging it back into c++. I made a little video about part of the process [1] but i never did part 2. The strategy for gta3 was to replace function by function of the game until we had everything replaced. for VC we evolved our existing code base by, again, reversing function by function until we had everything done. Just not by dll injection this time. [1] https://www.youtube.com/watch?v=22BeuOOERLo https://www.youtube.com/watch?v=22BeuOOERLo
- app4soft 6y agoAny ideas to make it usable under ATI RS600 on Linux with Mesa drivers?
- aap_ 6y agoI assume you have tested it and it doesn't work? Maybe the GPU is simply too old. I'm doing everything with shaders and it's possible they are just too much for this GPU. Supporting a wide range of hardware (especially old one that nobody has anymore), is not easy.
- app4soft 6y ago> I'm doing everything with shaders... upporting a wide range of hardware (especially old one that nobody has anymore), is not easy. What about adding just an option (e.g. `re3 --no-shaders`) for may it possible run re3 even on older hardware by disabling most of decorative shaders?
- aap_ 6y agoMaybe this site is the best place to ask. There are two major things we could use help with: - the games do not run particularly well on the raspberry pi (any version really) and I have a feeling it's at least partially due to the OpenGL layer. We could really use someone with how to optimize OpenGL (especially on the pi). - physics have always been wonky beyond 30fps. I think someone with a good knowledge of game physics could really help us here.
- sokoloff 6y agoI did some of the physics work (but was not the lead) for Papyrus racing games in the ‘95-‘97 era. Email’s in my profile if I can help. My recollection of Vice City is that the vehicle dynamics model is not particularly realistic but optimized for “fun gameplay with a hint of realism” whereas Papyrus optimized towards “realistic gameplay with a hint of fun”. :D Our physics engine was deeply, deeply tied to a particular isochronous rate (30 Hz, then later 60Hz, IIRC) with all kinds of hackery to make that happen and divorce rendering rate from physics rate. (On the PSX, we ran physics engine tied to the VBlank.)
- aap_ 6y agoYeah, i think so too. In Vice City the most surprising thing to me was that they actually modelled tire temperature. it's probably a bit crude but i found it interesting. As for the physics code, there's a big ProcessControl function all sorts of physical entities that does everything that has to happen to that entity each frame. some of it is physics, some not. My theory as to why varying timestep produces different results is that the physics calculations form some sort of differential equations whose solution depends on the timestep. That may be wrong but it makes sense to me at least. My approach to fixing it would be to log all sorts of forces applied to the physical objects each frame and log them so they can be compared. this is quite a bit of stuff to implement though so nobody has done it yet. if you or somebody is interested in that, we'd appreciate that a lot.
- rzzzt 6y agoThe physics engine of Vice City is definitely tied to the frame rate. If you switch off vsync, you can only drive in reverse veeery slowly.
- simonh 6y agoExcellent work, I replay these on my iPad every few years. It would be great to get these on the Switch.
- lostphilosopher 6y agoDo you use a controller? I’ve thought about trying one of the mobile ports but the idea of playing with a touch screen doesn’t seem viable. Is this solved somehow?
- simonh 6y agoIt’s an excellent experience in my opinion. The virtual D-pad is fine for driving. It’s not ideal for on foot gunplay, but serviceable and that’s really a side dish to the main course driving experience. In just one or two places in some of the games you get a bit stuck. In GtA3 there’s a challenge where you need to chuck a grenade through an upper story window that’s really tricky, for example, but apart from that it’s fine and IMHO these games are incredible value for money at their current prices.
- rambojazz 6y agoHow is it possible to RE a whole videogame? What's the process to go from bits to C++?
- deleted 6y ago[deleted]
- CodeArtisan 6y agothe bits are cpu instructions that you can analyze the data flow to revert the instructions into c++ code, either manually or assisted with a decompiler. https://en.wikipedia.org/wiki/Decompiler https://en.wikipedia.org/wiki/Decompiler https://en.wikipedia.org/wiki/Ghidra https://en.wikipedia.org/wiki/Ghidra https://ghidra-sre.org/ https://ghidra-sre.org/
- ThrowawayR2 6y agoYou have a long list of machine language instructions. You know (roughly) what the output of the compiler looks like when translating from source code to machine language for various constructs like functions, if statements, etc. (a compiler course helps here). Some reverse engineering tools provide partial automation for that to assist in this deciphering. After that, it's just a lot of painstaking, grinding work.
- marcodiego 6y agoOne thing I like about these re projects: they allow porting to different systems by recompiling. It is an interesting feat to run SDLPoP (re'd prince of persia) on modern systems.
- rvnx 6y agoThis is very very impressive ! This reminds me the efforts to reverse-engineer Super Mario 64 that led to SM64 PC (and looking forward to the Zelda Ocarina of Time reverse!)
- tnolet 6y agoAs a basic web / JS developer I’m envious that creators of this repo get to work on awesome methods and classes called “Crime.h” and “!ped->bRenderScorched && !ped->bBodyPartJustCameOff”. Beats “div.container” in my book.
- grokkedit 6y agoYou can still write a fully working page with a body but without a head, and a parent can do whatever it wants with their children, destroy them if necessary... It's not that bad...
- tmpz22 6y agoQuirky variable names does not a good code base make.
- spoonjim 6y agoIt’s not a quirky variable name, it’s just the description of what’s happening in the game.
- bzb6 6y agoRockstar forgot to strip the PlayStation binaries so I suppose those are the variable names of the original codebase.
- aap_ 6y agoThose are struct names though. So they wouldn't be in there. They did share some headers with the MTA developers though and that's where we got bRenderScorched from. i think the body part name is ours though.
- bzb6 6y agoWow... how did mta convince rockstar to share stuff like that? For what purpose?
- tnolet 6y agoRandom question from a web developer who has zero knowledge of C++ or game development: I see no tests in this repo. Is this normal, or am I missing something?
- aap_ 6y agoIt's quite hard to test heavily interactive code like this. Not totally impossible but you have to question if it's worth the effort.
- softwhale 6y agoIt may be interesting to note that GTA:V's leaked build logs featured some notes about an "auto play bot test failed at {mission}". I'm pretty sure they would have to have similar tests (although perhaps at a smaller scale) for older GTA's as well.
- rushilhalflife 6y agoI had a short stint at a AAA games company and I worked on one of the games. It was all written in C++ and we had absolutely 0 unit tests. All the testing was done by an army of QA guys.
- tnolet 6y agoThat is wildly interesting. Every SaaS startup doing some 100k of $$$ in ARR will probably have long testing suites. AAA games making possibly tens of millions. 0 tests.
- mschuster91 6y agoLots of things are at play here (excuse the bad pun): * SaaS stuff can be built very quickly (if you use cached node_modules or PHP composer's vendor directory) and deployed to parallel CI test runners that don't have much requirements. In contrast, your typical game takes much longer to (re)compile and as soon as you touch anything that requires graphics, you need some sort of (beefy) GPU in the server. Anyone who has ever tried virtualization with PCIe passthrough knows what a pain that is to set up, not to mention NVIDIA explicitly disallowing to virtualize their GPUs. * It's easy to do unit tests for simple entities and operations (e.g. collision detection, physics)... but games are full of global state mutated from across the place. * For everything that tests visual (or, god forbid, audio) output, computers are bad at testing. Change a texture? Great, now you have to re-check all your tests if they're still accurate and working. Humans are, for now, still better. * Hardware-specific bugs: these are one of a goddamn nightmare. For most software, it behaves identically across computers - let's take most Docker containerized software... it will essentially run on anything that's amd64. But games integrate with OSes, especially the OS 3D stack (DX, OpenGL, Vulkan, Metal and on top of that the GPU drivers and GPU firmware) so incredibly deep that there are lots and lots of potential issues and workarounds. It's simply impossible to test for these. * Closed source components in the loop: Games ship with all sorts of blobs, from video codecs to sound effect libraries. These are in your address space, but you can't really test them. * Human tests are cheaper than writing test suites and setting up CI test farms (see the first point in my list).
- albertzeyer 6y agoCan you give some more details how the reverse engineering was done exactly? (And maybe put that also in the README or somewhere, because I think this is probably interesting to some.) You write: > re3 was started sometime in the spring of 2018, initially as a way to test reversed collision and physics code inside the game. This was done by replacing single functions of the game with their reversed counterparts using a dll. How exactly? So the game EXE exported all the function symbol names? How did you know the function signature (arguments, return type)? And then it is enough to preload a DLL to replace the function? There is no global state or so which must be accessed?
- aap_ 6y agoMaybe you'll find the first commit instructive: https://github.com/GTAmodding/re3/tree/600bf0351476a5a21aabb5429132ddf7f52ac0b9 https://github.com/GTAmodding/re3/tree/600bf0351476a5a21aabb... We were very lucky that we had symbols for global things, which inludes function names and signatures but not return types. We then replaced the first instruction of a function with a jump to our own. global variables can be done with a reference to a raw address. virtual methods are a bit of a challenge but we found way to handle them.
- dom96 6y agoWow, that's cool, so you replaced each of the game's functions individually until you had the majority replaced and working stand-alone?
- butz 6y agoNow someone must add multiplayer to this.
- boyadjian 6y agoWOW, need ! Vice City was my favorite video game of all times. Will try to compile this soon
- noname120 6y agoCheck the other branches out! There is one for GTA Liberty City Stories[1], and another one for GTA Vice City[2]. This project enabled people to port GTA to the PS Vita device[3]. This device hasn't seen any games from the franchise, so this is a big thing. [1] https://github.com/GTAmodding/re3/tree/lcs https://github.com/GTAmodding/re3/tree/lcs [2] https://github.com/GTAmodding/re3/tree/miami https://github.com/GTAmodding/re3/tree/miami [3] https://github.com/TheOfficialFloW/re3 https://github.com/TheOfficialFloW/re3
- llarsson 6y agoDumb question: can I play this using assets from the PS2, it does it have to be the PC version?
- aap_ 6y agoIt can actually load most of the PS2 files, but generally no, you need PC files.
- jcpham2 6y agoI still play my PC CD Roms patched under wine so this is neat thanks!
- jonplackett 6y agoI remember when GTA 3 came out on PC and I found just a simple human-readable text file you could edit to modify all the settings for all the cars. We turned the Karuma into a physics defying monster. It was 1000x heavier than the tank, so as you drove down the road other cars just bounced off and flew over the horizon, or exploded on impact. Yet it still did 0-100 in 0.1 seconds and could drive up walls. One time we jumped it from the third island, over the middle island and landed on a skyscraper roof on the 1st island. From there we spent about an hour dropping grenades on police cars and the army who had no way to reach us up there. Fun times. Shame Twitch or YouTube didn’t exist back then to tell anyone about it!
- artemonster 6y agoIf I remember correctly the file called "handling.cfg" and it was a huge unreadable CSV table :) good times.
- jonplackett 6y agoYeah that rings a bell! Really couldn’t believe my luck when I found something so obvious (I still felt like I had mad hacker skills)
- asadlionpk 6y agoYes. It was a csv table with all the configs for physics engine for all vehicles.
- aap_ 6y agoNot exactly CSV but close. The good thing is now you get to see the...less than optimal code that reads this file: https://github.com/GTAmodding/re3/blob/master/src/vehicles/HandlingMgr.cpp#L119 https://github.com/GTAmodding/re3/blob/master/src/vehicles/H...
- pylua 6y agoYes ! I have a similiar memory. When I was a kid, My parents did not like me playing the game for many reasons, one being the in game prostitutes . I simply changed a file to have them removed .
- 4gotunameagain 6y agoI think that for games like those, which more or less defined entire generations of people (the ones that were into gaming ofc) in a lot of countries, it should be mandatory that their source code is made public after like, ten years or something. Sure, a company created it and it should be its intellectual property for enough time to return the investment etc. But something ingrained so much into people imo trancends that and should be considered belonging to the public realm.
- news_to_me 6y agoHave you ever heard of free software? ;)
- 4gotunameagain 6y agoI have never heard of compulsory source releasing
- jrrrr 6y agoI can imagine a policy mandating that for all commercial software, source code goes into escrow for automatic release 10 or 20 years later. The company doesn't have to be around anymore, rights don't have to be negotiated, etc. Short of that I wish more companies would do it voluntarily. I'm not into id games, but they're one example that comes to mind.
- spideymans 6y agoThis is critical imo, if only because software are cultural artifacts that should be able to be archived just like any other cultural artifacts.
- jeofken 6y agoAny law in the end requires a violent threat. This suggestion implies that someone who does not publish their source code should, after som ignored formal letters, get armed men sent to their office to kidnap whoever is responsible. All nice sounding laws, such as gov funded welfare, gov funded hospitals, gov funded schools, and the endless regulations, implicitly carry a threat of violence, and when judging the ethics of a carrying a law out, that violent threat must be included. Government is firstly about guns.
- mentos 6y agoOf all the systems that you guys reverse engineered here what do you think is the coolest? Ai/pathfinding, quest systems, large world rendering?
- aap_ 6y agoI think everybody did the thing he was interested in most. For me that was rendering, physics, vehicles, camera and ps2 specific stuff like VU0 collision detection.
- deleted 6y ago[deleted]
- m00dy 6y agoIm not sure how long it takes for someone to port this into webassembly and distribute the assets along with a decentralized anonymous file network ?
- aap_ 6y agoThere have been emscripten ports of this already. Distributing the assets is probably the most problematic thing.
- mraza007 6y agoOh no I’m so excited to try this out on Linux. By the way i just want to thank you from the bottom of my heart for reverse engineering it. Now i don’t have to use Wine to run my childhood game on Linux
- mraza007 6y agoI’m just curious to know how do developers reverse engineer
- olologin 6y agoIs it true that NPC cars are not checked for collision? And collision checking only starts when something happens near?
- OneGuy123 6y agoGuessing but most likely so. I remember cleary that once I touched a car in GTA (can't remember which) the car visibly "fallen" to the road for a very small amount: so it was clear it was driving on tracks and was not properly physics simulated.
- breakingcups 6y agoThis is sort of true, they are "on rails" (meaning no physics calculations are performed) until something collides with it, like a bullet, a car or a player. At that point they are handed off to a more expensive process which includes checking physics each frame.
- marosgrego 6y agoNow, somebody implement multiplayer!
- 65a 6y agoWhat is this premake binary, and where are the sources for it? EDIT: you can just install premake from package management, and use that instead of the checked in binaries.
- withmorten 6y agopremake is something like cmake. The sources are at https://github.com/premake/premake-core https://github.com/premake/premake-core. The binaries in the repo are from their github actions builds, from whenever I last updated the premake binaries in re3: https://github.com/GTAmodding/re3/pull/945 https://github.com/GTAmodding/re3/pull/945
- eimrine 6y agoCan we spend a minute for paying respect a license? How it is called?
- neilv 6y agoNice labor of love, and team accomplishment. I might have to re-buy Vice City (for game assets), just to try this on my Linux GPU box. I got into the GTA franchise when (after not playing video games for a long time) I happened to see an old PS2 Slim that someone had put out beside the trash on the curb, brought it home, and started buying old games on eBay for a few dollars. Adolescent humor and terrible themes&glorification aside, GTA Vice City led me to GTA San Andreas, which was both a compelling open world, and some great gameplay. I came to very much dislike members of other gangs, who were always starting crap when I was just going about my business -- which I suppose fit the story, and was also good mechanics for a game. At some point, I had to buy a PS3, for GTA IV, which had been out a while. The PlayStation was a GTAstation. At the end of the story mode, as the cast-of-thousands credits rolled, over a montage of views of the New York City world they'd built, my reaction was that this game was a significant achievement for humankind. And it was engaging enough, that, by endgame time, er, I might've developed muscle memory for consistent PVP headshots, high-speed "chicken" drive-bys, and literally driving circles around another player's car while concentrating fire on their driver position. One of the highest compliments you can receive is being wrongly accused of cheating. In the very limited time I have for gaming lately (none, at the moment), it's usually been Ubisoft open worlds, but, whenever I check back into GTA Online freeroam, I'm reminded that it's the most actively fun for me. Cycles of retribution with another player or three, for the most creative or spectacular ways of fighting each other. And sometimes I even have the personally fulfilling opportunity to teach-- OP Mk. II griefers, that the Buzzard is still to be respected. (But forget heist stealth missions with randoms... :) (Story-wise alone, The Last of Us might be my all-time favorite game. I dislike zombie stories in any medium, but, by the climax, I was misty-eyed, caring for the child character I was protecting.)
- randomswede 6y agoHow possible is it to use the PS2 version for extracting assets? If the answer is "do this" or "we don't know", I am (in principle) willing to spend some time looking into it, but if the answer is "it won't work", that is useful information as well.
- AnonsLadder 6y agoThis is so cool, I can't wait to see a multiplayer adaption now that it's open-source! Oh please reverse engineer GTA:IV and bring it back to life as well
- deleted 6y ago[deleted]
- xchip 6y agoThis is amazing. I hope someone takes on doing the same for GTA: Chinatown Wars
- ioncube 6y agoMy favorite game. GTA III, Vice City and San Andreas had mod allowing to play online (multiplayer) called "Multi Theft Auto" (for Vice City there was another one called VC:MP) As far as I remember, the MTA project was started by a guy who initially tried to build a trainer by reverse engineering one of the GTA games, in the end he ended with a multiplayer mod. Fun times.