5 ms·
Disassembling Jak and Daxter
- strags 10y ago"GOAL never had a source-line debugger, as far as I know, although there's no reason it couldn't given some effort to write one." Actually, it did. IIRC, the debug state required for converting run-time addresses to source lines was persisted in the compiler (which was a REPL, not a one-shot process like C), rather than exported in the form of symbols. The interface was pretty sketchy, but I'd say it was about as usable as GDB. You could set breakpoints, inspect registers, etc... A lot of debugging was "live" though - since all the code was hot-loadable, you could just insert a debug print in the middle of a function on the fly. It was a really nice, iterative way to work.
- kayamon 10y agoOh cool, I wondered if ND ever went to the trouble to write one or not. Thanks :)
- Arelius 10y agoSince the entire thing was hot-loadable, how was the game linked, did every function call go through a indirection, or did you potentially re-link the entire program if the new function couldn't fit at the address of the old one?
- kayamon 10y ago> did every function call go through a indirection Yes.
- Arelius 10y agoI wonder if that would still be an acceptable trade-off with high-performance games on modern memory architectures.
- kayamon 10y agoMy suspicion is "yes" -- the PS2 wasn't great at memory access, and modern CPU tend to have better caching/branch prediction anyway.
- sedachv 10y agoNo need to wonder. That is exactly how C++ vtables work.
- fla 10y agoFor virtual functions. Not all calls get an indirection. At least with any respectable compiler..
- sedachv 10y agoRight, and how many game developers are recommending to "never use virtual functions because they are slow" in 2017? I remember seeing statements like that in Usenet posts and articles from the 1990s.
- anewhnaccount 10y agohttps://shaneenishry.com/blog/2014/12/27/misconceptions-of-component-based-entity-systems/ https://shaneenishry.com/blog/2014/12/27/misconceptions-of-c... is from 2014. I tend to think it might actually make a difference in this case given it's a vtable indirection for every entity every frame (or for every subsystem * every subsystem tick).
- Arelius 10y agoAlso, "never use virtual functions" is very different than "don't use virtual functions everywhere" Common wisdom is that virtual function usage in hot loops can become problematic.
- Arelius 10y agoAlso, "never use virtual functions" is very different than "don't use virtual functions everywhere" Common wisdom is that virtual function usage in hot loops can become problematic.
- AdmiralAsshat 10y agoI don't work at Naughty Dog, and I don't have any secret knowledge of Jak & Daxter, except what I figured out myself from the disc. So a lot of this may well be wrong. Take a pinch of salt with it. Isn't there an HN regular who did work at Naughty Dog, at least in the Crash Bandicoot days? Was he still around by the J&D days to fact-check?
- strags 10y agoI know of at least a couple of ex-dogs who lurk around here. The article's pretty much correct, if memory serves. I started at ND in 2004, around the tail-end of Jak 3. After Andy had left, I picked up the Goal compiler, and hacked it around to generate code for the PSP. After about a week or two, we had the kernel booting, and hot-patching working. We built a whole engine that looked amazing, but unfortunately the project was shelved. Still, it was some of the most fun I've had in my career.
- bunkerbewohner 10y agoInteresting read! If you want to read more about the programming of Jak & Dexter, as well as its predecessor Crash Bandicoot, check out Andy Gavin's awesome blog: http://all-things-andy-gavin.com/2011/03/12/making-crash-bandicoot-gool-part-9/ http://all-things-andy-gavin.com/2011/03/12/making-crash-ban...
- dgfgfdagasdfgfa 10y agoA while ago he gave me the manuals to GOALENV and GOALASM, but not the right to post them. Hopefully it's long enough ago now he'd be willing to make them public!
- Posibyte 10y agoI have such immense respect for the kind of people that made GOAL. From the powerpoint, it states that it was made by one developer over a year-ish, and solely supported it through the development of Jak and Daxter. It's the kind of confidence like that, in a very aggressive market, where delays are killer, but knowing that you can make a tool that would help make an even better product. Apparently the team had difficulty picking it up and using it for a while, and lots of issues that would run the development machines into the ground. Through all that, they still produced a very refined and high quality product on a tight deadline. Anecdotally, introducing Angular to a group of seasoned developers, more experienced in RPG and JCL than anything web-wise, over the course of a month was enough to lose sleep on. Heavy respect for that developer and the team as a whole.
- tomyws 10y agoIf the author is reading, I found the disassembler source[1] interesting - MSVC hangs when you compile it for Release! [1] https://github.com/rmitton/goaldis https://github.com/rmitton/goaldis
- kayamon 10y agoYeah I noticed that :) I believe it might be the giant if-then chain I use for disassembling opcodes.
- greggman 10y agoI worked at Naughty Dog, shipped Crash Team Racing and worked for 6 months on Jak & Daxter. These are my understandings. If someone there as different recollections I'm happy to be corrected. The LOD system in Jak & Daxter was based on / inspired by the LOD system in Crash Team Racing written by Danny Chan. The easiest way to explain that system is to imagine a cube with 8 vertices. Subdivide it so it's got 9 quads per face using 48 vertices. Let the artists move the vertices wherever they want. The close up LOD is the full 48 vertices, the far LOD is the 8 vertices. At some distance between the 2 the 48 are interpolated until they match the 8 so there's no popping. You can see this happen in CTR pretty easily if you play the Coco Park track. If you can manage to keep the purple pillars on the left in view you'll see them morph. Unfortunately this video doesn't keep them in view but it shows the pillars I'm referring to https://www.youtube.com/watch?v=StURmtwJfLk#t=6m30s https://www.youtube.com/watch?v=StURmtwJfLk#t=6m30s Hiding the morph or rather building things that don't appear to morph was a skill the artists learned as they progressed. Jak & Daxter you don't see it much. The first Ratchet & Clank which uses the same tech from Naughty Dog you see it quite often as their artists were new to the technique. As for GOAL it was my first experience writing a large amount of LISP. Before that I only wrote a few emacs macros. Things I took away from it * Having your own compiler can be a huge benefit Whether or not LISP itself is awesome many of the benefits were because we had our own compiler anything we wanted could be added. Live reloading of functions is not impossible in other languages if you have your own compiler. Another simple example you could specify the byte offsets of fields of structs. As a game programmer that seemed awesome compared to C where you just had to go through contortions and add padding and pray. * Having a full language for macros is amazing I'm still surprised so few other languages have this. LISP lets you use the entire language at compile time. Query a database, read a file, emit code. In C/C++ most people use python to generate code in my experience. C++ has the template system but it's arguably not designed to do the things people make it do. In fact I'm surprised there aren't more transpilers from some reasonable language to C++ template meta programming. Even if that was popular though C++ templates can't read files or query databases at compile time. As one tiny example I designed a particle system. There was an init structure listing all the parameters for particles. To be efficient it needed the list of be in the same order as the fields in memory. The macros I wrote would take an unsorted list and sort at compile time, filling out any missing parameters with defaults so the code that emits a particle would walk straight forward, no conditionals, cache friendly. In any case I'm still waiting for a mainstream language to offer this * Having a REPL into the game is cool You could almost think of working on Jak & Daxter like opening up an webpage in your browser's devtools. A REPL at the bottom you can start inspecting things, changing variables, even more you could replace functions. The whole thing ran inside emacs. You could open a file, put your cursor in some function and press some hotkey to compile just that function live and patch it into the running game. This was better than say Unity in that Unity serializes the entire game, compiles everything, then deserializes back where as GOAL would just compile that one function. * No 3rd party libraries I'm not sure this was important back then but nowadays with so much open source and commercial libraries to help make a game it was a problem that the GOAL environment was not really conducive to using any 3rd party code since that would have been in C/C++ * No Visual Studio quality debugger I've been on lots of projects without a good debugger and it's always been frustrating. Writing your own language from scratch probably means no good source level debugger. From the REPL you could set breakpoints and step through code but it was like using gdb. Sure I get by but not nearly as fast as a real debugger with watches, various panes showing live memory, live variables, conditional breakpoints, etc... Maybe now a days you could easily make a web based backend or abuse the browser devtools with source maps and remote debugging protocol or something to get a good debugger with low effort? * Slow Dev system This might have been a problem that was fixed but when I left you'd start up the game and wait several minutes as in like 10 or maybe even 20 before you could play. The entire AST for the code would be put into GOAL and then the game would be running. If the game crashed you'd get a message in the REPL. Andy Gavin, the guy that created GOAL would look at the message, see the issue, type some magic incantation, and the game would come back instantly. For me though who didn't have a mental image of how the entire system worked most of the time I just had to reboot and wait the 10-20 minutes. How that was eventually solved I have no idea. Maybe all the programmers internalized the system and could do the same magic that Andy could do. Or maybe Andy found a way to make the system start up faster. I'd be curious to know the answer. Of course this was offset by being able to live edit functions for fast iteration. * LISP can be fast There's nothing inherently slow about LISP. Maybe most impls are not fast but GOAL was. Jak & Daxter has some of the largest worlds, with the most polygons on the screen and it runs at 60fps on PS2 where as most other games on PS2 run at 30fps or less and far less polygons. That includes whatever garbage collection it used (I honestly don't remember what it did there). In other words, whatever assumptions I had about higher level languages and/or garbage collection being bad for game dev were mostly proven wrong.
- bitwize 10y agoThe first Jak game I played was Jak II. It blew my God-damned mind. Completely smooth world transitions, completely smooth character and facial animations... and AUTO-SAVE. I used to tell my buddies "they could do auto-save because they wrote a custom threading system using coroutines -- in Lisp! -- that interleaved memory card I/O with the main engine in a way that C++ just couldn't do at the time!" They didn't understand, but oh well. Lisp had saved me from putting up with the game grinding gameplay to a halt while it was "Saving. Do not remove the MEMORY CARD (8MB) (for PlayStation® 2) or turn off the power."
- aktau 10y agoThis is basically the reason I visit HN at least once every day. What a fantastic article. Like the author, I've always had a fascination with ND games and especially the Jak & Daxter series (a fascination which only augmented when I learned how much of them was coded in a Scheme and I understood what that meant). Unlike the author, I've never had the perseverance or chops to disassemble the game like this. Props, and thank you!