3 ms·
Treyarch's Spiderman 2 on the PS2 was using lzo compressed data, but compressed in such way such that compressed chunks were up 64kb, decompressing to max 128kb
by malkia 11y ago
Treyarch's Spiderman 2 on the PS2 was using lzo compressed data, but compressed in such way such that compressed chunks were up 64kb, decompressing to max 128kb (avg ratio was 1:1.5 I think). Biggest speedup for the lzo-decompress, was to put a few gcc asm instructions to use the unaligned write, and I think I've used unaligned read one too. Also the data was padded already to 16 bytes (or even more). This allowed us to do 5/6 mb/sec uncompressed speed from disk DVD that was able to do 2 to 4mb/sec depending on whether it was close to the center, or on the edge (e.g. it was constant angular velocity, rather than constant speed - I think we were using this mode as it was less noisy, and more reliable). This helped us also for the japanese ps2, many of which have shipped with problems, limiting even their outer speed to 2mb/sec. (Spiderman2 was heavily streaming game).
The decompression was done on the IOP chip (e.g. the PS1 in the PS2), on the PS2 this chip was used for file I/O, sound, and few other minor things (I think controller too, and maybe memory card). So the lzo code originally was only doing 2/3mb/s decompression speed on the 40Mhz chip, but adding these two unaligned read/writes (I don't remember the details) gave us boost with 70-80% or more (again don't remember the details). ZLIB was out of the question, and lz4, lzfast, etc. were not known to us back then.
Later I've read that this unaligned read/write instruction had a bit of legal issue, but I can't find the article about it. Something about MIPS licensing it for it's use to other companies doing MIPS compatible chips.
As for the 64kb compressed -> 128kb uncompressed (on avg.) - Basically the way it was done - I've had only one 128kb buffer, and I would read the compressed chunk (from 8kb to 64kb aligned to 4kb (I think)) but I would read it at the end of that buffer, then LZO was pretty cool that it was able to decompress in-place, as long as your compressed data was a bit ahead of where it was going to get decompressed.
Then that buffer was transfered with the DMA to the PS2 CPU chip, and then the next one was read (not sure whether I tried double buffering, or not - details are escaping me now). I think Kung Fu Panda, and maybe one more game used it, but Spiderman 2 possibly had the best benefit. Still to this day I'm not sure how much of an impact this was, since the GC and Xbox versions shipped without compressing any data.
It was just a hint from a very cool coworker to try it out, and I did it :) Got help from few other colleagues to speed it up later...
- giovannibajo1 11y agoJust out of curiosity: LZO is very popular in games but it is GPL licensed. Were you violating the license by using a modifying the original C code in a proprietary codebase?
- malkia 11y agoWe've had the license purchased
- userbinator 11y agoLater I've read that this unaligned read/write instruction had a bit of legal issue, but I can't find the article about it. http://probell.com/Lexra/lexra.html http://probell.com/Lexra/lexra.html MIPS patented the instructions, likely knowing that they would offer a performance advantage. (The patent has since expired, however; but before that, I've written code to do unaligned accesses for the MIPS clones without them, and all the extra shifts/masks are quite bothersome.)