6 ms·

A pretty common trick that was part of game programmer lore back in the PS2 / Xbox era was to have a large static array hidden away in some code file somewhere.

A pretty common trick that was part of game programmer lore back in the PS2 / Xbox era was to have a large static array hidden away in some code file somewhere. When days before shipping you couldn't quite fit the release build in to memory this allowed a heroic programmer to miraculously 'find' a few hundred extra kilobytes of memory by reducing the size of the array by just enough to fit. There was a less common variant of this for finding some extra performance by reducing the iterations of a loop doing no-ops somewhere in the main game loop.

bastawhiz 9y ago
Sadly this usually works only once, especially if it's one person doing it. I used to work on the performance team at a Bay Area company. One of the things we did to keep our JavaScript bundle sizes under control was introduce a "ratchet". There was a threshold, enforced by CI, that you couldn't let the bundle size exceed without getting in touch with us first and figuring something out. [0] This worked wonders for a while, until a few different teams were starting new feature development. At that point, the ratchet was forcing teams to pause their feature development to do cleanup work, which made the PMs very unhappy. Engineers got salty because they would remove dead code, only to find that another engineer had gobbled up the space they'd freed before they could land their own commit. Engineers started working around the ratchet by hoarding dead code and disguising it to look "not dead" so that it could be easily removed later when a few dozen kilobytes were needed. There were many thing that weren't good at this company, and the culture around ownership of the shared codebase was definitely one of them. I'd like to think that there are plenty of teams that don't have this problem, but I'm inclined to think that it's human nature to subvert these sorts of things by default. [0] This was necessary because of the volume of tech debt. Teams/engineers had a bad habit of building new things, then not cleaning up the stuff the new things replaced. At one point, we estimated that over a third of the JavaScript was dead code. Some teams had gotten to the point where the codebase contained >2 versions of their product, while only one of them was physically accessible to users.
- mattnewport 9y ago
  Back in the era where this was somewhat common on games it only had to work once. In those days you burned a master for a console game and once it shipped that game was done. There were no zero day patches (or any other kind of patches) and no updates to the game once it shipped. Often it would be the lead programmer on the game who put this array in and it wouldn't necessarily be known about by everyone. It wouldn't have worked well if people were constantly grabbing bits of memory from it during development. It worked because it was a 'secret' and its use was reserved for shipping.
- Paul-ish 9y ago
  > Engineers started working around the ratchet by hoarding dead code and disguising it to look "not dead" so that it could be easily removed later when a few dozen kilobytes were needed. Sounds like an example of the cobra effect[1] [1] https://en.wikipedia.org/wiki/Cobra_effect https://en.wikipedia.org/wiki/Cobra_effect
chollida1 9y ago
I used to be a huge gamasutra reader so I've read the postmortem that described this. I was under the assumption that it was more of an apocryphal story that didn't actually happen. I mean how can someone hide a static array or no op loop from a team of 20+ programmers in a game code base. As far as I'm aware, it was never confirmed and seems a bit to far fetched to be real, but then again.... :)
- mattnewport 9y ago
  I've seen it done. In those days programming teams were usually smaller and this would be something put in place by a lead (usually a grizzled veteran of shipping multiple titles) and not advertised to the entire team. It might be hidden away somewhere like the core memory manager which would be owned by that lead and not generally touched by anyone else without consulting them. If someone happened to find it when running a memory profile they'd probably go and ask the lead about it and be let in on the 'secret'. I actually found one of these when pulled in to help a title ship that had been put in for a previous title in the franchise and forgotten about when the lead moved on. I found it when memory profiling and after talking with a few people we figured out what it was there for and I got to be the 'hero' who found some extra memory to ship.
  - Bartweiss 9y ago
    > If someone happened to find it when running a memory profile they'd probably go and ask the lead about it and be let in on the 'secret'. Perhaps the obvious question here is "hidden from who?" Blinding the entire dev team to the trick might be possible for the lead, but sounds like a pain. But squirreling away a bit of memory that won't be announced to PMs/artists/designers/producers? I can certainly picture a couple of programmers realizing that the person calling the shots intended to push them to the absolute limits of "what will fit", and deciding to fudge those limits. Hell, hedging is common practice for programmers and freelancers today when they anticipate bad requirements, and everything I know of the game development's history says the problem used to be much worse.
    - 9y ago
FRex 9y ago
I have seen it mentioned here on page 4: https://www.gamasutra.com/view/feature/132500/dirty_coding_tricks.php https://www.gamasutra.com/view/feature/132500/dirty_coding_t... There's yet another article on dirty game hacks that I recall due to EULA self-hack: https://www.gamasutra.com/view/feature/194772/dirty_game_development_tricks.php https://www.gamasutra.com/view/feature/194772/dirty_game_dev... I have no idea why this article doesn't link to these two and just to the 2017 one because it's not like these are current generation tricks, it's just that Brandon Sheffield took to write about them again (one of the articles above is actually his and the other one is by GDM that he is editor in chief of). Also, the Crash Bandicoot text is the most impressive one and the first in this article but it was very widely circulated for a while now on several sites (including on HN once), despite the "nobody was the wiser until now" at the start of the article.
- tekromancr 9y ago
  The one on the last page about using dexdrive was pretty cool. I used a similar technique to create ps1 save files that would trigger an exploit on the ps2 that would load unsigned code. I miss being able to do stuff like that. Modern systems are so locked down.
pjmlp 9y ago
Not only in game programming. Chet Haase & Romain Guy also told on their Android history keynote that the kernel team did the same. They have hidden 20 MB. https://www.youtube.com/watch?v=rimXGaUdaLg&t=773s https://www.youtube.com/watch?v=rimXGaUdaLg&t=773s
vvanders 9y ago
Hah yup, we kept 15M tucked away on the last game I shipped. Came in handy for demos more than anything else.

bastawhiz 9y ago

Sadly this usually works only once, especially if it's one person doing it. I used to work on the performance team at a Bay Area company. One of the things we did to keep our JavaScript bundle sizes under control was introduce a "ratchet". There was a threshold, enforced by CI, that you couldn't let the bundle size exceed without getting in touch with us first and figuring something out. [0] This worked wonders for a while, until a few different teams were starting new feature development. At that point, the ratchet was forcing teams to pause their feature development to do cleanup work, which made the PMs very unhappy. Engineers got salty because they would remove dead code, only to find that another engineer had gobbled up the space they'd freed before they could land their own commit. Engineers started working around the ratchet by hoarding dead code and disguising it to look "not dead" so that it could be easily removed later when a few dozen kilobytes were needed. There were many thing that weren't good at this company, and the culture around ownership of the shared codebase was definitely one of them. I'd like to think that there are plenty of teams that don't have this problem, but I'm inclined to think that it's human nature to subvert these sorts of things by default. [0] This was necessary because of the volume of tech debt. Teams/engineers had a bad habit of building new things, then not cleaning up the stuff the new things replaced. At one point, we estimated that over a third of the JavaScript was dead code. Some teams had gotten to the point where the codebase contained >2 versions of their product, while only one of them was physically accessible to users.

mattnewport 9y ago
Back in the era where this was somewhat common on games it only had to work once. In those days you burned a master for a console game and once it shipped that game was done. There were no zero day patches (or any other kind of patches) and no updates to the game once it shipped. Often it would be the lead programmer on the game who put this array in and it wouldn't necessarily be known about by everyone. It wouldn't have worked well if people were constantly grabbing bits of memory from it during development. It worked because it was a 'secret' and its use was reserved for shipping.
Paul-ish 9y ago
> Engineers started working around the ratchet by hoarding dead code and disguising it to look "not dead" so that it could be easily removed later when a few dozen kilobytes were needed. Sounds like an example of the cobra effect[1] [1] https://en.wikipedia.org/wiki/Cobra_effect https://en.wikipedia.org/wiki/Cobra_effect

chollida1 9y ago

I used to be a huge gamasutra reader so I've read the postmortem that described this. I was under the assumption that it was more of an apocryphal story that didn't actually happen. I mean how can someone hide a static array or no op loop from a team of 20+ programmers in a game code base. As far as I'm aware, it was never confirmed and seems a bit to far fetched to be real, but then again.... :)

mattnewport 9y ago
I've seen it done. In those days programming teams were usually smaller and this would be something put in place by a lead (usually a grizzled veteran of shipping multiple titles) and not advertised to the entire team. It might be hidden away somewhere like the core memory manager which would be owned by that lead and not generally touched by anyone else without consulting them. If someone happened to find it when running a memory profile they'd probably go and ask the lead about it and be let in on the 'secret'. I actually found one of these when pulled in to help a title ship that had been put in for a previous title in the franchise and forgotten about when the lead moved on. I found it when memory profiling and after talking with a few people we figured out what it was there for and I got to be the 'hero' who found some extra memory to ship.
- Bartweiss 9y ago
  > If someone happened to find it when running a memory profile they'd probably go and ask the lead about it and be let in on the 'secret'. Perhaps the obvious question here is "hidden from who?" Blinding the entire dev team to the trick might be possible for the lead, but sounds like a pain. But squirreling away a bit of memory that won't be announced to PMs/artists/designers/producers? I can certainly picture a couple of programmers realizing that the person calling the shots intended to push them to the absolute limits of "what will fit", and deciding to fudge those limits. Hell, hedging is common practice for programmers and freelancers today when they anticipate bad requirements, and everything I know of the game development's history says the problem used to be much worse.
  - 9y ago

FRex 9y ago

I have seen it mentioned here on page 4: https://www.gamasutra.com/view/feature/132500/dirty_coding_tricks.php https://www.gamasutra.com/view/feature/132500/dirty_coding_t... There's yet another article on dirty game hacks that I recall due to EULA self-hack: https://www.gamasutra.com/view/feature/194772/dirty_game_development_tricks.php https://www.gamasutra.com/view/feature/194772/dirty_game_dev... I have no idea why this article doesn't link to these two and just to the 2017 one because it's not like these are current generation tricks, it's just that Brandon Sheffield took to write about them again (one of the articles above is actually his and the other one is by GDM that he is editor in chief of). Also, the Crash Bandicoot text is the most impressive one and the first in this article but it was very widely circulated for a while now on several sites (including on HN once), despite the "nobody was the wiser until now" at the start of the article.

tekromancr 9y ago
The one on the last page about using dexdrive was pretty cool. I used a similar technique to create ps1 save files that would trigger an exploit on the ps2 that would load unsigned code. I miss being able to do stuff like that. Modern systems are so locked down.

pjmlp 9y ago

Not only in game programming. Chet Haase & Romain Guy also told on their Android history keynote that the kernel team did the same. They have hidden 20 MB. https://www.youtube.com/watch?v=rimXGaUdaLg&t=773s https://www.youtube.com/watch?v=rimXGaUdaLg&t=773s

vvanders 9y ago

Hah yup, we kept 15M tucked away on the last game I shipped. Came in handy for demos more than anything else.