5 ms·

StarCraft: Remastered – Emulating a buffer overflow for fun and profit [pdf]

249 points by jsnell 9y ago

ChrisRackauckas 9y ago
Awesome work! The community really is what makes SCBW great, allowing the use of the EUD maps again is part of what makes that tick. Now if Blizzard would just let ASL5 happen...
- my_usernam3 9y ago
  I'm not convinced Blizzard is the only culprit for the ASL delay. I know there's the 80k licensing fee, but afreeka didn't have to put them on blast in public. Negotiations should have happened behind closed doors.
  - ChrisRackauckas 9y ago
    Blizzard is paying Afreeca to host the GSL. Blizzard is paying others for Overwatch and Hearthstone tournaments. There's no reason for Blizzard to force organizations to pay to run a SCBW tournament. If Blizzard actually has a defendable position, they should share it.
- krzyk 9y ago
  What is ASL5? or EUD?
  - s_m_t 9y ago
    Afreeca Star League 5. The largest remaining Starcraft BW tournament. Extended Unit Death. A buffer overflow exploit that massively increased the power of the map editor's "scripting language" (called "triggers" in the map editor). It was called Extended Unit Death because the trigger used was intended for the map maker to specify some sort of action upon a player having a certain number of units die. By overflowing the unit field in this trigger you could read from any memory address and by doing the same to another set of triggers that manipulated the death count you could write to any memory address. It was patched out in 1.13 for obvious reasons, however, you can re-enable EUD functionality with custom launchers to play maps that take advantage of the exploits.
  - deleted 9y ago
    [deleted]
  - sanxiyn 9y ago
    EUD is the name of the buffer overflow whose emulation is discussed in the linked presentation.
a_t48 9y ago
This is really cool. It makes me wish I were still doing reverse engineering stuff for games...or that I had been smart enough to make a presentation on it while I still was.
- voltagex_ 9y ago
  You could still make a blog post or presentation on what you did.
  - a_t48 9y ago
    It's been far too long, unfortunately. I'm not even sure I still have the IDA database.
voltagex_ 9y ago
"Unfortunately, we did not have private or public symbols for StarCraft 1.16.1. I had to start reversing the game executable from scratch" So much history is lost because game companies don't archive their assets. * http://au.ign.com/articles/2013/06/27/original-kingdom-hearts-assets-lost http://au.ign.com/articles/2013/06/27/original-kingdom-heart... * https://adventuregamers.com/forums/viewthread/7766 https://adventuregamers.com/forums/viewthread/7766 * I also thought that the original Okami assets were lost with the closure of Clover Studio but I can't find a source.
- StillBored 9y ago
  In this case it sounds like they just lost the build artifacts, not the actual assets. Although, when you read about how people have recovered the source for old programs it makes complete sense. Without a central location where all the IP is stored, it becomes easy for older pieces to fall off the radar during company transitions, system upgrades, whatever. I suspect as people move away from large company wide SMB/NFS shares this is once again going to start being a common story. Particularly as people try to shrink their online storage footprint and move things to "archive" only. I've worked at places where I've had to repeating justify why i'm eating up a $LARGE_NUMBER GB for the last $SOME_NUMBER released versions of the product because the build process triggers a complete archive of the entire artifacts directories when a release build is specified. Online storage be it, S3 buckets, or LTO backed NFS shares have real costs associated with them, and invariably IT comes along and starts asking why they have to upgrade their systems, or justify their OPEX numbers. Having a few TB of build artifacts that are rarely (if ever as the versions get older) referenced sits right up there with keeping previous employee's machine images, or home directories live. The problem though is that putting things in an offline archive is just the first step in forgetting it exits. At that point your basically saying that in 10-15 years the only remaining copy of the data is going to be the git repo on some guys laptop sitting in his attic (the modern version of the floppies in shoeboxes).
- stryk 9y ago
  I remember reading about a story[1] last summer of a user on reddit buying a box full of "Blizzard stuff" on eBay and in it he found a gold cd-rom that was the master source code disk for Starcraft -- this wasn't even a hand-written label type thing either, it was ink-or-laser jet printed and had a custom sleeve and all. How does something like that end up in a box of junk for sale on eBay... crazy world. I think he ended up not ripping the contents, to the chagrin of many (including people advocating for source archival, but who knows how many were just using that as a cover and what they really wanted was to cheat at the game), and sending it back to blizzard and I think they flew him out to their game conference as a thank you. 1: https://kotaku.com/guy-finds-starcraft-source-code-and-returns-it-to-blizz-1794897125 https://kotaku.com/guy-finds-starcraft-source-code-and-retur... EDIT: found the story, and i was wrong about the sleeve looks like it was a regular jewel case, but still.
degenerate 9y ago
Mirror, since the download server seems a bit sluggish: https://www.docdroid.net/f1tjUfR/starcraft-eud-emulator.pdf https://www.docdroid.net/f1tjUfR/starcraft-eud-emulator.pdf
thrw94858 9y ago
Does anybody know which text editor is being used in his Python/C++ screenshots?
- ctrl-j 9y ago
  Looks like Visual Studio Code - Microsoft's cross platform code editor.
- modeless 9y ago
  https://www.sourceinsight.com https://www.sourceinsight.com
partycoder 9y ago
To have some context, this is one of the custom maps created using the hack: https://www.youtube.com/watch?v=yINtyK9YNik https://www.youtube.com/watch?v=yINtyK9YNik It is incredible the extent to which the game got hacked to make this possible.
- deleted 9y ago
  [deleted]
sneak 9y ago
That first slide after the title makes me sad for the author. “Don’t reverse engineer our apps. Also, here is why we didn’t patch things, because they would break tremendous value delivered to our platform by people who reverse engineered our apps.” Y’all remember when Blizzard sued those OSS devs for reimplementing their server protocol in bnetd?
sneak 9y ago
Anyone else’s eye see DOS window drawing characters in the editor and subconsciously try to parse Hangul for a second before realizing?
duskwuff 9y ago
Warcraft 3 had a similar bug in its scripting engine for a while -- a lack of type checking on return values made it possible to cast between incompatible types, allowing some of the complex types exposed to the scripting engine to be manipulated in unexpected and useful ways. Much like the EUD bug, this was used by mapmakers, but was patched by Blizzard after it was exploited to run arbitrary code. A workaround for one of the more common use cases (hash tables) was later re-added, but the maps that made use of this bug were permanently broken. http://jass.sourceforge.net/doc/retbug.shtml http://jass.sourceforge.net/doc/retbug.shtml
etiam 9y ago
Not the main point, but those are some very nice illustrations. Anybody know where they're from? Is there an art book?
- EamonnMR 9y ago
  In the remake they added those illustrations to spice up the explanatory paragraphs that explained the transition between some missions.
crummy 9y ago
This is the case for why sometimes when it comes to backwards compatibility you have to say no and break things. Except Blizzard put in a huge amount of work to maintain compatibility anyway. Really impressive
jchw 9y ago
I'm astounded how much effort was put into emulating this. It's damn near heroic levels of effort. The most tragic part is, most users won't even know. There are probably a few EUD maps out there that don't run on SC:R and everyone probably just thinks Blizzard is breaking things unnecessarily. But more than that, I would love to hear perspective from those who exploited this bug. At the very least, it must be amusing to see some of these things still working even though they really ought not. I mean, a buffer overflow read/write primitive reading and writing from and to data structures that no longer exist? That's really something. After enough years, a bug stops being a bug and starts being part of the personality of a piece of software or hardware. I like that they cared. And I'm sure, for sake of the some-17k maps making use of it, users will too. I don't play StarCraft, but I'll say this has my interest piqued almost enough to consider buying this and checking it out. At $15, it isn't too hard of a sell, especially knowing the care put into it.
- taneq 9y ago
  > After enough years, a bug stops being a bug and starts being part of the personality of a piece of software or hardware. I'd say this qualifies as an Ascended Bug / Ascended Glitch: http://tvtropes.org/pmwiki/pmwiki.php/Main/AscendedGlitch http://tvtropes.org/pmwiki/pmwiki.php/Main/AscendedGlitch
  - zawerf 9y ago
    A lot of starcraft is like that. One of the developers wrote about how his quick and dirty hack accidentally became an iconic technique (worker drilling): https://www.codeofhonor.com/blog/the-starcraft-path-finding-hack https://www.codeofhonor.com/blog/the-starcraft-path-finding-...
TeMPOraL 9y ago
Hm. I was into random-memory-access maps back when (I believe) they were first discovered; we didn't use EUD, but simple resource triggers - e.g. "For player $absurdNumber[0], increase Vespene Gas to $someWeirdValue". It probably wasn't as flexible as the bug described in the presentation, but still allowed for some fun - like runtime terrain changes, or runtime weapon changes (like gun -> nuke). AFAIR that loophole was closed within few weeks of showing up, with patch 1.13e. Unrelated: > StarCraft Remastered collects game telemetry (including map information, etc.) Yes. Of course it does. Fuck the Internet era. -- [0] - AFAIR only legitimately usable players in StarCraft were 1-8, player 12 was for "Neutral" critters & stuff; anything above 256 was beyond what the game has reserved space for, and so if you asked for the data of such high-number players, you were accessing unrelated game memory.
- ygra 9y ago
  >> StarCraft Remastered collects game telemetry (including map information, etc.) > Yes. Of course it does. > Fuck the Internet era. Well, to be honest, everything played on Battle.net is visible to Blizzard already and especially now that they support hosting games even when not reachable directly from the internet, there's even more data available (AFAIK map files were only transferred P2P in the old times, and the Battle.net servers didn't care about those at all).
- ixwt 9y ago
  > AFAIR that loophole was closed within few weeks of showing up, with patch 1.13e. Yes, they did fix it. They made it so you could no longer write using EUDs, but you could still read from what I remember (it's been quite a while, I remember experimenting using EUDs). > AFAIR only legitimately usable players in StarCraft were 1-8, player 12 was for "Neutral" critters & stuff; anything above 256 was beyond what the game has reserved space for, and so if you asked for the data of such high-number players, you were accessing unrelated game memory. Yup, though you could set a colors players using a non-default map editor. Doing so would cause odd coloring of units. AFAIR after player 12, the colors weren't explicitly colored. So you got mostly black, or whatever coloring was in the users memory at the time.
dkonofalski 9y ago
On a side note, I'm really excited for this release. I played tons of StarCraft and Brood War multiplayer at LANs growing up but never really got around to playing the single-player stories. Now, those games don't really run well on modern hardware and there's not even a Mac version available (the original was for the PowerPC architecture). It's nice to think that I'll be able to catch up on this and have a modern experience with it.

249 points by jsnell 9y ago

Awesome work! The community really is what makes SCBW great, allowing the use of the EUD maps again is part of what makes that tick. Now if Blizzard would just let ASL5 happen...

my_usernam3 9y ago
I'm not convinced Blizzard is the only culprit for the ASL delay. I know there's the 80k licensing fee, but afreeka didn't have to put them on blast in public. Negotiations should have happened behind closed doors.
- ChrisRackauckas 9y ago
  Blizzard is paying Afreeca to host the GSL. Blizzard is paying others for Overwatch and Hearthstone tournaments. There's no reason for Blizzard to force organizations to pay to run a SCBW tournament. If Blizzard actually has a defendable position, they should share it.
krzyk 9y ago
What is ASL5? or EUD?
- s_m_t 9y ago
  Afreeca Star League 5. The largest remaining Starcraft BW tournament. Extended Unit Death. A buffer overflow exploit that massively increased the power of the map editor's "scripting language" (called "triggers" in the map editor). It was called Extended Unit Death because the trigger used was intended for the map maker to specify some sort of action upon a player having a certain number of units die. By overflowing the unit field in this trigger you could read from any memory address and by doing the same to another set of triggers that manipulated the death count you could write to any memory address. It was patched out in 1.13 for obvious reasons, however, you can re-enable EUD functionality with custom launchers to play maps that take advantage of the exploits.
- deleted 9y ago
  [deleted]
- sanxiyn 9y ago
  EUD is the name of the buffer overflow whose emulation is discussed in the linked presentation.

a_t48 9y ago

This is really cool. It makes me wish I were still doing reverse engineering stuff for games...or that I had been smart enough to make a presentation on it while I still was.

voltagex_ 9y ago
You could still make a blog post or presentation on what you did.
- a_t48 9y ago
  It's been far too long, unfortunately. I'm not even sure I still have the IDA database.

voltagex_ 9y ago

"Unfortunately, we did not have private or public symbols for StarCraft 1.16.1. I had to start reversing the game executable from scratch" So much history is lost because game companies don't archive their assets. * http://au.ign.com/articles/2013/06/27/original-kingdom-hearts-assets-lost http://au.ign.com/articles/2013/06/27/original-kingdom-heart... * https://adventuregamers.com/forums/viewthread/7766 https://adventuregamers.com/forums/viewthread/7766 * I also thought that the original Okami assets were lost with the closure of Clover Studio but I can't find a source.

StillBored 9y ago
In this case it sounds like they just lost the build artifacts, not the actual assets. Although, when you read about how people have recovered the source for old programs it makes complete sense. Without a central location where all the IP is stored, it becomes easy for older pieces to fall off the radar during company transitions, system upgrades, whatever. I suspect as people move away from large company wide SMB/NFS shares this is once again going to start being a common story. Particularly as people try to shrink their online storage footprint and move things to "archive" only. I've worked at places where I've had to repeating justify why i'm eating up a $LARGE_NUMBER GB for the last $SOME_NUMBER released versions of the product because the build process triggers a complete archive of the entire artifacts directories when a release build is specified. Online storage be it, S3 buckets, or LTO backed NFS shares have real costs associated with them, and invariably IT comes along and starts asking why they have to upgrade their systems, or justify their OPEX numbers. Having a few TB of build artifacts that are rarely (if ever as the versions get older) referenced sits right up there with keeping previous employee's machine images, or home directories live. The problem though is that putting things in an offline archive is just the first step in forgetting it exits. At that point your basically saying that in 10-15 years the only remaining copy of the data is going to be the git repo on some guys laptop sitting in his attic (the modern version of the floppies in shoeboxes).
stryk 9y ago
I remember reading about a story[1] last summer of a user on reddit buying a box full of "Blizzard stuff" on eBay and in it he found a gold cd-rom that was the master source code disk for Starcraft -- this wasn't even a hand-written label type thing either, it was ink-or-laser jet printed and had a custom sleeve and all. How does something like that end up in a box of junk for sale on eBay... crazy world. I think he ended up not ripping the contents, to the chagrin of many (including people advocating for source archival, but who knows how many were just using that as a cover and what they really wanted was to cheat at the game), and sending it back to blizzard and I think they flew him out to their game conference as a thank you. 1: https://kotaku.com/guy-finds-starcraft-source-code-and-returns-it-to-blizz-1794897125 https://kotaku.com/guy-finds-starcraft-source-code-and-retur... EDIT: found the story, and i was wrong about the sleeve looks like it was a regular jewel case, but still.

degenerate 9y ago

Mirror, since the download server seems a bit sluggish: https://www.docdroid.net/f1tjUfR/starcraft-eud-emulator.pdf https://www.docdroid.net/f1tjUfR/starcraft-eud-emulator.pdf

thrw94858 9y ago

Does anybody know which text editor is being used in his Python/C++ screenshots?

ctrl-j 9y ago
Looks like Visual Studio Code - Microsoft's cross platform code editor.
modeless 9y ago
https://www.sourceinsight.com https://www.sourceinsight.com

partycoder 9y ago

To have some context, this is one of the custom maps created using the hack: https://www.youtube.com/watch?v=yINtyK9YNik https://www.youtube.com/watch?v=yINtyK9YNik It is incredible the extent to which the game got hacked to make this possible.

deleted 9y ago
[deleted]

sneak 9y ago

That first slide after the title makes me sad for the author. “Don’t reverse engineer our apps. Also, here is why we didn’t patch things, because they would break tremendous value delivered to our platform by people who reverse engineered our apps.” Y’all remember when Blizzard sued those OSS devs for reimplementing their server protocol in bnetd?

sneak 9y ago

Anyone else’s eye see DOS window drawing characters in the editor and subconsciously try to parse Hangul for a second before realizing?

duskwuff 9y ago

Warcraft 3 had a similar bug in its scripting engine for a while -- a lack of type checking on return values made it possible to cast between incompatible types, allowing some of the complex types exposed to the scripting engine to be manipulated in unexpected and useful ways. Much like the EUD bug, this was used by mapmakers, but was patched by Blizzard after it was exploited to run arbitrary code. A workaround for one of the more common use cases (hash tables) was later re-added, but the maps that made use of this bug were permanently broken. http://jass.sourceforge.net/doc/retbug.shtml http://jass.sourceforge.net/doc/retbug.shtml

etiam 9y ago

Not the main point, but those are some very nice illustrations. Anybody know where they're from? Is there an art book?

EamonnMR 9y ago
In the remake they added those illustrations to spice up the explanatory paragraphs that explained the transition between some missions.

crummy 9y ago

This is the case for why sometimes when it comes to backwards compatibility you have to say no and break things. Except Blizzard put in a huge amount of work to maintain compatibility anyway. Really impressive

jchw 9y ago

I'm astounded how much effort was put into emulating this. It's damn near heroic levels of effort. The most tragic part is, most users won't even know. There are probably a few EUD maps out there that don't run on SC:R and everyone probably just thinks Blizzard is breaking things unnecessarily. But more than that, I would love to hear perspective from those who exploited this bug. At the very least, it must be amusing to see some of these things still working even though they really ought not. I mean, a buffer overflow read/write primitive reading and writing from and to data structures that no longer exist? That's really something. After enough years, a bug stops being a bug and starts being part of the personality of a piece of software or hardware. I like that they cared. And I'm sure, for sake of the some-17k maps making use of it, users will too. I don't play StarCraft, but I'll say this has my interest piqued almost enough to consider buying this and checking it out. At $15, it isn't too hard of a sell, especially knowing the care put into it.

taneq 9y ago
> After enough years, a bug stops being a bug and starts being part of the personality of a piece of software or hardware. I'd say this qualifies as an Ascended Bug / Ascended Glitch: http://tvtropes.org/pmwiki/pmwiki.php/Main/AscendedGlitch http://tvtropes.org/pmwiki/pmwiki.php/Main/AscendedGlitch
- zawerf 9y ago
  A lot of starcraft is like that. One of the developers wrote about how his quick and dirty hack accidentally became an iconic technique (worker drilling): https://www.codeofhonor.com/blog/the-starcraft-path-finding-hack https://www.codeofhonor.com/blog/the-starcraft-path-finding-...

TeMPOraL 9y ago

Hm. I was into random-memory-access maps back when (I believe) they were first discovered; we didn't use EUD, but simple resource triggers - e.g. "For player $absurdNumber[0], increase Vespene Gas to $someWeirdValue". It probably wasn't as flexible as the bug described in the presentation, but still allowed for some fun - like runtime terrain changes, or runtime weapon changes (like gun -> nuke). AFAIR that loophole was closed within few weeks of showing up, with patch 1.13e. Unrelated: > StarCraft Remastered collects game telemetry (including map information, etc.) Yes. Of course it does. Fuck the Internet era. -- [0] - AFAIR only legitimately usable players in StarCraft were 1-8, player 12 was for "Neutral" critters & stuff; anything above 256 was beyond what the game has reserved space for, and so if you asked for the data of such high-number players, you were accessing unrelated game memory.

ygra 9y ago
>> StarCraft Remastered collects game telemetry (including map information, etc.) > Yes. Of course it does. > Fuck the Internet era. Well, to be honest, everything played on Battle.net is visible to Blizzard already and especially now that they support hosting games even when not reachable directly from the internet, there's even more data available (AFAIK map files were only transferred P2P in the old times, and the Battle.net servers didn't care about those at all).
ixwt 9y ago
> AFAIR that loophole was closed within few weeks of showing up, with patch 1.13e. Yes, they did fix it. They made it so you could no longer write using EUDs, but you could still read from what I remember (it's been quite a while, I remember experimenting using EUDs). > AFAIR only legitimately usable players in StarCraft were 1-8, player 12 was for "Neutral" critters & stuff; anything above 256 was beyond what the game has reserved space for, and so if you asked for the data of such high-number players, you were accessing unrelated game memory. Yup, though you could set a colors players using a non-default map editor. Doing so would cause odd coloring of units. AFAIR after player 12, the colors weren't explicitly colored. So you got mostly black, or whatever coloring was in the users memory at the time.

dkonofalski 9y ago

On a side note, I'm really excited for this release. I played tons of StarCraft and Brood War multiplayer at LANs growing up but never really got around to playing the single-player stories. Now, those games don't really run well on modern hardware and there's not even a Mac version available (the original was for the PowerPC architecture). It's nice to think that I'll be able to catch up on this and have a modern experience with it.