6 ms·

Three lessons I learnt from porting Diablo

83 points by petewarden 17y ago

ghshephard 17y ago
Has anyone ever seen this before? "There were hundreds of pieces of x86 assembler scattered throughout the code base, which was a problem since we were porting to the Playstation's MIPS processor. Usually just a couple of instructions long, and in the middle of functions, these snippets were pretty puzzling. Finally one of the team figured it out; somebody had struggled with C's signed/unsigned casting rules, and so they'd fallen back on the assembler instructions they understood! The whole team had a good laugh at that"
- jrbedard 17y ago
  On one heavily optimized 3D CAD software that I've worked on, I've seen a hundreds of x86/PPC assembly snippets in #ifdefs, embedded in the C code. It was basically for the software to be very fast on Windows and on Mac OS X PowerPCs. Some of those were legacy optimizations, because modern C++ compilers (VS 2003 and up) are smart enough to match that speed gain with better binary generation. But some other assembly parts were truly meant to recreate a customized and fast GPU-like pipeline on the CPU, using registers for really fast per-pixel/voxel operations. I've heard that Photoshop is also riddled with assembly code, that's why it took so long for Adobe to port it full-performance for the new Intel-based Macs in 2006. (Lots of PPC -> x86 mini-ports)
- benatkin 17y ago
  That's such a great bite-sized tale. Reminds me of http://folklore.org/ http://folklore.org/.
- elblanco 17y ago
  I've seen it for parts of code that need to be particularly performant. In the old days, when clock cycles were at a premium and compiler optimizers stunk, great hackers could squeeze an extra 10-30% out of a piece of code by rebuilding certain expensive parts of code in asm instead of relying on the compiler's version. Usually it involved a particularly arcade knowledge of the chip's timing or pipelining rules so that there were no wasted clock cycles, cache misses, or minimizing branch prediction errors....that kind of thing.
  - lt 17y ago
    That's writing a whole routine in asm, though. Embedding a couple lines of asm in a function can have the opposite effect - the optimizer can no longer make assumptions about some things disabling a large number of potential optimizations for that routine.
    - samlittlewood 17y ago
      Fortunately, the Watcom compiler (tool of choice at the time) had a mechanism for describing register use and side effects of embedded code fragments - somewhat like gcc. It was common for any 3d games to have 16.16 fixed point type all wrapped up in macros using asm fragments. The crucial thing was using a 64 bit intermediate for muldiv and mulacc expressions.
- samlittlewood 17y ago
  Yeah, I recall people doing similar with 'fistp' to get quicker float->int conversions. (There are quicker tricks using direct manipulation of the fp representation, but those have always had the decency to be hidden in macros). Also, I recall assembler code (65816) where the programmers had directly encoded instructions (DB $aa,$bb) since an earlier (probably 6502) assembler had not supported them. In those days, gamers programmers had more in common with scientists in their attitude to programming - they made games, programming was simply a means to that end. Also, games had pretty much zero maintenance after release.
- mahmud 17y ago
  Yes, it's not that unusual for DOS programmers to "fall back" to assembly; inline assembly was much cleaner and far more effective than the disparate OS-hooks built into the various compilers. You could do asm { pushf, pop ax } to save the flags register, and do whatever you want with the machine (e.g. masking interrupts) but to do the same in pure C you would have had to use MS, Borland or Watcom extensions (yes, inline assembly was more portable and stable than the various levels of C standardization supported by the compilers.) Other fun ones were variable length structs, and bitfields. http://www.ddj.com/cpp/184403480 http://www.ddj.com/cpp/184403480
- ghshephard 17y ago
  I'm sorry - I should have been more clear - I _get_ that you inline assembly when you need to do something your Compiler won't let you do (or makes difficult), and that clearly people sometimes will tweak some inline assembly to get a bit more performance - But has anyone ever seen a developer "Struggle with C's casting rules" - and switch to Assembler to over come that challenge? That just seems like an unexpected approach to that problem of understanding a kind of fundamental C concept...
  - gvb 17y ago
    No, I've never seen someone drop to assy due to something as simple as not understanding typecasting. Historically it has been for speed (used to be common, a lot less common now) or to do something the compiler/language doesn't support. The anecdote falls into the latter class, but only because the original code author didn't understand C well enough. When I started programming, the running joke was "you can write FORTRAN in any language, some are just more difficult."
  - wmf 17y ago
    One of my coworkers commonly uses pointer math rather than array indexing, so inline asm isn't that much of a stretch. Some C programmers are just thinking in assembly.
- DLWormwood 17y ago
  Yup, right here... http://news.ycombinator.com/item?id=975575 http://news.ycombinator.com/item?id=975575 I wonder if my response to that post triggered him making that blog for clarification. (Basically rebutting my quality complaint.)
elblanco 17y ago
My lessons from his writeup seem to be completely different: 1) Get a paycheck I can live on. 2) Get a mentor. 3) You don't have to finish a job before moving on if things aren't working out. (not that I'd agree with #3, that just seems to be my takeaway from the story).
- Flankk 17y ago
  My takeaway was this: "The third lesson I learnt was that you don't need great code to make a great product." I really believe this to be true. Coders are far too analytical for their own good. If a product works great, then the poor code can be fixed later.
  - berntb 17y ago
    I do admire the ability to ship. But some of the ones with that ability would be better off a little bit less goal driven and spending just a few hours a month reading up on their tools. (And the rest of the world would be better off, too...) Edit: (A bit clearer syntax.) I might add -- it is OK to do evil stuff if you are in a hurry, but at least isolate those part so they are easily replaceable. (Yes, most everyone should know this.)
  - barrkel 17y ago
    Games are somewhat of a special case, though - as long as it works enough for the vast majority of players to complete the game without hitting a roadblocker issue, it's good enough. It doesn't need to withstand longterm maintenance.
  - elblanco 17y ago
    You know, this is really true. My company has recently been making the slow switch from "over-designed and implemented, resulting in a very slow release cycle" to "minimally designed, quickly (but smartly) implemented, fast release cycles". This comes from two lessons to us: 1) No matter how long you spend designing a feature, it'll probably not be 100% of what the user wants. Do a quicker "draft" of the feature, solicit feedback, and release fast to fulfill customer needs. 2) No matter how long you spend trying to write elegant code, the software will have bugs, and other kinds of problems. Do it quick, observe it in the field, take notes and do bug patches. Getting the team on-board for these was hard due to the resistance of this being the perceived Microsoft way of doing things, but it has resulted in much higher customer satisfaction because they see continuous progress and feel like their feedback is turning into product (they feel part of the process) rather than tens of months of no progress followed by something they probably didn't want to begin with.
joss82 17y ago
"sometimes the worst projects are the best teachers." That's so true. At least for me.
- pgebhard 17y ago
  I agree, but you definitely wouldn't want to keep jumping from bad project to bad project. The end goal should be to take that very useful experience and turn it into a very successful project.
dazzawazza 17y ago
Unfortunately this is a very common introduction to the Video games industry. At least in the UK. The industry trades on how 'cool' it is to work on games knowing that there are hundreds of talented kids who will do anything to get in to it. They often end up doing poorly paid difficult work under tight deadlines with bosses that make The Office seems like heaven. Unfortunately people like Liddon are few and far between within the industry.

83 points by petewarden 17y ago

Has anyone ever seen this before? "There were hundreds of pieces of x86 assembler scattered throughout the code base, which was a problem since we were porting to the Playstation's MIPS processor. Usually just a couple of instructions long, and in the middle of functions, these snippets were pretty puzzling. Finally one of the team figured it out; somebody had struggled with C's signed/unsigned casting rules, and so they'd fallen back on the assembler instructions they understood! The whole team had a good laugh at that"

jrbedard 17y ago
On one heavily optimized 3D CAD software that I've worked on, I've seen a hundreds of x86/PPC assembly snippets in #ifdefs, embedded in the C code. It was basically for the software to be very fast on Windows and on Mac OS X PowerPCs. Some of those were legacy optimizations, because modern C++ compilers (VS 2003 and up) are smart enough to match that speed gain with better binary generation. But some other assembly parts were truly meant to recreate a customized and fast GPU-like pipeline on the CPU, using registers for really fast per-pixel/voxel operations. I've heard that Photoshop is also riddled with assembly code, that's why it took so long for Adobe to port it full-performance for the new Intel-based Macs in 2006. (Lots of PPC -> x86 mini-ports)
benatkin 17y ago
That's such a great bite-sized tale. Reminds me of http://folklore.org/ http://folklore.org/.
elblanco 17y ago
I've seen it for parts of code that need to be particularly performant. In the old days, when clock cycles were at a premium and compiler optimizers stunk, great hackers could squeeze an extra 10-30% out of a piece of code by rebuilding certain expensive parts of code in asm instead of relying on the compiler's version. Usually it involved a particularly arcade knowledge of the chip's timing or pipelining rules so that there were no wasted clock cycles, cache misses, or minimizing branch prediction errors....that kind of thing.
- lt 17y ago
  That's writing a whole routine in asm, though. Embedding a couple lines of asm in a function can have the opposite effect - the optimizer can no longer make assumptions about some things disabling a large number of potential optimizations for that routine.
  - samlittlewood 17y ago
    Fortunately, the Watcom compiler (tool of choice at the time) had a mechanism for describing register use and side effects of embedded code fragments - somewhat like gcc. It was common for any 3d games to have 16.16 fixed point type all wrapped up in macros using asm fragments. The crucial thing was using a 64 bit intermediate for muldiv and mulacc expressions.
samlittlewood 17y ago
Yeah, I recall people doing similar with 'fistp' to get quicker float->int conversions. (There are quicker tricks using direct manipulation of the fp representation, but those have always had the decency to be hidden in macros). Also, I recall assembler code (65816) where the programmers had directly encoded instructions (DB $aa,$bb) since an earlier (probably 6502) assembler had not supported them. In those days, gamers programmers had more in common with scientists in their attitude to programming - they made games, programming was simply a means to that end. Also, games had pretty much zero maintenance after release.
mahmud 17y ago
Yes, it's not that unusual for DOS programmers to "fall back" to assembly; inline assembly was much cleaner and far more effective than the disparate OS-hooks built into the various compilers. You could do asm { pushf, pop ax } to save the flags register, and do whatever you want with the machine (e.g. masking interrupts) but to do the same in pure C you would have had to use MS, Borland or Watcom extensions (yes, inline assembly was more portable and stable than the various levels of C standardization supported by the compilers.) Other fun ones were variable length structs, and bitfields. http://www.ddj.com/cpp/184403480 http://www.ddj.com/cpp/184403480
ghshephard 17y ago
I'm sorry - I should have been more clear - I _get_ that you inline assembly when you need to do something your Compiler won't let you do (or makes difficult), and that clearly people sometimes will tweak some inline assembly to get a bit more performance - But has anyone ever seen a developer "Struggle with C's casting rules" - and switch to Assembler to over come that challenge? That just seems like an unexpected approach to that problem of understanding a kind of fundamental C concept...
- gvb 17y ago
  No, I've never seen someone drop to assy due to something as simple as not understanding typecasting. Historically it has been for speed (used to be common, a lot less common now) or to do something the compiler/language doesn't support. The anecdote falls into the latter class, but only because the original code author didn't understand C well enough. When I started programming, the running joke was "you can write FORTRAN in any language, some are just more difficult."
- wmf 17y ago
  One of my coworkers commonly uses pointer math rather than array indexing, so inline asm isn't that much of a stretch. Some C programmers are just thinking in assembly.
DLWormwood 17y ago
Yup, right here... http://news.ycombinator.com/item?id=975575 http://news.ycombinator.com/item?id=975575 I wonder if my response to that post triggered him making that blog for clarification. (Basically rebutting my quality complaint.)

elblanco 17y ago

My lessons from his writeup seem to be completely different: 1) Get a paycheck I can live on. 2) Get a mentor. 3) You don't have to finish a job before moving on if things aren't working out. (not that I'd agree with #3, that just seems to be my takeaway from the story).

Flankk 17y ago
My takeaway was this: "The third lesson I learnt was that you don't need great code to make a great product." I really believe this to be true. Coders are far too analytical for their own good. If a product works great, then the poor code can be fixed later.
- berntb 17y ago
  I do admire the ability to ship. But some of the ones with that ability would be better off a little bit less goal driven and spending just a few hours a month reading up on their tools. (And the rest of the world would be better off, too...) Edit: (A bit clearer syntax.) I might add -- it is OK to do evil stuff if you are in a hurry, but at least isolate those part so they are easily replaceable. (Yes, most everyone should know this.)
- barrkel 17y ago
  Games are somewhat of a special case, though - as long as it works enough for the vast majority of players to complete the game without hitting a roadblocker issue, it's good enough. It doesn't need to withstand longterm maintenance.
- elblanco 17y ago
  You know, this is really true. My company has recently been making the slow switch from "over-designed and implemented, resulting in a very slow release cycle" to "minimally designed, quickly (but smartly) implemented, fast release cycles". This comes from two lessons to us: 1) No matter how long you spend designing a feature, it'll probably not be 100% of what the user wants. Do a quicker "draft" of the feature, solicit feedback, and release fast to fulfill customer needs. 2) No matter how long you spend trying to write elegant code, the software will have bugs, and other kinds of problems. Do it quick, observe it in the field, take notes and do bug patches. Getting the team on-board for these was hard due to the resistance of this being the perceived Microsoft way of doing things, but it has resulted in much higher customer satisfaction because they see continuous progress and feel like their feedback is turning into product (they feel part of the process) rather than tens of months of no progress followed by something they probably didn't want to begin with.

joss82 17y ago

"sometimes the worst projects are the best teachers." That's so true. At least for me.

pgebhard 17y ago
I agree, but you definitely wouldn't want to keep jumping from bad project to bad project. The end goal should be to take that very useful experience and turn it into a very successful project.

dazzawazza 17y ago

Unfortunately this is a very common introduction to the Video games industry. At least in the UK. The industry trades on how 'cool' it is to work on games knowing that there are hundreds of talented kids who will do anything to get in to it. They often end up doing poorly paid difficult work under tight deadlines with bosses that make The Office seems like heaven. Unfortunately people like Liddon are few and far between within the industry.