6 ms·

"roll those loops back up and don't compile in debug mode to achieve significant performance gains" is not the takeaway I thought I'd have going in, heh.

infogulch 5y ago
Unrolled loops trade instruction cache for cycles. Whether it's good depends on what your bottleneck is, in this case ram access is a bigger bottleneck than cpu due to being shared with the rendering coprocessor. I wonder if we'll hit this crossover point again in the future.
- monocasa 5y ago
  And the code N64 was particularly unrolled for whatever reason. IIRC, there was one game that bit me when writing a JIT N64 emulator because there was a single code stream with no branches that was larger than multiple MMU pages. My JIT kind of assumed that actual code in practice wouldn't be larger than the actual N64 I$ without branching. Womp womp. But hey, I guess I don't want to be judged on my mid-90s code either.
  - phire 5y ago
    From what I understand (as I didn't start writing code until the mid-2000s), loop unrolling was a very highly regarded optimisation of the early 90s. And it was a very long time before programmers (and compilers) realised just how bad of an idea loop unrolling was once we entered the era of superscalar processors with instruction caches and half-decent branch prediction.
    - hugey010 5y ago
      It never occurred to me that loop unrolling could end up being a de-optimization due to compiler optimizations!
      - quietbritishjim 5y ago
        The things the parent comment referred to (instruction caches and branch prediction) are in the processor hardware rather than other compiler optimisations.
    - stonemetal12 5y ago
      To be fair early 90s branch prediction was garbage. Back then it seemed like every new processor release was accompanied by a note on how branch prediction was improved by 80-90% compared to its predecessor.
      - AstralStorm 5y ago
        To be even more fair, counted constant sized loops never required particularly good branch prediction, especially if the code is well laid out so that your "miss" does literally one more cheap step. Misprediction there means you computed one more vector or value, not a huge loss compared to a cache miss. Branch misprediction hurts a lot when it forces a cache miss mostly, whether it's for data or for instructions, or ties up a deep instruction pipeline completely with lousy abort, which most CPUs do not have. (See: Nehalem.)
  - nomel 5y ago
    > And the code N64 was particularly unrolled for whatever reason. My naive assumption is that they performed the majority of testing with debug code (as you would). And, since they had exactly one chance to get things right (being a physical cartridge), they couldn't afford to do all the testing again, with the new/different binaries generated with debug turned off.
    - monocasa 5y ago
      My understanding is that the massively unrolling loops thing had to do with how different branches felt versus previous consoles. Compared to filling an I$ line (somewhere in the many dozens of cycles on an N64), a branch mispredict was very cheap. However branch mispredicts were very visible. Delay slots were present in the ISA to decrease their cost and the pipeline stalls were noted in pretty much all literature on RISC at the time. On top of that, the N64 really didn't have great performance monitoring tooling, but was complex enough of a memory hierarchy to have really benefited from such tooling had it been available. It's pretty hard to understand the system effects pieces of code have without that.
astrange 5y ago
It was common belief in that era that you made code faster by making it longer and more complicated. The opposite is true now (because of cache size and deep OoO) - simple predictable loops are great - but even then people tended to overdo it. Of course, it's a way worse problem that they compiled at -O0.
- AussieWog93 5y ago
  >Of course, it's a way worse problem that they compiled at -O0. There was a Modern Vintage Gamer video going into this. Apparently it wasn't a mistake or misunderstanding, but a bug that existed in the original version of the compiler that caused issues when compiled with optimisation. Even then, the main library that did most of the heavy lifting was compiled with o2. Super Mario 64 was a launch title, and they simply didn't have the time to work out the kinks in the compiler to get those few extra fps in Dire Dire Docks.
  - ReactiveJelly 5y ago
    Yeah it's hard to remember the time before you could get 2 near-perfect C/C++ compilers on the 5-ish most popular architectures. And the architectures have consolidated since then.
  - pcwalton 5y ago
    As I recall, that MVG video was incorrect—Nintendo could have compiled with optimizations and the game would have worked. The most plausible explanation to me (and, IIRC, the one that Giles Goddard speculated was the case) is that they simply didn't want to take the chance that optimization could surface unknown bugs in the code, and they knew that the unoptimized version worked and played fine, so they played it safe and shipped the known-good unoptimized version. This is a perfectly understandable decision in terms of risk management, even if in retrospect it was too conservative. Less likely, but still very plausible, was that turning off optimization was just a mistake; mistakes happen when you're rushing a release for a deadline, and build deployment was not as orderly and automated back then as it is today.
    - 5y ago
butwhywhyoh 5y ago
He did so much more than that. The not compiling in debug mode was well known before this video, and he addresses that. His speedups are independent of not compiling in debug mode.
- phillipcarter 5y ago
  I know, and I hadn't appreciated all the ways that doing something less CPU-efficient was actually faster due to the way the hardware was architected. I just actually shouted when he said that if you simply compiled for release mode it ran 30% faster.

infogulch 5y ago

Unrolled loops trade instruction cache for cycles. Whether it's good depends on what your bottleneck is, in this case ram access is a bigger bottleneck than cpu due to being shared with the rendering coprocessor. I wonder if we'll hit this crossover point again in the future.

monocasa 5y ago
And the code N64 was particularly unrolled for whatever reason. IIRC, there was one game that bit me when writing a JIT N64 emulator because there was a single code stream with no branches that was larger than multiple MMU pages. My JIT kind of assumed that actual code in practice wouldn't be larger than the actual N64 I$ without branching. Womp womp. But hey, I guess I don't want to be judged on my mid-90s code either.
- phire 5y ago
  From what I understand (as I didn't start writing code until the mid-2000s), loop unrolling was a very highly regarded optimisation of the early 90s. And it was a very long time before programmers (and compilers) realised just how bad of an idea loop unrolling was once we entered the era of superscalar processors with instruction caches and half-decent branch prediction.
  - hugey010 5y ago
    It never occurred to me that loop unrolling could end up being a de-optimization due to compiler optimizations!
    - quietbritishjim 5y ago
      The things the parent comment referred to (instruction caches and branch prediction) are in the processor hardware rather than other compiler optimisations.
  - stonemetal12 5y ago
    To be fair early 90s branch prediction was garbage. Back then it seemed like every new processor release was accompanied by a note on how branch prediction was improved by 80-90% compared to its predecessor.
    - AstralStorm 5y ago
      To be even more fair, counted constant sized loops never required particularly good branch prediction, especially if the code is well laid out so that your "miss" does literally one more cheap step. Misprediction there means you computed one more vector or value, not a huge loss compared to a cache miss. Branch misprediction hurts a lot when it forces a cache miss mostly, whether it's for data or for instructions, or ties up a deep instruction pipeline completely with lousy abort, which most CPUs do not have. (See: Nehalem.)
- nomel 5y ago
  > And the code N64 was particularly unrolled for whatever reason. My naive assumption is that they performed the majority of testing with debug code (as you would). And, since they had exactly one chance to get things right (being a physical cartridge), they couldn't afford to do all the testing again, with the new/different binaries generated with debug turned off.
  - monocasa 5y ago
    My understanding is that the massively unrolling loops thing had to do with how different branches felt versus previous consoles. Compared to filling an I$ line (somewhere in the many dozens of cycles on an N64), a branch mispredict was very cheap. However branch mispredicts were very visible. Delay slots were present in the ISA to decrease their cost and the pipeline stalls were noted in pretty much all literature on RISC at the time. On top of that, the N64 really didn't have great performance monitoring tooling, but was complex enough of a memory hierarchy to have really benefited from such tooling had it been available. It's pretty hard to understand the system effects pieces of code have without that.

astrange 5y ago

It was common belief in that era that you made code faster by making it longer and more complicated. The opposite is true now (because of cache size and deep OoO) - simple predictable loops are great - but even then people tended to overdo it. Of course, it's a way worse problem that they compiled at -O0.

AussieWog93 5y ago
>Of course, it's a way worse problem that they compiled at -O0. There was a Modern Vintage Gamer video going into this. Apparently it wasn't a mistake or misunderstanding, but a bug that existed in the original version of the compiler that caused issues when compiled with optimisation. Even then, the main library that did most of the heavy lifting was compiled with o2. Super Mario 64 was a launch title, and they simply didn't have the time to work out the kinks in the compiler to get those few extra fps in Dire Dire Docks.
- ReactiveJelly 5y ago
  Yeah it's hard to remember the time before you could get 2 near-perfect C/C++ compilers on the 5-ish most popular architectures. And the architectures have consolidated since then.
- pcwalton 5y ago
  As I recall, that MVG video was incorrect—Nintendo could have compiled with optimizations and the game would have worked. The most plausible explanation to me (and, IIRC, the one that Giles Goddard speculated was the case) is that they simply didn't want to take the chance that optimization could surface unknown bugs in the code, and they knew that the unoptimized version worked and played fine, so they played it safe and shipped the known-good unoptimized version. This is a perfectly understandable decision in terms of risk management, even if in retrospect it was too conservative. Less likely, but still very plausible, was that turning off optimization was just a mistake; mistakes happen when you're rushing a release for a deadline, and build deployment was not as orderly and automated back then as it is today.
  - 5y ago

butwhywhyoh 5y ago

He did so much more than that. The not compiling in debug mode was well known before this video, and he addresses that. His speedups are independent of not compiling in debug mode.

phillipcarter 5y ago
I know, and I hadn't appreciated all the ways that doing something less CPU-efficient was actually faster due to the way the hardware was architected. I just actually shouted when he said that if you simply compiled for release mode it ran 30% faster.