11 ms·

I'd like to see more of this on HN, actually interesting technical content. The video is amazingly well put together. I don't know if it is sped up or not, my o

by rljy 9y ago

I'd like to see more of this on HN, actually interesting technical content. The video is amazingly well put together. I don't know if it is sped up or not, my only experiences with windows now adays are through my non-technical relatives computers which I'm asked to fix, but I was surprised at how quick windows appears to be in this video. But what bothers me, is the bug itself. It is shocking to me that someone set a hard limit on a pool and didn't add any code for replenishing it. I would just never do such a thing. One of the first programming environments that I used was MicroWorlds LOGO, and they had a cheaper version which allowed a maximum of 200 turtles, and a more expensive version that differed only in that the number of turtles was unlimited. My father got me the cheaper one and since then I've had a religious hatred of arbitrary limits in programs. But even if you didn't have this experience, I'd have a hard time respecting any programmer who would have left that pool unreplentished and called it a finished project. I accept the fact that we all make mistakes and make bugs, but that wasn't a mistake bug, that was consciously sucking out of laziness and not even bothering to gracefully handle failure.

revelation 9y ago
This is a console game, and consoles are always starved for memory since it's such a huge BOM cost so they put as little in as they can get away with. This ends up screwing the console game developers who have to start accounting for every single byte. So there are a lot of these things in games where they just run a script on all their assets and what not to figure out how much memory they will need for it, then allocate fixed pools at the start. These programmers would love to just throw that stuff into a vector but given their growth strategy you can end up wasting half your memory. Even worse, every element added into the vector might now cause it resize and when that happens during gameplay it might just ruin your 16 ms frame target. A fixed pool is as fast as you are going to get, no random chance your game is going to stutter and no waste. Here is a great talk on this: https://www.youtube.com/watch?v=8KIvWJUYbDA https://www.youtube.com/watch?v=8KIvWJUYbDA
- timthelion 9y ago
  Thanks, I learned something, though if they really were accounting for every byte, I'd say that function call obfuscation really didn't help! But I understand what you're getting at.
  - tgb 9y ago
    It's possible that's only in the PC version. I don't see anything on Google about Securom on xbox, so this seems likely.
teej 9y ago
This limit is anything but arbitrary. There's a few things to keep in mind: - This code was written for the XBox 360 which only has 512MB of memory. - Because the game was made for a console, it needs to go through the notorious Xbox certification process. This involves running the game for hours at a time at maximum load to ensure no memory leaks exist and that the game runs at a consistent 60 fps. - Games are not the same as long-living applications. They are made on a project basis. There are constant trade-offs where good enough is the correct decision. - This is a port, so there's care to be taken in reducing the footprint of changed code. It's unfortunate that this particular bug wasn't found before release but I think the root cause wasn't necessarily this pool size being low but instead insufficient testing.
- vvanders 9y ago
  Oh yeah. Hard limit pools are the bread and butter of console engines. You want to preallocate everything because fragmentation will murder you. Source: Ex-gamdev who's shipped things on x360, ps3 and psp.
  - zapperdapper 9y ago
    Interesting. Yes, fragmentation is a big problem on memory constrained systems. So the memory pool approach is quite a nice solution, but did you use caching allocators where memory could be allocated and marked as purgeable, so it could be reclaimed when memory gets low? Just curious. By the way do you know any good resources that cover memory management on consoles?
    - vvanders 9y ago
      The problem with making pools purgeable is you've now just moved your memory fragmentation problems up a level in the stack to the allocator. Now when you need to bring in a ~30mb level asset you might have one of those cells right in the middle again. For reference we usually split the psp 8MB/24MB System/Video+Audio so you'd have to get everything in < 8MB. For stuff that was highly dynamic we'd use Lua in a fixed block of 400kb. Usually that was enough to do all the game state and could be reset from level to level. Best public information I've seen on this stuff comes from Mike Acton, he's got lots of great stuff around Data Oriented Design which touches on this(and also why caches are so friggen important).
adamnemecek 9y ago
> how quick windows appears to be in this video. Your relatives probably have 2gb of memory and a i3 cpu or something similar.
- StudentStuff 9y ago
  Don't forget the spinning, humming & clicking rust they use rather than an SSD. One errant kid hits it too quickly and shatter goes those platters!
  - PhasmaFelis 9y ago
    SSDs have many advantages over rotary drives, but reliability is not one of them.
    - Klinky 9y ago
      I've yet to see any reliable evidence that SSDs are less reliable than hard drives. Plenty of anecdotes, but those also exist aplenty for hard drives. Personally I've had multiple hard drives fail on me, but no SSDs.
      - StudentStuff 9y ago
        I've seen early model SSDs fail (eg 60GB and 120GB consumer SSDs years ago), but an 850 Pro will keep on going for a long time.
      - gambiting 9y ago
        At my work I have to get a new SSD every 6-12 months because they just fail after writing around ~100TB to them, or if they don't straight up fail they become slow and start giving you read errors. I can write even a petabyte to a normal HDD and it will still work fine, but SSDs have a finite number of writes, and if you do what we do, you're going to run out within months rather than years.
        9y ago
gruez 9y ago
>but I was surprised at how quick windows appears to be in this video. SSD + disabling animations go a long way to increase "speediness"
- gambiting 9y ago
  Yep, I've recently set up a PC with an i5 750, so literally the first-gen i5. 4GB + Win 10, but it was running off an old 500GB hard drive - the machine was nearly unusable, things were taking minutes to launch. I ended up installing an older 60GB SSD into it and it's almost as usable as a modern computer, chrome starts within seconds of clicking the icon, everything is nice and responsive. I bet we could keep a lot of older computer going just fine if they had their hdds replaced with SSDs.
sclangdon 9y ago
You have to bare in mind that it's not really a bug in the original game. This "bug" only effects people modding the game outside of the original plan, and would never have caused a problem otherwise. I'd agree that it's sloppy coding, but I can also see it from the original developers point-of-view. Why check something you know isn't going to change?
- djmips 9y ago
  I agree with you but I wouldn't call it sloppy coding.

by rljy 9y ago

revelation 9y ago

This is a console game, and consoles are always starved for memory since it's such a huge BOM cost so they put as little in as they can get away with. This ends up screwing the console game developers who have to start accounting for every single byte. So there are a lot of these things in games where they just run a script on all their assets and what not to figure out how much memory they will need for it, then allocate fixed pools at the start. These programmers would love to just throw that stuff into a vector but given their growth strategy you can end up wasting half your memory. Even worse, every element added into the vector might now cause it resize and when that happens during gameplay it might just ruin your 16 ms frame target. A fixed pool is as fast as you are going to get, no random chance your game is going to stutter and no waste. Here is a great talk on this: https://www.youtube.com/watch?v=8KIvWJUYbDA https://www.youtube.com/watch?v=8KIvWJUYbDA

timthelion 9y ago
Thanks, I learned something, though if they really were accounting for every byte, I'd say that function call obfuscation really didn't help! But I understand what you're getting at.
- tgb 9y ago
  It's possible that's only in the PC version. I don't see anything on Google about Securom on xbox, so this seems likely.

teej 9y ago

This limit is anything but arbitrary. There's a few things to keep in mind: - This code was written for the XBox 360 which only has 512MB of memory. - Because the game was made for a console, it needs to go through the notorious Xbox certification process. This involves running the game for hours at a time at maximum load to ensure no memory leaks exist and that the game runs at a consistent 60 fps. - Games are not the same as long-living applications. They are made on a project basis. There are constant trade-offs where good enough is the correct decision. - This is a port, so there's care to be taken in reducing the footprint of changed code. It's unfortunate that this particular bug wasn't found before release but I think the root cause wasn't necessarily this pool size being low but instead insufficient testing.

vvanders 9y ago
Oh yeah. Hard limit pools are the bread and butter of console engines. You want to preallocate everything because fragmentation will murder you. Source: Ex-gamdev who's shipped things on x360, ps3 and psp.
- zapperdapper 9y ago
  Interesting. Yes, fragmentation is a big problem on memory constrained systems. So the memory pool approach is quite a nice solution, but did you use caching allocators where memory could be allocated and marked as purgeable, so it could be reclaimed when memory gets low? Just curious. By the way do you know any good resources that cover memory management on consoles?
  - vvanders 9y ago
    The problem with making pools purgeable is you've now just moved your memory fragmentation problems up a level in the stack to the allocator. Now when you need to bring in a ~30mb level asset you might have one of those cells right in the middle again. For reference we usually split the psp 8MB/24MB System/Video+Audio so you'd have to get everything in < 8MB. For stuff that was highly dynamic we'd use Lua in a fixed block of 400kb. Usually that was enough to do all the game state and could be reset from level to level. Best public information I've seen on this stuff comes from Mike Acton, he's got lots of great stuff around Data Oriented Design which touches on this(and also why caches are so friggen important).

adamnemecek 9y ago

> how quick windows appears to be in this video. Your relatives probably have 2gb of memory and a i3 cpu or something similar.

StudentStuff 9y ago
Don't forget the spinning, humming & clicking rust they use rather than an SSD. One errant kid hits it too quickly and shatter goes those platters!
- PhasmaFelis 9y ago
  SSDs have many advantages over rotary drives, but reliability is not one of them.
  - Klinky 9y ago
    I've yet to see any reliable evidence that SSDs are less reliable than hard drives. Plenty of anecdotes, but those also exist aplenty for hard drives. Personally I've had multiple hard drives fail on me, but no SSDs.
    - StudentStuff 9y ago
      I've seen early model SSDs fail (eg 60GB and 120GB consumer SSDs years ago), but an 850 Pro will keep on going for a long time.
    - gambiting 9y ago
      At my work I have to get a new SSD every 6-12 months because they just fail after writing around ~100TB to them, or if they don't straight up fail they become slow and start giving you read errors. I can write even a petabyte to a normal HDD and it will still work fine, but SSDs have a finite number of writes, and if you do what we do, you're going to run out within months rather than years.
      - 9y ago

gruez 9y ago

>but I was surprised at how quick windows appears to be in this video. SSD + disabling animations go a long way to increase "speediness"

gambiting 9y ago
Yep, I've recently set up a PC with an i5 750, so literally the first-gen i5. 4GB + Win 10, but it was running off an old 500GB hard drive - the machine was nearly unusable, things were taking minutes to launch. I ended up installing an older 60GB SSD into it and it's almost as usable as a modern computer, chrome starts within seconds of clicking the icon, everything is nice and responsive. I bet we could keep a lot of older computer going just fine if they had their hdds replaced with SSDs.

sclangdon 9y ago

You have to bare in mind that it's not really a bug in the original game. This "bug" only effects people modding the game outside of the original plan, and would never have caused a problem otherwise. I'd agree that it's sloppy coding, but I can also see it from the original developers point-of-view. Why check something you know isn't going to change?

djmips 9y ago
I agree with you but I wouldn't call it sloppy coding.