4 ms·
A 15% framerate increase from 3 weeks of work isn't particularly impressive. It would be more meaningful if the measurement was in terms of frametime (i.e. elap
by kevingadd 14y ago
A 15% framerate increase from 3 weeks of work isn't particularly impressive. It would be more meaningful if the measurement was in terms of frametime (i.e. elapsed time per frame) since framerate is not linear.
From looking at the patch (nice of them to provide it), it seems like most of the changes are putting mutexes around things to guard against simultaneous access and then parallelizing some loops. It seems to me like anyone with basic knowledge of the game's codebase could have made these changes without any support from a tool like the one they're trying to sell, and probably done it in far less than 3 weeks. If your goal is to run a piece of code in parallel, you don't need a program to walk through that code and identify all the shared state it manipulates (and then go through and make sure that state has a mutex guard). If the software they're trying to sell did that for you, now that would be nice. It seems unlikely that you could automatically modify C++ source code without a lot of constraints, though.
Software to aid authoring parallel C++, or detect places where you can safely parallelize existing code, would certainly be valuable but this is a pretty poor demonstration of any actual advantages offered by their software.
EDIT: From watching the included demo video, the game is running well below 60fps which suggests that they're running it on a low end graphics card (or generally low-clocked machine) to improve the gains from their optimizations. I don't think their results are necessarily real-world as a result of this - someone playing Doom 3 on an actual gaming rig would probably not see a 15% framerate increase because their framerate would already be well above 60fps (framerate is nonlinear, as I mentioned above).
- jspthrowaway 14y agoCame for the dismissive hand-waving that I fully expected to be the top comment and I wasn't disappointed. What's the improvement factor where you'd be impressed? 50%? 60%? It's remarkable in my opinion how little work it took to improve upon an already very-optimized engine, in the constraints that they set for the developer, by a developer who's never worked with game engines in his career. Edit: No longer the top comment; it was when I commented.
- kevingadd 14y agoFramerate isn't a comparative measurement. You have to measure elapsed time. Instead of spending 3 weeks wrapping data structures in mutexes for a 15% framerate increase, you could probably get an equivalent speedup by reducing shader detail, texture size, or any number of other options. This kind of measurement is not meaningful without additional information: Is this improved performance for low-spec GPUs? High-spec CPUs? universal performance improvement for all users? Low end machines typically do not have lots of cores; will they actually benefit tremendously from parallelism? As I said before, a high-spec machine already runs Doom 3 fine, so I question whether the gains here would actually be significant at all. EDIT: Also, gains of this size can easily be produced by changes as simple as turning on Profile-Guided Optimization in your compiler. We'd also need to know what settings they used to compile the game to know whether the results are actually realistic.
- jspthrowaway 14y ago> Instead of spending 3 weeks wrapping data structures in mutexes for a 15% framerate increase, you could probably get an equivalent speedup by reducing shader detail, texture size, or any number of other options. I see why we differ in opinion here: you feel that regressing backwards and sacrificing visual quality is a better approach than parallelizing the same exact engine with the same quality. I don't think we're going to see eye-to-eye on this one.
- kevingadd 14y agoPC game development is about delivering a good experience for users on a wide range of hardware configurations. A given game is not going to automatically run perfect on every hardware configuration. When considering performance issues for a particular customer, you have to weigh the cost of improvement. 3 weeks of an engineer's time for 15% speedup (even if it were in elapsed frame time, which it's not) is NOT an easy decision. Those 3 weeks could be spent doing far more valuable things, like improving the tools used by the rest of the development team, fixing crash bugs, or working on downloadable content that will bring in more money for the studio. They could also be spent making architectural changes that provide much larger performance wins through design improvements that require actual knowledge about the design of the application and the significance of the choices made. When you're doing performance optimization in game development, you go for the biggest, cheapest wins first, based on actual measurements and an understanding of what's wrong. Given that the optimized version of the game is not running at 60fps (and the demo video contains obvious rendering glitches) I don't think it's unfair to question whether CPU parallelism was the most obvious bottleneck here. Lastly, dropping texture size or shader detail when running on a low spec machine is not a regression. If the machine is not capable of running at max rendering detail due to GPU constraints, no degree of CPU parallelism will overcome this.
- ww520 14y agoWow. You really invoked some hates. All your long and thoughtful comments have been voted down. I would say you have hit a nerve.
- jspthrowaway 14y agoI, like pg and hopefully other Hacker News readers, am pretty sick of every story about something cool being immediately derailed by a know-it-all in an industry dismissing the innovation as "not cool enough". pg calls it middlebrow dismissal[1]. Rather than being treated as an imperfect launching-off point for further innovation, everything has to be world-changing perfect to have a reasonably sane comments section. I'm actually glad this thread got pushed down, so maybe some actually informative discussion can rise up rather than an analysis of how minutely the people behind this screwed up. [1]: http://news.ycombinator.com/item?id=4693920 http://news.ycombinator.com/item?id=4693920
- bitwize 14y agoWell, there are a lot of Apple fans here, and a core part of the Apple aesthetic is "if it isn't perfect and doesn't have that 'wow' factor, better to leave it out".
- taligent 14y agoThe only way you know if your product is imperfect is by other people constructively criticising it. But instead you would rather all of us stand around in a circle cheering them on about how wonderful it is and politely ignore any problems. Sounds like a guaranteed way to fail.
- jspthrowaway 14y agoDismissal is not constructive criticism. There's a sizable difference between "meh, nothing to see here" (which is the original comment in this thread) and "here's how this could be better".
- 14y ago
- jblow 14y agoThis is not just middlebrow dismissal, it's outright wrong. In the AAA world we would love to spend only 3 weeks of an engineer's time to get a 15% speedup. Seriously, that is a great deal, it's like, where do I sign up? However, it becomes substantially less impressive when you notice that you're using 2x or 4x the amount of processor hardware (2 or 4 cores) and only getting a 15% speedup. In a by-hand implementation that would be very disappointing. If it were fully automated that would still be pretty valuable, but it appears that this isn't. So it seems to be of questionable utility.
- FrankBooth 14y agoWhen were you in the AAA world, Jonathan?
- jblow 14y agoI did a lot of consulting / contract work at AAA companies between 2000-2005. "Parachute in and make the E3 demo work / implement some tough feature / etc". Sometimes it was less well-defined than that.
- confluence 14y agoWoah you're the guy that did Braid - glad to see you on HN :D Love to hear more about your 2000-2005 paratrooper work at the AAA shops - have you discussed that at all anywhere? I've always been interested in seeing how the really tough stuff gets done at AAA shops.
- groby_b 14y agoThe same way it gets done elsewhere - you find a person who's an expert at that kind of stuff and tell them to get it done :) The parachute-in often happens for teams that don't have a whole staff of low-level engineers around. Either small studios using an off-the-shelf engine (that can still be customized), or studios in a large conglomerate. (EA e.g. used to be quite happy to ship experts around. At least back in my day.)