14 ms·

Thoughts on Modern C++ and Game Dev

150 points by lebek 8y ago

jayd16 8y ago
I don't want to weigh in on the rest of the content but the characterization of the game industry is pretty accurate in my experience. I would expand more on the first bullet point of why game devs don't test. Tests are anti-agile and game development is extremely agile. Usually you don't know what kind of game you're making until you're done.
- oselhn 8y ago
  That's not true. From my experience unit tests are great for agile. It will allow you to create "trusted" modules which you can move around and rework much easier (you can also treat your tests as executable documentation). Without tests you can't safely do any change to existing code especially if you are changing code written by someone else. You have to risk it and than spent considerable time in debugger if it breaks something unrelated to your feature.
  - coldtea 8y ago
    >That's not true. From my experience unit tests are great for agile. Not when you're constantly prototyping, which is what game dev essentially is for the most part of the process...
    - lifthrasiir 8y ago
      > which is what game dev essentially is for the most part of the process... Except that it isn't. You don't maintain the equal velocity of changes throughout the process, even for indie games. And there are always portions of the game amenable to tests.
      - coldtea 8y ago
        Changes until the last minute to crucial gameplay elements are not uncommon..
      - jayd16 8y ago
        Its not like tests are impossible. It's more that TDD is almost impossible.
      - omg_ketchup 8y ago
        Sure, menus and networking. Maybe loot distribution, level generation, etc. Not so much actual gameplay though. That does constantly get tweaked.
- otikik 8y ago
  I think the reason is that tests have a very obvious up-front cost, while the time they save is distributed in the future, in a non-immediately obvious way. I still think that they end up saving time, with some exceptions like UI code, which are more easily tested "by hand". Game project managers are infamous for not being great planners, so it wouldn't surprise me that they dismissed automated tests as "a waste of time" or "something that we can't do now because we don't have time now" (so we end up wasting more time in the end, having to do death marches, etc)
  - justinhj 8y ago
    Or perhaps game projects are extremely difficult to manage? You think if it was just a matter of competence then these companies would put billion dollar revenue on the line not hiring the best they can find?
  - blktiger 8y ago
    Modern games have only increased the benefits a game studio can gain from testing as well. Games are now moving into service territory which only increases the amount of time spent maintaining the game while continuing to add to it.
    - midnightclubbed 8y ago
      This. Determining when the effort should be applied is the tricky part. Games are still hit driven and get cancelled/re-purposed during development. You can spend a lot of QA engineering time developing systems to test functionality that never ships (case in point would be Fortnight - the original shipped game did not need to be tested against the current 100 player game instances and huge load but they could have spent a bunch of time testing AI systems that are no longer any part of the game).
      - blktiger 8y ago
        I don't disagree that it's important to understand when something is purely a proof of concept vs something that will stick around to evaluate the costs. However, the AI systems are still in Fortnite (and they even used those systems for the Haloween event). The major money making part of Fortnite has been the battle royale mode though. If they open up the main game to be free-to-play similar to the battle royale mode those systems will probably end up being used quite a bit.
- de_watcher 8y ago
  A sufficiently big game in an established genre with in-house engine and expansions has several levels of automatic tests.
  - pferde 8y ago
    In my experience, such games only have manual testing, done by the people who bought the game.
    - de_watcher 8y ago
      My experience is from working on one of the games with automatic tests. From the playing experience: yes, there are way more games with internal manual and external community testing.
      - SeanBoocock 8y ago
        Yeah at a certain level game productions will usually have automatic “smoke” tests for general build stability and I’ve worked on one that had automatic feature tests with replayed input. These were generally useful for catching obvious crashes and regressions, but the overhead only makes sense for a certain level of production. Could also see, and have heard of, more rigorous functional testing of things like a procedural generation pipeline that are otherwise harder to get sufficient coverage of manually.
- rafaelvasco 8y ago
  I mostly. Agree. Tests can be useful in some specific games, specific cases, but in general they're much harder to do in gamedev compared to other areas. Maybe one case for gamedev would be to test the calculations of character damage in a RPG based on several factors. But that is an isolated case.
- tokyodude 8y ago
  I'm not sure what most teams do now-a-days but I went to GDC in Koln and saw the Croteam talk. Over 10 or so years by programmers just adding a little here and there as it occurred to them they had build a pretty cool testing system. First they had made it so if someone was playing the game and saw a bug they could press the "file a bug" key, type in a description and the game would save out enough info to bring someone back to that point in the game, same camera, and possibly other state. From the bug database they could click a link that would launch the game back into that state, let someone verify the fix and mark it as fixes. The also had a waypoint system for bots to play through the puzzles (this was the Talos Principle they were talking about). If the bots ever got stuck, as in didn't make to the next waypoint within some time limit they bots would file a bug using the system above. https://www.gdcvault.com/play/1022784/Fast-Iteration-Tools-in-the https://www.gdcvault.com/play/1022784/Fast-Iteration-Tools-i... As another interesting idea apparently the creator of Thumber built a URL system so people press a button which would generate a URL into the clipboard, they could then paste that URL into Slack (or email/chat/etc) that would launch the game in a particular state to pass that other users on the team.
  - corysama 8y ago
    Similarly, on a game I worked on we had a “controller monkey” that would spam all possible player actions (move, jump, attack, special powers) rapidly and randomly. We then had the testers record multiple paths exploring through each level. Every few seconds the monkey-controlled player character would be teleported a bit further down the path to ensure progress and coverage. Dozens of people would set the monkey to run all night when they went home for the day. In the morning we would have a dozen fresh crash dumps.
    - pfranz 8y ago
      I feel like when people just say "tests" there's a lot of conflating. Usually, on a site like this or on a blog post, "tests" are referring to automated unit-testing or talking about TDD...which is probably most rare and most fought against in game dev (from what I've seen from an adjacent industry). Testing, in general, is pretty essential to writing code that does what you want. Test code is just automating what you'd be doing manually and it's a lot faster to have the code do it than for me to do it 10 times. Even if that's printing out a value or showing it in a debugger. Testing frameworks or libraries to fuzz out problems and harden code are quite common. Game dev often has a lot of manual play testers. Most engines and dev consoles have a lot of tools to either just record the screen or save state when problems are seen. Heck, most "cheat codes" were added to jump around the came to test for bugs. That's technically "test code."
- honkycat 8y ago
  I agree /w testing game development being less of a priority than in other industries, but I feel that it is because code quality tends to take a back seat in gamedev. Tests are NOT anti-agile, that is just dumb. I feel like a bunch of hacker news hipsters read an article about TDD 3 years ago and then said "Yep that's my opinion! Tests are bad." Despite every major software company requiring unit tests for their production code-bases. ( Hint: It's because they did the research and found tests beneficial. ) Tests enable agility. Let's just get this out of the way now: Tests are not about catching bugs. Tests are about allowing your to safely refactor your code without breaking previously declared behavior. Testing enables you to iterate and refactor code without constantly releasing new regressions. Testing IS code quality. If you lack tests you lack a core piece of code quality.
pulsarpietro 8y ago
I am a bit puzzled by this: "Before about the early 90s, we didn’t trust C compilers, so we wrote in assembly." There a lot of games released during the 80's, were they really all written in assembly ? https://www.myabandonware.com/browse/year/ https://www.myabandonware.com/browse/year/ I don't have experience in the game industry at all, I must add.
- vardump 8y ago
  Heavier things, like graphics processing were typically written in assembler. Or sound mixing. Some things like texture mapping you could only write in assembler, because you'd need to use x86 lower/higher half of word (like AL and AH registers) due to register pressure. Spilling to stack could have caused 50%+ slowdown. 486 era you needed assembler to work around quirks like AGI stalls. On Pentium the reason for assembler was to use FPU efficiently in parallel with normal code (FPU per pixel divide for perspective correction). Of course you also needed to carefully hand optimize for Pentium U and V pipes. If you did it correctly, you could execute up to 2 instructions per clock. If not, you lose up to half of the performance (or even more if you messed up register dependency chains, which were a bit weird sometimes). One also needs to remember compilers in the nineties were not very amazing at optimization. You could run circles around them by using assembler. Mind you, I still need to write some things in assembler even on modern x86. But it's pretty little nowadays. SIMD stuff (SSE/AVX) you can mostly do in "almost assembler" with instruction intrinsics, but without needing to worry about instruction scheduling and so on.
  - pulsarpietro 8y ago
    I must have been a bloody hard and interesting work, back then.
    - vardump 8y ago
      Things are much harder nowadays due to complexity. From almost impossible to understand CPU cores to massive amounts of third party code to the modern requirements (IoT, ouch!). Debugging predictable single thread single core system was also child's play compared to distributed networked beasts each running on lots of cores and thousands of threads. Nineties problems were contained in a small box. Oh, and no internet like today, so needed to order books and magazines. And to use BBS and usenet. Even then, a lot of it was reinventing the wheel again and again. Modern problems are sometimes nearly uncontained (think software like web browsers, etc.).
arka2147483647 8y ago
>Epilogue (... snip ...) > 1. Do nothing (...) You can deal with that by imposing rules on what is and isn’t allowed in your codebase, (...) This is what everybody is already doing in gamedev > 2. Get involved (...) C++ committee participation is open to everyone. (...) Most game dev studios are Small or Mediums sized companies, and don't really have the time to waste in Committee meatings...
- Jyaif 8y ago
  > Most game dev studios are Small or Mediums sized companies, and don't really have the time to waste in Committee meatings.. Irrelevant. What counts is where the C++ game devs are, and it's in the big companies. And participating in the design of a language is not a waste of time...
  - SomeHacker44 8y ago
    Its a matter of perspective, what constitutes a waste of time. If you are under pressure to ship something now/soon, and may not exist as a company in the next standards cycle, then it is probably a waste of time for that company.
CoolGuySteve 8y ago
Is Visual Studio really the best debugger? Every time I use it I get really frustrated by the difficulty of entering complex instructions. The GUI is more discoverable but I find myself missing gdb ‘s functions and parser. However, one thing in gdb that’s become steadily worse is the ability to evaluate STL’s operator[] and the like in optimized code, with the debugger frequently whining about inlining. It’s pretty horrible having to decipher the _m_data or whatever of various implementations. I’m actually not sure if gcc is not compiling the inlines into the object code (I thought it was required by the standard) or if gdb just can’t find them.
- pjmlp 8y ago
  Yes, given the graphical tooling for multi-core, GPGPU, data visualization, edit-and-continue, mixing Assembly with code (even on .NET), interaction with GUI components on WPF/UWP apps,...
  - CoolGuySteve 8y ago
    That kind of gets to the heart of what I’m saying. These graphical tools are great as long as they do what you need. But they are less composable and customizable than an expression parser. Like in the article he mentions not being able to see custom data types, but my .gdbinit has a few pretty printers in it for exactly that purpose. And when you do get something customized in MSVC like a specific PGO build or something, it tends to be tightly coupled to that project. It’s less easy to cut and paste into another project since the primary interface is really a dozen little text fields modifying XML somewhere.
    - pjmlp 8y ago
      VS supports displaying custom types. Regarding gdb, during the mid-90's I got by calling it from XEmacs, until I discovered DDD. I got spoiled with Borland debuggers, typing n, s, l, p all the time and drawing structures on paper gets tiring after a while.
    - coffeeaddicted 8y ago
deng 8y ago
So, the solution to bad debug performance is essentially YAGNI? I'm afraid that isn't a very convincing argument. If your code is several orders of magnitude slower in debug mode, then this is a problem. Simply downplaying this with arguments like "single-step debugging is a last resort" or "just write better tests" won't make this problem vanish. Just like exploding compile times are not solved with "just buy Incredibuild". But his argument fits well with C++'s history of finding exceedingly complex solutions for simple problems. Want to have efficient matrix calculation? Well, who needs native support for matrices when you can do the same with expression templates and static polymorphism/CRTP (see: Eigen library). The last section of the article says you either do nothing or you get involved. I'm afraid it is missing the obvious third option: switch to another language which actually supports your use case.
- cheez 8y ago
  For what it's worth, I only use a debugger when I have a crash.
- neutronicus 8y ago
  > But his argument fits well with C++'s history of finding exceedingly complex solutions for simple problems. Want to have efficient matrix calculation? Well, who needs native support for matrices when you can do the same with expression templates and static polymorphism/CRTP (see: Eigen library). I have to defend C++ here - "native matrices" is under-specified. In practice, "Matrix" is one of the leakiest abstractions in programming and you have to care about representation and choice of algorithm pretty much from the get-go, and IMO C++ is actually the best available option for managing that complexity, especially when you're solving large systems in parallel (and it's worth pointing out that one of the front-running open-source libs in this space is written in C++[1]). [1] https://github.com/trilinos/Trilinos https://github.com/trilinos/Trilinos
  - deng 8y ago
    For many projects you won't need that complexity. You just want to directly map basic matrix operations to the usual BLAS/LAPACK calls. Fortran 90+ does that job well, for instance, and performance will usually be better than a C++ library (yes, I've tested against Eigen and Armadillo, although that was years ago). Combine that with the enormous compile times and the absolute ridiculous error messages for even simplest syntax errors, and I never looked back. Fortran may have a bad rep, but the newer iterations are actually pretty good for that kind of stuff.
shmerl 8y ago
Game developers should start using more Rust.
- rafaelvasco 8y ago
  Yeah it's an awesome language. Has its downsides (language enforced memory micromanagement is a good thing but can get annoying sometimes.) but it's one of the best we have now. For now i'll stay with my beloved C#.
  - satsuma 8y ago
    +1 for csharp, got to use it in an introductory unity class and fell in love with it.
  - shmerl 8y ago
    C# doesn't sound like a good option for games development though (except may be for scripting used in various engines). It's now dominated by C++ for a good reason, since it requires tight performance control. So Rust is a valid candidate for fixing C++ issues. C# - not really.
- uglycoyote 8y ago
  For console development (Sony, Nintendo, XBox), anything but C++ has never seemed like an option because all of the development tools and libraries provided for working on those consoles are C++ centric. But I'm curious if there are any console development companies that are successfully using Rust or other languages which perhaps can link with C++ libraries? We use C# in our studio for tools, and are able to link it with our game C++ so that we can run some of the game's subsystems within the tools (e.g. animation engine) but shipping the game with C# code is not an option for several reasons, performance being the most important, but also we need to build our game for the console using CLang/LLVM, and I suspect it's not possible to write C# which interfaces with C++ using LLVM, only with Microsoft's compiler.
  - steveklabnik 8y ago
    For Rust, Chucklefish has written rust for all the current generation of consoles: https://github.com/rust-lang/prev.rust-lang.org/blob/master/pdfs/Rust-Chucklefish-Whitepaper.pdf https://github.com/rust-lang/prev.rust-lang.org/blob/master/... There are other game companies doing stuff, but we have less details including platform: EA’s SEED division, Ready At Dawn, and Embark (some ex-SEED devs making a new studio where Rust is the primary language.)
youdontknowtho 8y ago
Whenever the response to a twitter argument is "get involved and make the change you want to see happen" you can expect absolutely nothing to change. Part of this is just people complaining on a platform that over values short pithy complaints.
overgard 8y ago
A few thoughts: * None of the problems that have been commented on are unique to the games industry at all. Slow debug builds suck for all C++ developers and weird template meta-programming is confusing for practically everyone. * He makes these broad hand-wavey statements like "individuals don't feel pain from slow compile times", or "big companies can just can throw processor power at it" to which I would say, BS. Fast iteration in C++ is really hard because of the delay and it's a big problem for everyone. * "Participate more" -- isn't that exactly what people are doing on twitter? Not everyone can go to CppCon.
- andrewmcwatters 8y ago
  My question is, where are all these "big companies" who can throw more processor power at these problems? Because frankly, every major company I've been at uses the same commodity or cloud hardware everyone else does, so I just don't see it. It's a moot point. Rarely do I see workstation-grade hardware in the wild, and when I have, they're build slaves that are incredibly anti-agile.
  - mattnewport 8y ago
    EA I know gives developers very powerful workstation class developer machines because I used to work there and have friends who still do and if anything it sounds like they've got even more powerful on a relative basis since I left.
- mattnewport 8y ago
  I think slow debug performance is a bigger problem for games than many other applications. It's annoying for everyone but the nature of games as interactive experiences means you often have to play a game to reproduce a bug easily and often with a full production level where the bug was reported, not with simpler test content. If you can't maintain a playable frame rate in debug builds this can be a problem. This problem was worst in my experience in the Xbox 360 / PS3 generation because the in order processors handled debug builds very poorly and were different enough from a PC that it was common to have to debug on target rather than on a PC build on a much more powerful development machine. It's less of an issue with current generation consoles that are basically PCs as they don't suffer as badly with debug performance and many issues can be debugged on a PC build on a more powerful system. It may be more of an issue for mobile still. Fortunately many of the newer features of C++ 17 and 20 help both with improving debug performance and with simplifying / reducing the need for "weird template meta-programming". Several also help with compile times and modules in particular are quite focused on tackling the biggest root cause of slow compiles in C++.
edoo 8y ago
Most industry game dev is also done on top of C++ engines and libraries. I can see Go being used in the near future as the big engines offer bindings but I bet in 5-10 years the average startup is using something like C#. It is slow as beans but eventually CPU speed will make it much more reasonable for real use. The Unity engine is a good example. It has a weird easy powerful super bloated paradigm.
- andrewmcwatters 8y ago
  You might want to stretch that estimate out. I'm running an Intel Core i5-3550 from 2012. Furthermore, I foresee no reason to upgrade in the next 4 years. The current i5 on userbenchmark.com's front page is the 9600k, which says it's ~53% faster than my 3550. 7 years later. CPU performance is barely going anywhere. Developers should instead try to figure out how to do more with less growth. GPUs are also overpriced, and playing older games and comparing them to new ones doesn't show great payoff. As far as I'm concerned, we've plateaued. Maybe going from a GTX 760 to a 1060 would give me a few more frames, but frankly, more often than not, the games are programmed like utter shit.
  - HippoBaro 8y ago
    This. When you think about it, it's an exciting time to be a dev. We need to be clever at stuff and can't just expect next-generation CPUs to make coffee for us.
- jokoon 8y ago
  You just mentioned 2 garbage collected languages on a game dev topic. I don't think they're adequate. Remember how the article was very insistent about being able to control memory and CPU resources. Those are one of the few reasons C++ is not dead. Rust? I don't see it either.
  - andrewmcwatters 8y ago
    A big part of this is ecosystem. You have so many game libraries and software that are written in C and C++. To many gamedevs, Rust is just a systems language Go. It doesn't bring anything significant to the table compared to just using a limited subset of C++.
pmarin 8y ago
>Stop going to GDC as your one conference per year and start going to CppCon. In fact the most popular CppCon video in Youtube is from Mike Acton: “Data Oriented Design and C++”
arandr0x 8y ago
I'm not in games but my industry is adjacent and I have the same gripes with c++. (Which the author characterized very well.) Although build and debug times definitely are an issue regardless of the performance of the hardware. I find reading C++ "standards" papers onerous and feel like they're written in a way that's deliberately inaccessible. I don't much like the idea of going to CppCon -- even if my company funded it, which maybe they would, I feel like I'd be marginalized for not using template metaprogramming, not knowing the new hotness by heart, and generally being a proponent of C-with-classes. I just feel like so much of the C++ "standards" work feels like it's led by academics who think the concerns of working programmers like me are beneath them. Is there a way I can "get involved" and does my voice have any value?
- doctorRetro 8y ago
  Upvoted. And I just want to say I agree completely. Your comments re: "not knowing the new hotness" and "led by academics who think the concerns of working programmers like me are beneath them" really strikes a chord with me. I think this is a problem in all of programming these days and has really soured me on the industry.
- mattnewport 8y ago
  If you want to get involved and for your voice to have value then you have to educate yourself on the subject. Unfortunately too many of my former colleagues in the games industry (I'm now in a "games adjacent" industry too) fail to do this before complaining about C++ and have the same attitude of "it's not fair that I should have to know what I'm talking about before anyone will listen to my complaining seriously". Videos of all CppCon talks from the last several years are freely available on YouTube and if you took the time to watch them you'd see that many of them are by working programmers and not academics, quite a few of them in the games industry. You'd also learn that the committee is quite focused on simplifying the use of the language and on finding more usable ways to get the benefits of template metaprogramming. You would also find explanations of many features that are more accessible than standards papers and occasional explanations of why standardese is the way it is - nobody, even the most academic speakers, claims to find the standard the most accessible way to learn about a new feature.
mark-r 8y ago
Maybe the QA folks are worse off in game dev companies, but they seem to be second class citizens everywhere else too. Which is a pity, since as stated they are worth their weight in gold. A developer has a mindset of how do I make this work, a QA person has a mindset of how do I make this break - they are completely complimentary.

150 points by lebek 8y ago

I don't want to weigh in on the rest of the content but the characterization of the game industry is pretty accurate in my experience. I would expand more on the first bullet point of why game devs don't test. Tests are anti-agile and game development is extremely agile. Usually you don't know what kind of game you're making until you're done.

oselhn 8y ago
That's not true. From my experience unit tests are great for agile. It will allow you to create "trusted" modules which you can move around and rework much easier (you can also treat your tests as executable documentation). Without tests you can't safely do any change to existing code especially if you are changing code written by someone else. You have to risk it and than spent considerable time in debugger if it breaks something unrelated to your feature.
- coldtea 8y ago
  >That's not true. From my experience unit tests are great for agile. Not when you're constantly prototyping, which is what game dev essentially is for the most part of the process...
  - lifthrasiir 8y ago
    > which is what game dev essentially is for the most part of the process... Except that it isn't. You don't maintain the equal velocity of changes throughout the process, even for indie games. And there are always portions of the game amenable to tests.
    - coldtea 8y ago
      Changes until the last minute to crucial gameplay elements are not uncommon..
    - jayd16 8y ago
      Its not like tests are impossible. It's more that TDD is almost impossible.
    - omg_ketchup 8y ago
      Sure, menus and networking. Maybe loot distribution, level generation, etc. Not so much actual gameplay though. That does constantly get tweaked.
otikik 8y ago
I think the reason is that tests have a very obvious up-front cost, while the time they save is distributed in the future, in a non-immediately obvious way. I still think that they end up saving time, with some exceptions like UI code, which are more easily tested "by hand". Game project managers are infamous for not being great planners, so it wouldn't surprise me that they dismissed automated tests as "a waste of time" or "something that we can't do now because we don't have time now" (so we end up wasting more time in the end, having to do death marches, etc)
- justinhj 8y ago
  Or perhaps game projects are extremely difficult to manage? You think if it was just a matter of competence then these companies would put billion dollar revenue on the line not hiring the best they can find?
- blktiger 8y ago
  Modern games have only increased the benefits a game studio can gain from testing as well. Games are now moving into service territory which only increases the amount of time spent maintaining the game while continuing to add to it.
  - midnightclubbed 8y ago
    This. Determining when the effort should be applied is the tricky part. Games are still hit driven and get cancelled/re-purposed during development. You can spend a lot of QA engineering time developing systems to test functionality that never ships (case in point would be Fortnight - the original shipped game did not need to be tested against the current 100 player game instances and huge load but they could have spent a bunch of time testing AI systems that are no longer any part of the game).
    - blktiger 8y ago
      I don't disagree that it's important to understand when something is purely a proof of concept vs something that will stick around to evaluate the costs. However, the AI systems are still in Fortnite (and they even used those systems for the Haloween event). The major money making part of Fortnite has been the battle royale mode though. If they open up the main game to be free-to-play similar to the battle royale mode those systems will probably end up being used quite a bit.
de_watcher 8y ago
A sufficiently big game in an established genre with in-house engine and expansions has several levels of automatic tests.
- pferde 8y ago
  In my experience, such games only have manual testing, done by the people who bought the game.
  - de_watcher 8y ago
    My experience is from working on one of the games with automatic tests. From the playing experience: yes, there are way more games with internal manual and external community testing.
    - SeanBoocock 8y ago
      Yeah at a certain level game productions will usually have automatic “smoke” tests for general build stability and I’ve worked on one that had automatic feature tests with replayed input. These were generally useful for catching obvious crashes and regressions, but the overhead only makes sense for a certain level of production. Could also see, and have heard of, more rigorous functional testing of things like a procedural generation pipeline that are otherwise harder to get sufficient coverage of manually.
rafaelvasco 8y ago
I mostly. Agree. Tests can be useful in some specific games, specific cases, but in general they're much harder to do in gamedev compared to other areas. Maybe one case for gamedev would be to test the calculations of character damage in a RPG based on several factors. But that is an isolated case.
tokyodude 8y ago
I'm not sure what most teams do now-a-days but I went to GDC in Koln and saw the Croteam talk. Over 10 or so years by programmers just adding a little here and there as it occurred to them they had build a pretty cool testing system. First they had made it so if someone was playing the game and saw a bug they could press the "file a bug" key, type in a description and the game would save out enough info to bring someone back to that point in the game, same camera, and possibly other state. From the bug database they could click a link that would launch the game back into that state, let someone verify the fix and mark it as fixes. The also had a waypoint system for bots to play through the puzzles (this was the Talos Principle they were talking about). If the bots ever got stuck, as in didn't make to the next waypoint within some time limit they bots would file a bug using the system above. https://www.gdcvault.com/play/1022784/Fast-Iteration-Tools-in-the https://www.gdcvault.com/play/1022784/Fast-Iteration-Tools-i... As another interesting idea apparently the creator of Thumber built a URL system so people press a button which would generate a URL into the clipboard, they could then paste that URL into Slack (or email/chat/etc) that would launch the game in a particular state to pass that other users on the team.
- corysama 8y ago
  Similarly, on a game I worked on we had a “controller monkey” that would spam all possible player actions (move, jump, attack, special powers) rapidly and randomly. We then had the testers record multiple paths exploring through each level. Every few seconds the monkey-controlled player character would be teleported a bit further down the path to ensure progress and coverage. Dozens of people would set the monkey to run all night when they went home for the day. In the morning we would have a dozen fresh crash dumps.
  - pfranz 8y ago
    I feel like when people just say "tests" there's a lot of conflating. Usually, on a site like this or on a blog post, "tests" are referring to automated unit-testing or talking about TDD...which is probably most rare and most fought against in game dev (from what I've seen from an adjacent industry). Testing, in general, is pretty essential to writing code that does what you want. Test code is just automating what you'd be doing manually and it's a lot faster to have the code do it than for me to do it 10 times. Even if that's printing out a value or showing it in a debugger. Testing frameworks or libraries to fuzz out problems and harden code are quite common. Game dev often has a lot of manual play testers. Most engines and dev consoles have a lot of tools to either just record the screen or save state when problems are seen. Heck, most "cheat codes" were added to jump around the came to test for bugs. That's technically "test code."
honkycat 8y ago
I agree /w testing game development being less of a priority than in other industries, but I feel that it is because code quality tends to take a back seat in gamedev. Tests are NOT anti-agile, that is just dumb. I feel like a bunch of hacker news hipsters read an article about TDD 3 years ago and then said "Yep that's my opinion! Tests are bad." Despite every major software company requiring unit tests for their production code-bases. ( Hint: It's because they did the research and found tests beneficial. ) Tests enable agility. Let's just get this out of the way now: Tests are not about catching bugs. Tests are about allowing your to safely refactor your code without breaking previously declared behavior. Testing enables you to iterate and refactor code without constantly releasing new regressions. Testing IS code quality. If you lack tests you lack a core piece of code quality.

pulsarpietro 8y ago

I am a bit puzzled by this: "Before about the early 90s, we didn’t trust C compilers, so we wrote in assembly." There a lot of games released during the 80's, were they really all written in assembly ? https://www.myabandonware.com/browse/year/ https://www.myabandonware.com/browse/year/ I don't have experience in the game industry at all, I must add.

vardump 8y ago
Heavier things, like graphics processing were typically written in assembler. Or sound mixing. Some things like texture mapping you could only write in assembler, because you'd need to use x86 lower/higher half of word (like AL and AH registers) due to register pressure. Spilling to stack could have caused 50%+ slowdown. 486 era you needed assembler to work around quirks like AGI stalls. On Pentium the reason for assembler was to use FPU efficiently in parallel with normal code (FPU per pixel divide for perspective correction). Of course you also needed to carefully hand optimize for Pentium U and V pipes. If you did it correctly, you could execute up to 2 instructions per clock. If not, you lose up to half of the performance (or even more if you messed up register dependency chains, which were a bit weird sometimes). One also needs to remember compilers in the nineties were not very amazing at optimization. You could run circles around them by using assembler. Mind you, I still need to write some things in assembler even on modern x86. But it's pretty little nowadays. SIMD stuff (SSE/AVX) you can mostly do in "almost assembler" with instruction intrinsics, but without needing to worry about instruction scheduling and so on.
- pulsarpietro 8y ago
  I must have been a bloody hard and interesting work, back then.
  - vardump 8y ago
    Things are much harder nowadays due to complexity. From almost impossible to understand CPU cores to massive amounts of third party code to the modern requirements (IoT, ouch!). Debugging predictable single thread single core system was also child's play compared to distributed networked beasts each running on lots of cores and thousands of threads. Nineties problems were contained in a small box. Oh, and no internet like today, so needed to order books and magazines. And to use BBS and usenet. Even then, a lot of it was reinventing the wheel again and again. Modern problems are sometimes nearly uncontained (think software like web browsers, etc.).

arka2147483647 8y ago

>Epilogue (... snip ...) > 1. Do nothing (...) You can deal with that by imposing rules on what is and isn’t allowed in your codebase, (...) This is what everybody is already doing in gamedev > 2. Get involved (...) C++ committee participation is open to everyone. (...) Most game dev studios are Small or Mediums sized companies, and don't really have the time to waste in Committee meatings...

Jyaif 8y ago
> Most game dev studios are Small or Mediums sized companies, and don't really have the time to waste in Committee meatings.. Irrelevant. What counts is where the C++ game devs are, and it's in the big companies. And participating in the design of a language is not a waste of time...
- SomeHacker44 8y ago
  Its a matter of perspective, what constitutes a waste of time. If you are under pressure to ship something now/soon, and may not exist as a company in the next standards cycle, then it is probably a waste of time for that company.

CoolGuySteve 8y ago

Is Visual Studio really the best debugger? Every time I use it I get really frustrated by the difficulty of entering complex instructions. The GUI is more discoverable but I find myself missing gdb ‘s functions and parser. However, one thing in gdb that’s become steadily worse is the ability to evaluate STL’s operator[] and the like in optimized code, with the debugger frequently whining about inlining. It’s pretty horrible having to decipher the _m_data or whatever of various implementations. I’m actually not sure if gcc is not compiling the inlines into the object code (I thought it was required by the standard) or if gdb just can’t find them.

pjmlp 8y ago
Yes, given the graphical tooling for multi-core, GPGPU, data visualization, edit-and-continue, mixing Assembly with code (even on .NET), interaction with GUI components on WPF/UWP apps,...
- CoolGuySteve 8y ago
  That kind of gets to the heart of what I’m saying. These graphical tools are great as long as they do what you need. But they are less composable and customizable than an expression parser. Like in the article he mentions not being able to see custom data types, but my .gdbinit has a few pretty printers in it for exactly that purpose. And when you do get something customized in MSVC like a specific PGO build or something, it tends to be tightly coupled to that project. It’s less easy to cut and paste into another project since the primary interface is really a dozen little text fields modifying XML somewhere.
  - pjmlp 8y ago
    VS supports displaying custom types. Regarding gdb, during the mid-90's I got by calling it from XEmacs, until I discovered DDD. I got spoiled with Borland debuggers, typing n, s, l, p all the time and drawing structures on paper gets tiring after a while.
  - coffeeaddicted 8y ago

deng 8y ago

So, the solution to bad debug performance is essentially YAGNI? I'm afraid that isn't a very convincing argument. If your code is several orders of magnitude slower in debug mode, then this is a problem. Simply downplaying this with arguments like "single-step debugging is a last resort" or "just write better tests" won't make this problem vanish. Just like exploding compile times are not solved with "just buy Incredibuild". But his argument fits well with C++'s history of finding exceedingly complex solutions for simple problems. Want to have efficient matrix calculation? Well, who needs native support for matrices when you can do the same with expression templates and static polymorphism/CRTP (see: Eigen library). The last section of the article says you either do nothing or you get involved. I'm afraid it is missing the obvious third option: switch to another language which actually supports your use case.

cheez 8y ago
For what it's worth, I only use a debugger when I have a crash.
neutronicus 8y ago
> But his argument fits well with C++'s history of finding exceedingly complex solutions for simple problems. Want to have efficient matrix calculation? Well, who needs native support for matrices when you can do the same with expression templates and static polymorphism/CRTP (see: Eigen library). I have to defend C++ here - "native matrices" is under-specified. In practice, "Matrix" is one of the leakiest abstractions in programming and you have to care about representation and choice of algorithm pretty much from the get-go, and IMO C++ is actually the best available option for managing that complexity, especially when you're solving large systems in parallel (and it's worth pointing out that one of the front-running open-source libs in this space is written in C++[1]). [1] https://github.com/trilinos/Trilinos https://github.com/trilinos/Trilinos
- deng 8y ago
  For many projects you won't need that complexity. You just want to directly map basic matrix operations to the usual BLAS/LAPACK calls. Fortran 90+ does that job well, for instance, and performance will usually be better than a C++ library (yes, I've tested against Eigen and Armadillo, although that was years ago). Combine that with the enormous compile times and the absolute ridiculous error messages for even simplest syntax errors, and I never looked back. Fortran may have a bad rep, but the newer iterations are actually pretty good for that kind of stuff.

shmerl 8y ago

Game developers should start using more Rust.

rafaelvasco 8y ago
Yeah it's an awesome language. Has its downsides (language enforced memory micromanagement is a good thing but can get annoying sometimes.) but it's one of the best we have now. For now i'll stay with my beloved C#.
- satsuma 8y ago
  +1 for csharp, got to use it in an introductory unity class and fell in love with it.
- shmerl 8y ago
  C# doesn't sound like a good option for games development though (except may be for scripting used in various engines). It's now dominated by C++ for a good reason, since it requires tight performance control. So Rust is a valid candidate for fixing C++ issues. C# - not really.
uglycoyote 8y ago
For console development (Sony, Nintendo, XBox), anything but C++ has never seemed like an option because all of the development tools and libraries provided for working on those consoles are C++ centric. But I'm curious if there are any console development companies that are successfully using Rust or other languages which perhaps can link with C++ libraries? We use C# in our studio for tools, and are able to link it with our game C++ so that we can run some of the game's subsystems within the tools (e.g. animation engine) but shipping the game with C# code is not an option for several reasons, performance being the most important, but also we need to build our game for the console using CLang/LLVM, and I suspect it's not possible to write C# which interfaces with C++ using LLVM, only with Microsoft's compiler.
- steveklabnik 8y ago
  For Rust, Chucklefish has written rust for all the current generation of consoles: https://github.com/rust-lang/prev.rust-lang.org/blob/master/pdfs/Rust-Chucklefish-Whitepaper.pdf https://github.com/rust-lang/prev.rust-lang.org/blob/master/... There are other game companies doing stuff, but we have less details including platform: EA’s SEED division, Ready At Dawn, and Embark (some ex-SEED devs making a new studio where Rust is the primary language.)

youdontknowtho 8y ago

Whenever the response to a twitter argument is "get involved and make the change you want to see happen" you can expect absolutely nothing to change. Part of this is just people complaining on a platform that over values short pithy complaints.

overgard 8y ago

A few thoughts: * None of the problems that have been commented on are unique to the games industry at all. Slow debug builds suck for all C++ developers and weird template meta-programming is confusing for practically everyone. * He makes these broad hand-wavey statements like "individuals don't feel pain from slow compile times", or "big companies can just can throw processor power at it" to which I would say, BS. Fast iteration in C++ is really hard because of the delay and it's a big problem for everyone. * "Participate more" -- isn't that exactly what people are doing on twitter? Not everyone can go to CppCon.

andrewmcwatters 8y ago
My question is, where are all these "big companies" who can throw more processor power at these problems? Because frankly, every major company I've been at uses the same commodity or cloud hardware everyone else does, so I just don't see it. It's a moot point. Rarely do I see workstation-grade hardware in the wild, and when I have, they're build slaves that are incredibly anti-agile.
- mattnewport 8y ago
  EA I know gives developers very powerful workstation class developer machines because I used to work there and have friends who still do and if anything it sounds like they've got even more powerful on a relative basis since I left.
mattnewport 8y ago
I think slow debug performance is a bigger problem for games than many other applications. It's annoying for everyone but the nature of games as interactive experiences means you often have to play a game to reproduce a bug easily and often with a full production level where the bug was reported, not with simpler test content. If you can't maintain a playable frame rate in debug builds this can be a problem. This problem was worst in my experience in the Xbox 360 / PS3 generation because the in order processors handled debug builds very poorly and were different enough from a PC that it was common to have to debug on target rather than on a PC build on a much more powerful development machine. It's less of an issue with current generation consoles that are basically PCs as they don't suffer as badly with debug performance and many issues can be debugged on a PC build on a more powerful system. It may be more of an issue for mobile still. Fortunately many of the newer features of C++ 17 and 20 help both with improving debug performance and with simplifying / reducing the need for "weird template meta-programming". Several also help with compile times and modules in particular are quite focused on tackling the biggest root cause of slow compiles in C++.

edoo 8y ago

Most industry game dev is also done on top of C++ engines and libraries. I can see Go being used in the near future as the big engines offer bindings but I bet in 5-10 years the average startup is using something like C#. It is slow as beans but eventually CPU speed will make it much more reasonable for real use. The Unity engine is a good example. It has a weird easy powerful super bloated paradigm.

andrewmcwatters 8y ago
You might want to stretch that estimate out. I'm running an Intel Core i5-3550 from 2012. Furthermore, I foresee no reason to upgrade in the next 4 years. The current i5 on userbenchmark.com's front page is the 9600k, which says it's ~53% faster than my 3550. 7 years later. CPU performance is barely going anywhere. Developers should instead try to figure out how to do more with less growth. GPUs are also overpriced, and playing older games and comparing them to new ones doesn't show great payoff. As far as I'm concerned, we've plateaued. Maybe going from a GTX 760 to a 1060 would give me a few more frames, but frankly, more often than not, the games are programmed like utter shit.
- HippoBaro 8y ago
  This. When you think about it, it's an exciting time to be a dev. We need to be clever at stuff and can't just expect next-generation CPUs to make coffee for us.
jokoon 8y ago
You just mentioned 2 garbage collected languages on a game dev topic. I don't think they're adequate. Remember how the article was very insistent about being able to control memory and CPU resources. Those are one of the few reasons C++ is not dead. Rust? I don't see it either.
- andrewmcwatters 8y ago
  A big part of this is ecosystem. You have so many game libraries and software that are written in C and C++. To many gamedevs, Rust is just a systems language Go. It doesn't bring anything significant to the table compared to just using a limited subset of C++.

pmarin 8y ago

>Stop going to GDC as your one conference per year and start going to CppCon. In fact the most popular CppCon video in Youtube is from Mike Acton: “Data Oriented Design and C++”

arandr0x 8y ago

I'm not in games but my industry is adjacent and I have the same gripes with c++. (Which the author characterized very well.) Although build and debug times definitely are an issue regardless of the performance of the hardware. I find reading C++ "standards" papers onerous and feel like they're written in a way that's deliberately inaccessible. I don't much like the idea of going to CppCon -- even if my company funded it, which maybe they would, I feel like I'd be marginalized for not using template metaprogramming, not knowing the new hotness by heart, and generally being a proponent of C-with-classes. I just feel like so much of the C++ "standards" work feels like it's led by academics who think the concerns of working programmers like me are beneath them. Is there a way I can "get involved" and does my voice have any value?

doctorRetro 8y ago
Upvoted. And I just want to say I agree completely. Your comments re: "not knowing the new hotness" and "led by academics who think the concerns of working programmers like me are beneath them" really strikes a chord with me. I think this is a problem in all of programming these days and has really soured me on the industry.
mattnewport 8y ago
If you want to get involved and for your voice to have value then you have to educate yourself on the subject. Unfortunately too many of my former colleagues in the games industry (I'm now in a "games adjacent" industry too) fail to do this before complaining about C++ and have the same attitude of "it's not fair that I should have to know what I'm talking about before anyone will listen to my complaining seriously". Videos of all CppCon talks from the last several years are freely available on YouTube and if you took the time to watch them you'd see that many of them are by working programmers and not academics, quite a few of them in the games industry. You'd also learn that the committee is quite focused on simplifying the use of the language and on finding more usable ways to get the benefits of template metaprogramming. You would also find explanations of many features that are more accessible than standards papers and occasional explanations of why standardese is the way it is - nobody, even the most academic speakers, claims to find the standard the most accessible way to learn about a new feature.

mark-r 8y ago

Maybe the QA folks are worse off in game dev companies, but they seem to be second class citizens everywhere else too. Which is a pity, since as stated they are worth their weight in gold. A developer has a mindset of how do I make this work, a QA person has a mindset of how do I make this break - they are completely complimentary.