4 ms·
I don't want to weigh in on the rest of the content but the characterization of the game industry is pretty accurate in my experience. I would expand more on t
by jayd16 8y ago
I don't want to weigh in on the rest of the content but the characterization of the game industry is pretty accurate in my experience.
I would expand more on the first bullet point of why game devs don't test. Tests are anti-agile and game development is extremely agile. Usually you don't know what kind of game you're making until you're done.
- oselhn 8y agoThat's not true. From my experience unit tests are great for agile. It will allow you to create "trusted" modules which you can move around and rework much easier (you can also treat your tests as executable documentation). Without tests you can't safely do any change to existing code especially if you are changing code written by someone else. You have to risk it and than spent considerable time in debugger if it breaks something unrelated to your feature.
- coldtea 8y ago>That's not true. From my experience unit tests are great for agile. Not when you're constantly prototyping, which is what game dev essentially is for the most part of the process...
- lifthrasiir 8y ago> which is what game dev essentially is for the most part of the process... Except that it isn't. You don't maintain the equal velocity of changes throughout the process, even for indie games. And there are always portions of the game amenable to tests.
- coldtea 8y agoChanges until the last minute to crucial gameplay elements are not uncommon..
- jayd16 8y agoIts not like tests are impossible. It's more that TDD is almost impossible.
- omg_ketchup 8y agoSure, menus and networking. Maybe loot distribution, level generation, etc. Not so much actual gameplay though. That does constantly get tweaked.
- otikik 8y agoI think the reason is that tests have a very obvious up-front cost, while the time they save is distributed in the future, in a non-immediately obvious way. I still think that they end up saving time, with some exceptions like UI code, which are more easily tested "by hand". Game project managers are infamous for not being great planners, so it wouldn't surprise me that they dismissed automated tests as "a waste of time" or "something that we can't do now because we don't have time now" (so we end up wasting more time in the end, having to do death marches, etc)
- justinhj 8y agoOr perhaps game projects are extremely difficult to manage? You think if it was just a matter of competence then these companies would put billion dollar revenue on the line not hiring the best they can find?
- blktiger 8y agoModern games have only increased the benefits a game studio can gain from testing as well. Games are now moving into service territory which only increases the amount of time spent maintaining the game while continuing to add to it.
- midnightclubbed 8y agoThis. Determining when the effort should be applied is the tricky part. Games are still hit driven and get cancelled/re-purposed during development. You can spend a lot of QA engineering time developing systems to test functionality that never ships (case in point would be Fortnight - the original shipped game did not need to be tested against the current 100 player game instances and huge load but they could have spent a bunch of time testing AI systems that are no longer any part of the game).
- blktiger 8y agoI don't disagree that it's important to understand when something is purely a proof of concept vs something that will stick around to evaluate the costs. However, the AI systems are still in Fortnite (and they even used those systems for the Haloween event). The major money making part of Fortnite has been the battle royale mode though. If they open up the main game to be free-to-play similar to the battle royale mode those systems will probably end up being used quite a bit.
- de_watcher 8y agoA sufficiently big game in an established genre with in-house engine and expansions has several levels of automatic tests.
- pferde 8y agoIn my experience, such games only have manual testing, done by the people who bought the game.
- de_watcher 8y agoMy experience is from working on one of the games with automatic tests. From the playing experience: yes, there are way more games with internal manual and external community testing.
- SeanBoocock 8y agoYeah at a certain level game productions will usually have automatic “smoke” tests for general build stability and I’ve worked on one that had automatic feature tests with replayed input. These were generally useful for catching obvious crashes and regressions, but the overhead only makes sense for a certain level of production. Could also see, and have heard of, more rigorous functional testing of things like a procedural generation pipeline that are otherwise harder to get sufficient coverage of manually.
- rafaelvasco 8y agoI mostly. Agree. Tests can be useful in some specific games, specific cases, but in general they're much harder to do in gamedev compared to other areas. Maybe one case for gamedev would be to test the calculations of character damage in a RPG based on several factors. But that is an isolated case.
- tokyodude 8y agoI'm not sure what most teams do now-a-days but I went to GDC in Koln and saw the Croteam talk. Over 10 or so years by programmers just adding a little here and there as it occurred to them they had build a pretty cool testing system. First they had made it so if someone was playing the game and saw a bug they could press the "file a bug" key, type in a description and the game would save out enough info to bring someone back to that point in the game, same camera, and possibly other state. From the bug database they could click a link that would launch the game back into that state, let someone verify the fix and mark it as fixes. The also had a waypoint system for bots to play through the puzzles (this was the Talos Principle they were talking about). If the bots ever got stuck, as in didn't make to the next waypoint within some time limit they bots would file a bug using the system above. https://www.gdcvault.com/play/1022784/Fast-Iteration-Tools-in-the https://www.gdcvault.com/play/1022784/Fast-Iteration-Tools-i... As another interesting idea apparently the creator of Thumber built a URL system so people press a button which would generate a URL into the clipboard, they could then paste that URL into Slack (or email/chat/etc) that would launch the game in a particular state to pass that other users on the team.
- corysama 8y agoSimilarly, on a game I worked on we had a “controller monkey” that would spam all possible player actions (move, jump, attack, special powers) rapidly and randomly. We then had the testers record multiple paths exploring through each level. Every few seconds the monkey-controlled player character would be teleported a bit further down the path to ensure progress and coverage. Dozens of people would set the monkey to run all night when they went home for the day. In the morning we would have a dozen fresh crash dumps.
- pfranz 8y agoI feel like when people just say "tests" there's a lot of conflating. Usually, on a site like this or on a blog post, "tests" are referring to automated unit-testing or talking about TDD...which is probably most rare and most fought against in game dev (from what I've seen from an adjacent industry). Testing, in general, is pretty essential to writing code that does what you want. Test code is just automating what you'd be doing manually and it's a lot faster to have the code do it than for me to do it 10 times. Even if that's printing out a value or showing it in a debugger. Testing frameworks or libraries to fuzz out problems and harden code are quite common. Game dev often has a lot of manual play testers. Most engines and dev consoles have a lot of tools to either just record the screen or save state when problems are seen. Heck, most "cheat codes" were added to jump around the came to test for bugs. That's technically "test code."
- honkycat 8y agoI agree /w testing game development being less of a priority than in other industries, but I feel that it is because code quality tends to take a back seat in gamedev. Tests are NOT anti-agile, that is just dumb. I feel like a bunch of hacker news hipsters read an article about TDD 3 years ago and then said "Yep that's my opinion! Tests are bad." Despite every major software company requiring unit tests for their production code-bases. ( Hint: It's because they did the research and found tests beneficial. ) Tests enable agility. Let's just get this out of the way now: Tests are not about catching bugs. Tests are about allowing your to safely refactor your code without breaking previously declared behavior. Testing enables you to iterate and refactor code without constantly releasing new regressions. Testing IS code quality. If you lack tests you lack a core piece of code quality.