5 ms·

Random question from a web developer who has zero knowledge of C++ or game development: I see no tests in this repo. Is this normal, or am I missing something?

• aap_ 6y ago
  It's quite hard to test heavily interactive code like this. Not totally impossible but you have to question if it's worth the effort.
  • softwhale 6y ago
    It may be interesting to note that GTA:V's leaked build logs featured some notes about an "auto play bot test failed at {mission}". I'm pretty sure they would have to have similar tests (although perhaps at a smaller scale) for older GTA's as well.
• rushilhalflife 6y ago
  I had a short stint at a AAA games company and I worked on one of the games. It was all written in C++ and we had absolutely 0 unit tests. All the testing was done by an army of QA guys.
  • tnolet 6y ago
    That is wildly interesting. Every SaaS startup doing some 100k of $$$ in ARR will probably have long testing suites. AAA games making possibly tens of millions. 0 tests.
    • mschuster91 6y ago
      Lots of things are at play here (excuse the bad pun): * SaaS stuff can be built very quickly (if you use cached node_modules or PHP composer's vendor directory) and deployed to parallel CI test runners that don't have much requirements. In contrast, your typical game takes much longer to (re)compile and as soon as you touch anything that requires graphics, you need some sort of (beefy) GPU in the server. Anyone who has ever tried virtualization with PCIe passthrough knows what a pain that is to set up, not to mention NVIDIA explicitly disallowing to virtualize their GPUs. * It's easy to do unit tests for simple entities and operations (e.g. collision detection, physics)... but games are full of global state mutated from across the place. * For everything that tests visual (or, god forbid, audio) output, computers are bad at testing. Change a texture? Great, now you have to re-check all your tests if they're still accurate and working. Humans are, for now, still better. * Hardware-specific bugs: these are one of a goddamn nightmare. For most software, it behaves identically across computers - let's take most Docker containerized software... it will essentially run on anything that's amd64. But games integrate with OSes, especially the OS 3D stack (DX, OpenGL, Vulkan, Metal and on top of that the GPU drivers and GPU firmware) so incredibly deep that there are lots and lots of potential issues and workarounds. It's simply impossible to test for these. * Closed source components in the loop: Games ship with all sorts of blobs, from video codecs to sound effect libraries. These are in your address space, but you can't really test them. * Human tests are cheaper than writing test suites and setting up CI test farms (see the first point in my list).
    • otabdeveloper4 6y ago
      Because unit tests are usually worthless, except in some specific circumstances.
      • throw1234651234 6y ago
        This. We write unit tests when we do math on financial transactions, clear cut migrations of data, and other very well-defined, high risk circumstances. The other use-case for tests is explaining the expectation to new developers / teams under tight timelines. Example - we had a "surge" team come in - we were writing software that pulls data out of emails in hundreds of formats. We set up infrastructure for it that says - here is your input, here is your ouput, here is 100 sample records, make unit tests pass for them. This worked amazingly well when you had a team writing RegEx, another team writing csv stuff, one team using ML, etc. Basically, the goal was "make it work, in 2 weeks, no one will ever touch this code again." Btw, the latter is ALWAYS, ALWAYS a lie - if a company sinks a few hundred thousand into a project, you WILL touch it again. Anyway, this was more of a rant - basically unit tests ARE useless 90% of the time. Unit tests in Angular that are basically integration tests are absolutely disgusting. Keep in mind that if you have coverage, any time you modify a class, you are now fixing a bunch of broken tests. I guess that's why I am for testing individual methods and SRP.
      • nmfisher 6y ago
        While I totally agree with you, I also wonder if unit testing is used in the engine itself.
    • ThrowawayR2 6y ago
      That's because in the games industry there is no expectation of a v 2.0 (most games never sell well enough to warrant it) and therefore little concern about mantainability or the future. It's hard enough to get the 1.0 release out the door without the studio running out of budget.
• Yavvn 6y ago
  Game designer checking in. Unit tests are very rarely used in games. This is partially because they are so interactive, and partially because it would take a lot of time away from something else. The video game industry has always been under severe crunch. Why would you waste time writing something to check for a bug rather than fixing a different bug or adding a cool new feature? It’s not the healthiest approach but it was inevitable now that we can patch games at any moment. Here is an example of a game called Factorio running its own unit tests. It’s a lot easier for this game as its deterministic and doesn’t need player interaction to function. https://youtu.be/LXnyTZBmfXM https://youtu.be/LXnyTZBmfXM There have also been attempts to use machine learning agents that act as human players. It’s going to be a while but the idea is that it would automate playtests which are used to hunt down the same bugs. https://youtu.be/ZZsSx6kAi6Y https://youtu.be/ZZsSx6kAi6Y
• softwhale 6y ago
  I'm pretty sure that Rockstar North would have some tests in place. The creators of the reverse engineered project, simply couldn't access these tests as they wouldn't be included in the build. (At least, I'd imagine). Also: It may be interesting to note that GTA:V's leaked build logs featured some notes about an "auto play bot test failed at {mission}". I'm pretty sure they would have to have similar tests (although perhaps at a smaller scale) for older GTA's as well.
• stefan_ 6y ago
  Lots of games are throwaway code - you essentially start over for the next one. That's wildly inefficient of course, which is why the generic reusable parts have been factored out into huge engines like Unreal Engine, and those generally have some test coverage.
• phendrenad2 6y ago
  It's normal. It's incredibly difficult to unit test most things in games (graphics, sound, player controls). Also most game logic is pretty simple and it either works or it very noticeably doesn't. That said, often game logic has some unit tests.