4 ms·

I come back to this quote over and over when talking about desktop and mobile apps/games, especially since they tend to be highly state driven view collections

I come back to this quote over and over when talking about desktop and mobile apps/games, especially since they tend to be highly state driven view collections and devs are always trying to come up with DRY patterns to bury important features in subclasses or helper utils. > A large fraction of the flaws in software development are due to programmers not fully understanding all the possible states their code may execute in.

pmarreck 9y ago
This is also simultaneously a strong argument for unit tests (which encode knowledge of that state into a proof of sorts). The limit of a programmer is his/her brain's ability to contain all these possible states. Bugs always come from missing some mental modeling of state or having a flawed conception of it, either at the point of design, the version 1, or the rewrite. OO just buries that state "elsewhere", so things seem easier superficially to contain mentally (but the state is still there, ready to get corrupted and pounce on you). FP makes the state explicit, so you're forced to deal with it at all times (this perhaps not uncoincidentally also makes FP easier to unit-test AND reason about). If managing that state upfront becomes unwieldy, then that becomes a good code smell/indicator that your design is suboptimal. The upshot of all this is that I think that using FP in conjunction with unit-testing reduces bug production by some statistically-significant amount, especially as a codebase grows. We definitely need more empirical data about this, though. But that's what my intuition says.
- grogenaut 9y ago
  one of the hard things about game dev and testing is that there are all sorts of bits of code that don't really have strong "success criteria". A lot of it comes down to "did that feel good" or "does that look better" which is very hard to quantify in unit tests. However I think there's also a huge swath of code in games that can be unit tested. Scorekeeping systems, ai evaluation trees, etc. The industry as a whole has much more of an integration or manual testing bent focus than a unit testing one. I will say that coming from a no tester all dev unit / integration test world to no automated test, 30-120 high skill manual tester world (30 gameplay testers, every other employee using new builds 2x daily), there is definitely something to having lots of really good manual testers with very short feedback loops. People noticed bugs that were hard to test for within 10-15 minutes of checkin on a regular basis. Really if I did a studio I'd have both but then I'd likely be spending too much money and go out of business.
  - lallysingh 9y ago
    Why would unit testing cost extra? It saves money.
    - lgas 9y ago
      Like most technical work, it saves money when done well and costs money when done poorly. Most people find themselves somewhere in the middle most of the time so it turns out the discussion is a little more nuanced than that.
      - grogenaut 9y ago
        Exactly! But it's so much easier to be an unyielding zealot than to think and understand why and if something is working. As I said above I'm a big advocate of tdd... Do it all the time. So when I say I wouldn't do it for parts of games maybe I have a reason (to gp not you)
    - coldtea 9y ago
      Only if you intend to change/refactor stuff later. Which in games never happens: engines are mostly thrown out at the end of the game.
      - pmarreck 9y ago
        Hasn't Gamebryo evolved since the earliest days at Bethesda?
    - grogenaut 9y ago
      Engineers have to do work.. I would unit test core libs... I would also hire manual tester because they were very useful. I'd likely not chase down 100% automated coverage. The goal is to ship a game not have 100% coverage. And I work 100% pure tdd in my day job and side projects so take that for what it's worth.
  - bunderbunder 9y ago
    > there are all sorts of bits of code that don't really have strong "success criteria" The approval testing model is really useful for this sort of stuff. http://approvaltests.com/ http://approvaltests.com/
- nickpsecurity 9y ago
  It's an even stronger argument for Dijkstra's method of Design by Contract, esp with spec-based generation of tests. You not only document the intended behavior: you'll know exactly where it went wrong if any kind of testing violates one of the specs.
- deleted 9y ago
  [deleted]

pmarreck 9y ago

This is also simultaneously a strong argument for unit tests (which encode knowledge of that state into a proof of sorts). The limit of a programmer is his/her brain's ability to contain all these possible states. Bugs always come from missing some mental modeling of state or having a flawed conception of it, either at the point of design, the version 1, or the rewrite. OO just buries that state "elsewhere", so things seem easier superficially to contain mentally (but the state is still there, ready to get corrupted and pounce on you). FP makes the state explicit, so you're forced to deal with it at all times (this perhaps not uncoincidentally also makes FP easier to unit-test AND reason about). If managing that state upfront becomes unwieldy, then that becomes a good code smell/indicator that your design is suboptimal. The upshot of all this is that I think that using FP in conjunction with unit-testing reduces bug production by some statistically-significant amount, especially as a codebase grows. We definitely need more empirical data about this, though. But that's what my intuition says.

grogenaut 9y ago
one of the hard things about game dev and testing is that there are all sorts of bits of code that don't really have strong "success criteria". A lot of it comes down to "did that feel good" or "does that look better" which is very hard to quantify in unit tests. However I think there's also a huge swath of code in games that can be unit tested. Scorekeeping systems, ai evaluation trees, etc. The industry as a whole has much more of an integration or manual testing bent focus than a unit testing one. I will say that coming from a no tester all dev unit / integration test world to no automated test, 30-120 high skill manual tester world (30 gameplay testers, every other employee using new builds 2x daily), there is definitely something to having lots of really good manual testers with very short feedback loops. People noticed bugs that were hard to test for within 10-15 minutes of checkin on a regular basis. Really if I did a studio I'd have both but then I'd likely be spending too much money and go out of business.
- lallysingh 9y ago
  Why would unit testing cost extra? It saves money.
  - lgas 9y ago
    Like most technical work, it saves money when done well and costs money when done poorly. Most people find themselves somewhere in the middle most of the time so it turns out the discussion is a little more nuanced than that.
    - grogenaut 9y ago
      Exactly! But it's so much easier to be an unyielding zealot than to think and understand why and if something is working. As I said above I'm a big advocate of tdd... Do it all the time. So when I say I wouldn't do it for parts of games maybe I have a reason (to gp not you)
  - coldtea 9y ago
    Only if you intend to change/refactor stuff later. Which in games never happens: engines are mostly thrown out at the end of the game.
    - pmarreck 9y ago
      Hasn't Gamebryo evolved since the earliest days at Bethesda?
  - grogenaut 9y ago
    Engineers have to do work.. I would unit test core libs... I would also hire manual tester because they were very useful. I'd likely not chase down 100% automated coverage. The goal is to ship a game not have 100% coverage. And I work 100% pure tdd in my day job and side projects so take that for what it's worth.
- bunderbunder 9y ago
  > there are all sorts of bits of code that don't really have strong "success criteria" The approval testing model is really useful for this sort of stuff. http://approvaltests.com/ http://approvaltests.com/
nickpsecurity 9y ago
It's an even stronger argument for Dijkstra's method of Design by Contract, esp with spec-based generation of tests. You not only document the intended behavior: you'll know exactly where it went wrong if any kind of testing violates one of the specs.
deleted 9y ago
[deleted]