4 ms·

moved from the blog comments section: Woah, man. Woah. I think you're missing the primary benefit here, which is that he avoids regressions. By the time you a

moved from the blog comments section: Woah, man. Woah. I think you're missing the primary benefit here, which is that he avoids regressions. By the time you are adding feature 256, you have 255 features which are going to possibly be affected by the code changes you introduce. This is a huge part of software engineering, and one that has gotten lots of attention over the years. The process you describe for your development sounds extremely tedious, but probably won't break down until month 2 of development on a team of one. Once you reach a level of complexity beyond this, that's where automated testing proves it's salt. The style of testing you're describing is commonly referred to as integration testing or acceptance testing, because it is designed to test the full stack in harmony. There are a number of great frameworks out there to help you do this. Cucumber is the one that's gotten the most love in the circles I am familiar with. You can write your steps in python or javascript, so don't worry that it's written in Ruby. The typical thing to do once you've started doing automated testing is to actually write your tests first, watch them fail, then write the code to make the test pass. This forces you to ensure you have good test coverage (every feature is tested) and has been shown to result in better designed systems. You have tons of reading to do if you want to learn more about this, but hit me up if you want a basic rundown.

0x12 15y ago
> This forces you to ensure you have good test coverage (every feature is tested) And it tests the tests too!
bazookaBen 15y ago
you're talking about Test-Driven-Development (TDD). Any good resources for integrating TDD into javascript based games? I'm still looking for that gem. To expand on testing a bit: at one end, there's unit testing where one tests maybe a single object's behavior in the game. I think for game dev, testing for absolute values are not as important as deterministic ones. We would rather want to know if a player has jumped, died or shot something. Eventually, a functional testing is needed when objects interact with one another. It's like a system of systems. That one's more complicated to design. It might go as far as designing a test that runs through a complete level within miliseconds. Anyone have ideas/resources on this?
- chrisrhoden 15y ago
  I am well aware of testing, TDD, BDD, and have been plugged into the scene for a while. For the server and client both, I really like Jasmine as a testing framework. Of course, there's always cucumber :) http://pivotal.github.com/jasmine/ http://pivotal.github.com/jasmine/
estel 15y ago
Given that the op mentioned that he was an Android developer, it's worth noting that the Android SDK possibly has the very worst testing environment of any widely used system on the planet ever* . * possibly an exaggeration, but it's definitely very very bad.
- chrisrhoden 15y ago
  Java has good unit testing frameworks. Try Obj-C/Cocoa.
- pnathan 15y ago
  So I was working on developing a multicast serial driver. Since I was developing the serial driver, I had to take... over... communication... from the copied-from-ROM driver. And since there was only 1 working communication port, how did I debug? How did I even run a test? The board I was developing on was amazing. It had 1 (one) LED. Blink. Blink. Blink (fast).
  - shabble 15y ago
    This setup is almost exactly like the one in which I first learned C (it had a massive 2048 bytes of SRAM!). As a bonus, it used a dodgy proprietary compiler with a whole sack of undocumented "technical limitations." The good part is I now have a really good understanding of pointer manipulation. The bad part is using it sometimes makes me twitch. Edit: Here's someone who's got us both beat: http://web.archive.org/web/20070613032334/http://ipodlinux.org/stories/piezo/ http://web.archive.org/web/20070613032334/http://ipodlinux.o...
  - JonnieCache 15y ago
    You should have just taped a photoresistor over the LED and serialized your error codes back into your dev box with morse code. But I guess then you'd have to debug that code and you'd be back where you started.
  - atomicdog 15y ago
    http://3.bp.blogspot.com/_AxcE8NTHeDU/TOW6kRd-FGI/AAAAAAAAAyw/nvIFtsPS5TU/s400/pike.jpg http://3.bp.blogspot.com/_AxcE8NTHeDU/TOW6kRd-FGI/AAAAAAAAAy...
genbattle 15y ago
I think games are hard (if not impossible) to test with unit tests or codified tests like the ones you mentioned. At the company I work for we rely mostly on manual testing for testing the games and applications we make. We do some automated testing in the form of Tinytask tasks that are left on overnight to hammer an application. In terms of release testing, especially with a custom game engine, there's no easy way to codify a test which actually plays through a game where there is some random mechanic, or checks for things like the visual accuracy, windows flashing up on the screen when they're not supposed to, etc. Web development frameworks like Selenium are great for UI testing, but they require identifying interface elements by their IDs, or performing a 100% image recognition match for targeting. If anyone knows of a good UI testing framework that would work for games/DirectX apps, I'd definitely love to hear about it. I'm not bagging automated testing, I think it's a great way to save time and avoid the boring bits. I just don't think it's very relevant to game design, but for the OP's enterprise app job it would definitely be suitable.
- chipsy 15y ago
  I think the #1 reason why games don't need a lot of automated testing is serialization, or rather the common lack of it in most game worlds. If the data gets corrupt, you can reset the game and all is well again. When the rare corner cases pop up, it's easy to play whack-a-mole with them, because they're all "shallow" in the sense of the game starting from a clean reset frequently, so you can usually reproduce the bug quickly. Of course, if the game does need heavy-duty serialization of everything, all bugs are potentially deadly. And this is largely the case for the line-of-business, social, and productivity apps, because that data is considered mission-critical. Corruption is not OK, reverting to backups is to be avoided. When a game has that kind of requirement, things get a lot tougher - and so games have naturally evolved to favor minimizing the save data to nothing or a few stats. And this is borne out by looking for contrapositives. There are two genres that have a history of tending to be buggy because of some form of long-term data corruption: Large-scope RPGs and turn-based strategy games. It's hard to reach the bugs in those games, so it's also hard to fix them.
  - genbattle 15y ago

0x12 15y ago

> This forces you to ensure you have good test coverage (every feature is tested) And it tests the tests too!

bazookaBen 15y ago

you're talking about Test-Driven-Development (TDD). Any good resources for integrating TDD into javascript based games? I'm still looking for that gem. To expand on testing a bit: at one end, there's unit testing where one tests maybe a single object's behavior in the game. I think for game dev, testing for absolute values are not as important as deterministic ones. We would rather want to know if a player has jumped, died or shot something. Eventually, a functional testing is needed when objects interact with one another. It's like a system of systems. That one's more complicated to design. It might go as far as designing a test that runs through a complete level within miliseconds. Anyone have ideas/resources on this?

chrisrhoden 15y ago
I am well aware of testing, TDD, BDD, and have been plugged into the scene for a while. For the server and client both, I really like Jasmine as a testing framework. Of course, there's always cucumber :) http://pivotal.github.com/jasmine/ http://pivotal.github.com/jasmine/

estel 15y ago

Given that the op mentioned that he was an Android developer, it's worth noting that the Android SDK possibly has the very worst testing environment of any widely used system on the planet ever* . * possibly an exaggeration, but it's definitely very very bad.

chrisrhoden 15y ago
Java has good unit testing frameworks. Try Obj-C/Cocoa.
pnathan 15y ago
So I was working on developing a multicast serial driver. Since I was developing the serial driver, I had to take... over... communication... from the copied-from-ROM driver. And since there was only 1 working communication port, how did I debug? How did I even run a test? The board I was developing on was amazing. It had 1 (one) LED. Blink. Blink. Blink (fast).
- shabble 15y ago
  This setup is almost exactly like the one in which I first learned C (it had a massive 2048 bytes of SRAM!). As a bonus, it used a dodgy proprietary compiler with a whole sack of undocumented "technical limitations." The good part is I now have a really good understanding of pointer manipulation. The bad part is using it sometimes makes me twitch. Edit: Here's someone who's got us both beat: http://web.archive.org/web/20070613032334/http://ipodlinux.org/stories/piezo/ http://web.archive.org/web/20070613032334/http://ipodlinux.o...
- JonnieCache 15y ago
  You should have just taped a photoresistor over the LED and serialized your error codes back into your dev box with morse code. But I guess then you'd have to debug that code and you'd be back where you started.
- atomicdog 15y ago
  http://3.bp.blogspot.com/_AxcE8NTHeDU/TOW6kRd-FGI/AAAAAAAAAyw/nvIFtsPS5TU/s400/pike.jpg http://3.bp.blogspot.com/_AxcE8NTHeDU/TOW6kRd-FGI/AAAAAAAAAy...

genbattle 15y ago

I think games are hard (if not impossible) to test with unit tests or codified tests like the ones you mentioned. At the company I work for we rely mostly on manual testing for testing the games and applications we make. We do some automated testing in the form of Tinytask tasks that are left on overnight to hammer an application. In terms of release testing, especially with a custom game engine, there's no easy way to codify a test which actually plays through a game where there is some random mechanic, or checks for things like the visual accuracy, windows flashing up on the screen when they're not supposed to, etc. Web development frameworks like Selenium are great for UI testing, but they require identifying interface elements by their IDs, or performing a 100% image recognition match for targeting. If anyone knows of a good UI testing framework that would work for games/DirectX apps, I'd definitely love to hear about it. I'm not bagging automated testing, I think it's a great way to save time and avoid the boring bits. I just don't think it's very relevant to game design, but for the OP's enterprise app job it would definitely be suitable.

chipsy 15y ago
I think the #1 reason why games don't need a lot of automated testing is serialization, or rather the common lack of it in most game worlds. If the data gets corrupt, you can reset the game and all is well again. When the rare corner cases pop up, it's easy to play whack-a-mole with them, because they're all "shallow" in the sense of the game starting from a clean reset frequently, so you can usually reproduce the bug quickly. Of course, if the game does need heavy-duty serialization of everything, all bugs are potentially deadly. And this is largely the case for the line-of-business, social, and productivity apps, because that data is considered mission-critical. Corruption is not OK, reverting to backups is to be avoided. When a game has that kind of requirement, things get a lot tougher - and so games have naturally evolved to favor minimizing the save data to nothing or a few stats. And this is borne out by looking for contrapositives. There are two genres that have a history of tending to be buggy because of some form of long-term data corruption: Large-scope RPGs and turn-based strategy games. It's hard to reach the bugs in those games, so it's also hard to fix them.
- genbattle 15y ago