4 ms·

I always go back and forth on this one. It always feels great when the tests catch an introduced bug, but it is really worth the time you could be spending on

I always go back and forth on this one. It always feels great when the tests catch an introduced bug, but it is really worth the time you could be spending on testing out new features with users? I wonder if the solution is to be careful to write very simple, well designed code.

nostrademons 19y ago
I've gone back and forth on this issue too. I don't really have a firm opinion on the subject, so all I can do is share my experiences. At inAsphere.com (2000 teen content dot-com), we had no tests and didn't really miss them. Our software was dead-simple though: a bunch of 3rd-party forum and link directory stuff. We got killed for other reasons (i.e. there wasn't really a market and we kinda sucked content-wise). At Traxit (2000-2001 venture backed startup I worked at, selling remote access software to get through firewalls) we had no unit tests, and it sank the company. We got into a QA death spiral, where every bug that QA found made the engineers introduce 5 more, and the product quality suffered so much that we cancelled our launch. Cancelling the launch is the absolute worst thing you can do if you have venture funding, because the VCs will swoop in, throw out most of the management team, and try to pick up the pieces. Of course they'll fail, so the company folded about 6 months after I left. At FictionAlley (2002-2005 nonprofit Harry Potter fanfiction archive), I started with comprehensive unit tests, determined not to repeat the mistakes at Traxit. I then got frustrated with how I was spending as much time writing test code as production code, and abandoned them. It eventually launched, a little buggy, and the users were reasonably happy despite some annoyance at the bugs. However, the lack of up-to-date tests has made it very difficult for new developers to come up to speed and change things, which means that imperfections in the architecture have kinda fossilized. At my previous project at my current employer (2006 IDE plugin), I released without any unit tests. I then spent 2-3 weeks immediately after release writing a fairly comprehensive test suite for it. Overall it's been pretty successful: product quality is fairly high, and the tests have caught several insidious bugs since. I find they're most useful as a sanity check to show me I'm about to do something stupid. Often I'll do something that looks like a trivial refactoring and then roll back when I find out it's made a whole bunch of tests fail. At my current project at my employer (2007 financial webapp), we have no unit tests. We are getting along okay without them, but that could be because we haven't really met reality yet. When my employer has released previous products (again, without any unit tests or QA), we've had to deal with some pretty pissed-off customers because of the product quality or lack thereof. At my startup (2007, flash games website), I've had fairly comprehensive unit tests from the beginning. I haven't been running them lately, and they're a bit out of date, again because it takes as much time to maintain the tests as to write the code. While I was running them, they were useful in the same way as above: they keep me from doing stupid things and making silly mistakes. I plan to go back and make them pass 100% before or soon after launch. If I had to summarize, I'd recommend that you skip them until after launch, and then any time you don't have an imminent feature request, spend that firming up the test suite. That way you don't delay your release too much but they'll still be there for you when you need them, and you also don't start writing them until the feature set has somewhat stabilized. They're a big investment, but they do pay you back.
- palish 19y ago
  For some reason, I really like the philosophy of "Don't write unit tests until right before you launch, then maintain them". Good observation.
  - richcollins 19y ago
    I like that as well
  - irrelative 19y ago
    I second that. Once you've launched, your product is usually fairly stable (in terms of domain, big features, etc) and the tests are worth it for confidence in smaller changes/maintenance. Before then, your code base changes so often that updating the tests start dragging down your productivity.

nostrademons 19y ago

I've gone back and forth on this issue too. I don't really have a firm opinion on the subject, so all I can do is share my experiences. At inAsphere.com (2000 teen content dot-com), we had no tests and didn't really miss them. Our software was dead-simple though: a bunch of 3rd-party forum and link directory stuff. We got killed for other reasons (i.e. there wasn't really a market and we kinda sucked content-wise). At Traxit (2000-2001 venture backed startup I worked at, selling remote access software to get through firewalls) we had no unit tests, and it sank the company. We got into a QA death spiral, where every bug that QA found made the engineers introduce 5 more, and the product quality suffered so much that we cancelled our launch. Cancelling the launch is the absolute worst thing you can do if you have venture funding, because the VCs will swoop in, throw out most of the management team, and try to pick up the pieces. Of course they'll fail, so the company folded about 6 months after I left. At FictionAlley (2002-2005 nonprofit Harry Potter fanfiction archive), I started with comprehensive unit tests, determined not to repeat the mistakes at Traxit. I then got frustrated with how I was spending as much time writing test code as production code, and abandoned them. It eventually launched, a little buggy, and the users were reasonably happy despite some annoyance at the bugs. However, the lack of up-to-date tests has made it very difficult for new developers to come up to speed and change things, which means that imperfections in the architecture have kinda fossilized. At my previous project at my current employer (2006 IDE plugin), I released without any unit tests. I then spent 2-3 weeks immediately after release writing a fairly comprehensive test suite for it. Overall it's been pretty successful: product quality is fairly high, and the tests have caught several insidious bugs since. I find they're most useful as a sanity check to show me I'm about to do something stupid. Often I'll do something that looks like a trivial refactoring and then roll back when I find out it's made a whole bunch of tests fail. At my current project at my employer (2007 financial webapp), we have no unit tests. We are getting along okay without them, but that could be because we haven't really met reality yet. When my employer has released previous products (again, without any unit tests or QA), we've had to deal with some pretty pissed-off customers because of the product quality or lack thereof. At my startup (2007, flash games website), I've had fairly comprehensive unit tests from the beginning. I haven't been running them lately, and they're a bit out of date, again because it takes as much time to maintain the tests as to write the code. While I was running them, they were useful in the same way as above: they keep me from doing stupid things and making silly mistakes. I plan to go back and make them pass 100% before or soon after launch. If I had to summarize, I'd recommend that you skip them until after launch, and then any time you don't have an imminent feature request, spend that firming up the test suite. That way you don't delay your release too much but they'll still be there for you when you need them, and you also don't start writing them until the feature set has somewhat stabilized. They're a big investment, but they do pay you back.

palish 19y ago
For some reason, I really like the philosophy of "Don't write unit tests until right before you launch, then maintain them". Good observation.
- richcollins 19y ago
  I like that as well
- irrelative 19y ago
  I second that. Once you've launched, your product is usually fairly stable (in terms of domain, big features, etc) and the tests are worth it for confidence in smaller changes/maintenance. Before then, your code base changes so often that updating the tests start dragging down your productivity.