10 ms·

When you shouldn't bother unit testing

44 points by invalid_arg 13y ago

smanuel 13y ago
The tl;dr part should just say "If you haven't started coding yet, then you shouldn't bother much about unit testing" which is.. well... common sense.
- thirsteh 13y ago
  Then test-driven development defies common sense. Its whole point is to write a runnable specification in the form of a test suite before you write the actual code.
  - kleiba 13y ago
    ...which can have its benefits, but can also go terribly wrong if you get lost. (Case in point, Peter Norvig comments on a famous series of TDD blog posts that had the goal of writing a Sudoku solver, but ultimately failed: "And I could see, right from the start, you put these two together, and you could solve this Sudoku thing. He didn’t know that so he was sort of blundering in the dark even though all his code “worked” because he had all these test cases." http://pindancing.blogspot.com.au/2009/09/sudoku-in-coders-at-work.html http://pindancing.blogspot.com.au/2009/09/sudoku-in-coders-a...)
    - thirsteh 13y ago
      Definitely. I'm not a big fan of TDD myself. I think that an advanced system can add a lot of the same checks, leaving you to do just the scrutinous testing via e.g. QuickCheck.
- norswap 13y ago
  Isn't TDD all about writing tests before coding?
  - warmwaffles 13y ago
    I like MDD, money driven development. It usually works wonders ;) I mix between TDD and not writing tests until after a section of the code is done. It really depends on what I'm trying to accomplish. If I'm spiking things, I tend to just write tests to confirm my work and make sure I'm sane. Other times I know exactly what I want and write the tests first then code. Really just depends on the situation.
    - blowski 13y ago
      > I like MDD, money driven development. Good line. A lot of people will shoot back with "in the long run, it will cost you more" but the point is - a startup doesn't even know if there is a long run.
      - warmwaffles 13y ago
        > "in the long run, it will cost you more" This is true if you don't have tests or your tests lack. But if you write tests as you work or a little ahead of yourself, the cost difference will probably be negligible. To each his own.
    - marcosdumay 13y ago
      That. TDD isn't lean. Writting tests for things that were already proven is.
      - dragonwriter 13y ago
        > TDD isn't lean. Overtesting isn't lean, but TDD is a practice which fits very well within lean (particularly, its a practice which prevents waste by doing things that aren't needed, since it concretely bounds what is necessary, as well as the more-often discussed effect of preventing additional downstream waste in the form of rework because of incorrectly implemented work that gets into production.) > Writting tests for things that were already proven is. I'm not sure what that sentence means; it sounds like something decidedly non-lean (if something is "already proven", writing tests for it seems to be a task known to be redundant, and thus pure waste.)
jerf 13y ago
I'd caveat the "you don't know the framework well" by first taking a moment to establish that you can meaningfully test the framework. There's a whole whackload of frameworks out there of all different sorts, even very popular ones, that haven't given a single thought to testability (or only thought about it so late in the process it was too late to do anything about it), either of the framework code itself, or the user code using it, and generally this results in a framework that can't be tested with anything lower than integration testing. I consider this a major negative for a framework. I can't rigorously prove this, but I suspect that also points at having a framework that is going to tend to gratuitously break your code frequently if you try to upgrade, because they'll have no tests for telling if they have. And neither do you.
nahname 13y ago
I've found one simple rule works well for the teams I have been on. If you can afford to re-write the entire application from scratch (considering time and money) then you can afford to skip unit testing. On a side note. You will never learn TDD from a video. If you want to learn TDD, get very familiar with testing in general, pick a domain you are comfortable with and then force yourself to write tests for everything before code. Repeat 2-3 times and you will be better at TDD than 80% of the people who are presenting on it.
- seren 13y ago
  Do you think it makes sense to try to learn TDD on a language you don't know ? So let's say I want to pick Go (as a hobby, not for a particular professional need in the short term), would it make sense to try doing some TDD on my first smallish applications ?
  - Noughmad 13y ago
    Why not? It's nice to make a fresh start, as with a new language you will not fall into your old habits so easily.
  - nahname 13y ago
    I don't think there is anytime not to try to learn it. It will be harder the more unfamiliar you are with everything else (language, libraries, domain, etc...)
Chris_Newton 13y ago
From the introduction in the article, about the merits of TDD: As well as the simple fact that your code will be more reliable, it forces you to write your code in a way that is testable, which in turn will make it loosely coupled and modular. One of the major drawbacks of many current unit testing tools is that in some contexts they can have exactly the opposite effect. If you expose implementation details through your modules’ interfaces that would otherwise be hidden, just in order to support your unit tests, then that makes your code less modular and potentially more tightly coupled. It could be a direct exposure, for example by adding public methods to a class that allow tests to access otherwise private fields in languages like Java or C++. It could also be indirect exposure, for example using a dependency injection scheme to initialize everything so you can substitute test dummies for live code, but then using something at the top of the tree that essentially has to know everything about everything in your application in order to set up all the injected parts. These things happen to a greater or lesser extent depending on many factors: which tools you have, how dynamic your language is, and more. However, I would challenge any claim that using something like TDD necessarily implies that your code will evolve a robust, modular design. Extensive unit testing can promote modularity in some ways, but it’s definitely a two-way street in most contexts today.
- JulianWasTaken 13y ago
  Michael Feathers has a good talk where he discusses this a bit: http://vimeo.com/43659006 http://vimeo.com/43659006
- rst 13y ago
  One trick I've used in Scala code is to put the test suite in the same package as the code under test, which gives the tests access to package-private stuff which are normally inaccessible to other code. That probably could work for Java as well. (And in C++, I guess you could declare the test suite as a "friend" of the code under test --- "C++, the language in which friends can access your private members!".) I feel a bit funny about this, though; it does make the tests explicitly dependent on stuff that, to the rest of the code, is a hidden implementation detail. So, it's probably better to test only the public interface wherever that's feasible...
- 13y ago
textminer 13y ago
The argument against testing seems to be "the Spike", when you're learning a language/concept/framework or are testing out ideas. But startups (at least the two I've worked for the last two years) seem way more heavily invested in the spike than in the carefully-tested production environment. Personally speaking, it's even more exciting to work on. It took me a good year and half to properly appreciate well-optimized, well-tested code in production, and enjoy the challenges associated there (which aren't trivial-- the neatest, ugliest NumPy script can be entirely useless at scale, or in a timeframe useful to users). But I still personally feel flat when the knowledge gains associated with a Spike aren't as frequent. How have other developers and math-types dealt with that disappointing feeling of a plateau? Besides just changing jobs entirely, I mean.
- jabbernotty 13y ago
  > But I still personally feel flat when the knowledge gains associated with a Spike aren't as frequent. How have other developers and math-types dealt with that disappointing feeling of a plateau? Besides just changing jobs entirely, I mean. I'm not entirely sure if I understand you, but it sounds like we are thinking about the same thing. I've started a job working with C# about 8 months ago. It is a comfortable language to work with. But this job, it's problems (the ones we solve) and C# haven't really exposed me to anything new, which leaves me feeling unfulfilled. Is that what you are talking about?
falsedan 13y ago
Kind of hoping that the linked story was a continuation of Ovid's thoughts on (not unit-)testing: http://blogs.perl.org/users/ovid/2013/04/how-to-be-agile-without-testing.html http://blogs.perl.org/users/ovid/2013/04/how-to-be-agile-wit... http://blogs.perl.org/users/ovid/2013/04/when-must-you-test-your-code.html http://blogs.perl.org/users/ovid/2013/04/when-must-you-test-... http://blogs.perl.org/users/ovid/2013/04/code-evolution-versus-intelligent-design.html http://blogs.perl.org/users/ovid/2013/04/code-evolution-vers...
papsosouid 13y ago
I wish TDD advocates would stop equating unit testing with TDD. Just because I am not a member of your wacky cult, doesn't mean I don't use unit tests. This article isn't about when to skip tests, it is about when to skip TDD. Big difference (and the real answer is always).
- Roboprog 13y ago
  I like automated tests when they are low hanging fruit. Most often though, I'll settle for specs that are detailed enough to tell how to operate a feature, rather than just a hand-wave. I'm not too happy when coworkers commit code and don't bother to run it at all :-( See the comment somebody made elsewhere about "money driven development". The cost-benefit ratio has to make sense.
- kasey_junk 13y ago
  TDD is a design tool nothing more. It happens to be a design tool that creates unit tests as a nice side benefit. If you have a better design tool in your arsenal, feel free to use it, but before you slam TDD as a cult, I'd ask what is your design process and why/how is it better than TDD?
  - papsosouid 13y ago
    I am not slamming it, it is a cult. Read what he wrote, these people literally think you either do TDD, or you are wrong. Despite all the actual testing of TDD having found it offers no benefits at all.
    - Morendil 13y ago
      Citation needed.
    - kasey_junk 13y ago
      A) Using the term "cult" is almost certainly a slam. B) Who are "these" people? I am certainly a fan of TDD but I know that there are lots of other processes that work. I know lots of other people that feel like me. C) I would love to see what testing has indicated about TDD, what studies are you referring to? Finally, I'm not interesting in raving about design processes. I am interested in hearing about how other technical people work. What design process do you use? Why do you prefer it to TDD? How have you gone about comparing the 2?
languagehacker 13y ago
Totally disagree with these assertions. You _should_ know these things, but you need to learn them by doing. You'll never learn this stuff if you don't do it.

44 points by invalid_arg 13y ago

The tl;dr part should just say "If you haven't started coding yet, then you shouldn't bother much about unit testing" which is.. well... common sense.

thirsteh 13y ago
Then test-driven development defies common sense. Its whole point is to write a runnable specification in the form of a test suite before you write the actual code.
- kleiba 13y ago
  ...which can have its benefits, but can also go terribly wrong if you get lost. (Case in point, Peter Norvig comments on a famous series of TDD blog posts that had the goal of writing a Sudoku solver, but ultimately failed: "And I could see, right from the start, you put these two together, and you could solve this Sudoku thing. He didn’t know that so he was sort of blundering in the dark even though all his code “worked” because he had all these test cases." http://pindancing.blogspot.com.au/2009/09/sudoku-in-coders-at-work.html http://pindancing.blogspot.com.au/2009/09/sudoku-in-coders-a...)
  - thirsteh 13y ago
    Definitely. I'm not a big fan of TDD myself. I think that an advanced system can add a lot of the same checks, leaving you to do just the scrutinous testing via e.g. QuickCheck.
norswap 13y ago
Isn't TDD all about writing tests before coding?
- warmwaffles 13y ago
  I like MDD, money driven development. It usually works wonders ;) I mix between TDD and not writing tests until after a section of the code is done. It really depends on what I'm trying to accomplish. If I'm spiking things, I tend to just write tests to confirm my work and make sure I'm sane. Other times I know exactly what I want and write the tests first then code. Really just depends on the situation.
  - blowski 13y ago
    > I like MDD, money driven development. Good line. A lot of people will shoot back with "in the long run, it will cost you more" but the point is - a startup doesn't even know if there is a long run.
    - warmwaffles 13y ago
      > "in the long run, it will cost you more" This is true if you don't have tests or your tests lack. But if you write tests as you work or a little ahead of yourself, the cost difference will probably be negligible. To each his own.
  - marcosdumay 13y ago
    That. TDD isn't lean. Writting tests for things that were already proven is.
    - dragonwriter 13y ago
      > TDD isn't lean. Overtesting isn't lean, but TDD is a practice which fits very well within lean (particularly, its a practice which prevents waste by doing things that aren't needed, since it concretely bounds what is necessary, as well as the more-often discussed effect of preventing additional downstream waste in the form of rework because of incorrectly implemented work that gets into production.) > Writting tests for things that were already proven is. I'm not sure what that sentence means; it sounds like something decidedly non-lean (if something is "already proven", writing tests for it seems to be a task known to be redundant, and thus pure waste.)

jerf 13y ago

I'd caveat the "you don't know the framework well" by first taking a moment to establish that you can meaningfully test the framework. There's a whole whackload of frameworks out there of all different sorts, even very popular ones, that haven't given a single thought to testability (or only thought about it so late in the process it was too late to do anything about it), either of the framework code itself, or the user code using it, and generally this results in a framework that can't be tested with anything lower than integration testing. I consider this a major negative for a framework. I can't rigorously prove this, but I suspect that also points at having a framework that is going to tend to gratuitously break your code frequently if you try to upgrade, because they'll have no tests for telling if they have. And neither do you.

nahname 13y ago

I've found one simple rule works well for the teams I have been on. If you can afford to re-write the entire application from scratch (considering time and money) then you can afford to skip unit testing. On a side note. You will never learn TDD from a video. If you want to learn TDD, get very familiar with testing in general, pick a domain you are comfortable with and then force yourself to write tests for everything before code. Repeat 2-3 times and you will be better at TDD than 80% of the people who are presenting on it.

seren 13y ago
Do you think it makes sense to try to learn TDD on a language you don't know ? So let's say I want to pick Go (as a hobby, not for a particular professional need in the short term), would it make sense to try doing some TDD on my first smallish applications ?
- Noughmad 13y ago
  Why not? It's nice to make a fresh start, as with a new language you will not fall into your old habits so easily.
- nahname 13y ago
  I don't think there is anytime not to try to learn it. It will be harder the more unfamiliar you are with everything else (language, libraries, domain, etc...)

Chris_Newton 13y ago

From the introduction in the article, about the merits of TDD: As well as the simple fact that your code will be more reliable, it forces you to write your code in a way that is testable, which in turn will make it loosely coupled and modular. One of the major drawbacks of many current unit testing tools is that in some contexts they can have exactly the opposite effect. If you expose implementation details through your modules’ interfaces that would otherwise be hidden, just in order to support your unit tests, then that makes your code less modular and potentially more tightly coupled. It could be a direct exposure, for example by adding public methods to a class that allow tests to access otherwise private fields in languages like Java or C++. It could also be indirect exposure, for example using a dependency injection scheme to initialize everything so you can substitute test dummies for live code, but then using something at the top of the tree that essentially has to know everything about everything in your application in order to set up all the injected parts. These things happen to a greater or lesser extent depending on many factors: which tools you have, how dynamic your language is, and more. However, I would challenge any claim that using something like TDD necessarily implies that your code will evolve a robust, modular design. Extensive unit testing can promote modularity in some ways, but it’s definitely a two-way street in most contexts today.

JulianWasTaken 13y ago
Michael Feathers has a good talk where he discusses this a bit: http://vimeo.com/43659006 http://vimeo.com/43659006
rst 13y ago
One trick I've used in Scala code is to put the test suite in the same package as the code under test, which gives the tests access to package-private stuff which are normally inaccessible to other code. That probably could work for Java as well. (And in C++, I guess you could declare the test suite as a "friend" of the code under test --- "C++, the language in which friends can access your private members!".) I feel a bit funny about this, though; it does make the tests explicitly dependent on stuff that, to the rest of the code, is a hidden implementation detail. So, it's probably better to test only the public interface wherever that's feasible...
13y ago

textminer 13y ago

The argument against testing seems to be "the Spike", when you're learning a language/concept/framework or are testing out ideas. But startups (at least the two I've worked for the last two years) seem way more heavily invested in the spike than in the carefully-tested production environment. Personally speaking, it's even more exciting to work on. It took me a good year and half to properly appreciate well-optimized, well-tested code in production, and enjoy the challenges associated there (which aren't trivial-- the neatest, ugliest NumPy script can be entirely useless at scale, or in a timeframe useful to users). But I still personally feel flat when the knowledge gains associated with a Spike aren't as frequent. How have other developers and math-types dealt with that disappointing feeling of a plateau? Besides just changing jobs entirely, I mean.

jabbernotty 13y ago
> But I still personally feel flat when the knowledge gains associated with a Spike aren't as frequent. How have other developers and math-types dealt with that disappointing feeling of a plateau? Besides just changing jobs entirely, I mean. I'm not entirely sure if I understand you, but it sounds like we are thinking about the same thing. I've started a job working with C# about 8 months ago. It is a comfortable language to work with. But this job, it's problems (the ones we solve) and C# haven't really exposed me to anything new, which leaves me feeling unfulfilled. Is that what you are talking about?

falsedan 13y ago

Kind of hoping that the linked story was a continuation of Ovid's thoughts on (not unit-)testing: http://blogs.perl.org/users/ovid/2013/04/how-to-be-agile-without-testing.html http://blogs.perl.org/users/ovid/2013/04/how-to-be-agile-wit... http://blogs.perl.org/users/ovid/2013/04/when-must-you-test-your-code.html http://blogs.perl.org/users/ovid/2013/04/when-must-you-test-... http://blogs.perl.org/users/ovid/2013/04/code-evolution-versus-intelligent-design.html http://blogs.perl.org/users/ovid/2013/04/code-evolution-vers...

papsosouid 13y ago

I wish TDD advocates would stop equating unit testing with TDD. Just because I am not a member of your wacky cult, doesn't mean I don't use unit tests. This article isn't about when to skip tests, it is about when to skip TDD. Big difference (and the real answer is always).

Roboprog 13y ago
I like automated tests when they are low hanging fruit. Most often though, I'll settle for specs that are detailed enough to tell how to operate a feature, rather than just a hand-wave. I'm not too happy when coworkers commit code and don't bother to run it at all :-( See the comment somebody made elsewhere about "money driven development". The cost-benefit ratio has to make sense.
kasey_junk 13y ago
TDD is a design tool nothing more. It happens to be a design tool that creates unit tests as a nice side benefit. If you have a better design tool in your arsenal, feel free to use it, but before you slam TDD as a cult, I'd ask what is your design process and why/how is it better than TDD?
- papsosouid 13y ago
  I am not slamming it, it is a cult. Read what he wrote, these people literally think you either do TDD, or you are wrong. Despite all the actual testing of TDD having found it offers no benefits at all.
  - Morendil 13y ago
    Citation needed.
  - kasey_junk 13y ago
    A) Using the term "cult" is almost certainly a slam. B) Who are "these" people? I am certainly a fan of TDD but I know that there are lots of other processes that work. I know lots of other people that feel like me. C) I would love to see what testing has indicated about TDD, what studies are you referring to? Finally, I'm not interesting in raving about design processes. I am interested in hearing about how other technical people work. What design process do you use? Why do you prefer it to TDD? How have you gone about comparing the 2?

languagehacker 13y ago

Totally disagree with these assertions. You _should_ know these things, but you need to learn them by doing. You'll never learn this stuff if you don't do it.