7 ms·

I hear this argument a lot, but I do not think it is valid. What happens is that you simply put in the time before a bug occurs with every unit test that you wr

I hear this argument a lot, but I do not think it is valid. What happens is that you simply put in the time before a bug occurs with every unit test that you write. Which isn't very efficient, seeing as not all of your units will lead to bugs. So yes, once you have all of your Units covered 100% it will be easier to find bugs, but you have invested a lot of time in order to get here and in order to keep the unit-test-suite maintained.

Klathmon 8y ago
It's a trade off of when you want to spend the time. Sure, it's less efficient to write unit tests ahead of time for every case, but in a lot of cases fixing a bug faster when it's discovered is much more important than even double the hours spent during less "pressing" times.
- skohan 8y ago
  But I think the argument that the article is making is that it's not only the case that you're moving the time spent fixing bugs, (i.e. a greedy vs lazy strategy), the issue is also that the baseline maintenance cost of the code base is increased by orders of magnitude when unit tests are considered. I tend to agree - in my personal experience in organizations with strict TDD culture, a perverse incentive often emerges to preserve existing flawed architecture over obviously better solutions just because it's so painful to deal with all the tests. One of software development's most powerful properties is the ability to iterate quickly: it's foolish to prioritize dogmatic beliefs about testing over that quality.
  - Klathmon 8y ago
    Yeah I absolutely agree that in many cases tests become more of burden than a help, but I think that isn't a problem with testing but with it's application. It's just an extension of your code, if you are going to throw some code away to change an interface, then throw the tests away too. If you are afraid to do that because of the time spent, then you probably spent too much time on writing tests. While I am dogmatic about tests, I also believe that around 50% code coverage is normally enough in most application codebases. Cover the important parts, the parts that are hard to test manually, the "core" pieces that lots of other areas rely on, and some tests for bugs that you want to prevent happening again. If you want to quickly iterate, go for it! You shouldn't have all that many tests in the parts you are changing frequently. But to change a core aspect of the codebase, or a really complicated aspect of it, then the extra work of rewriting the tests shouldn't be all that bad. I'd shorten your last statement. It's foolish to prioritize dogmatic beliefs. Everything has a time and a place, and moderation is key.
pron 8y ago
But the goal of tests isn't just to find bugs in newly-written code. It's to defend against regressions, and a regression is even harder to localize, given the team may not even be well-familiar with the failing code.
- bluGill 8y ago
  This is both correct and incorrect. Correct that the goal is to prevent regressions. I claim (I don't know how to study this) 80% of your tests will never fail and so they could safely be deleted - but I have no insight into which tests will fail so I say keep them all. Incorrect because in fact it isn't hard to localize failures: it is something in the code you just touched!
  - abecedarius 8y ago
    I don't understand how you could live the experience that the problem is always with the just-edited code. The closest I can come is supposing that you've always worked with thoroughly unit-tested code (and correct, well-documented libraries, etc.).
    - bluGill 8y ago
      I'm lucky enough that the code base I work on was a "big rewrite" not long enough with the goal of having everything tested. Also we are an embedded system where we know exactly which version of each library to support and upgrading any library is in itself a big deal done as a separate exercise.
  - pron 8y ago
    > it is something in the code you just touched! Yes, but you also need to localize the effect of the bug to know why is it that the code you changed broke the program (and remember that we're talking about a case where the rest of the program is not familiar to you enough). Good unit tests can help you find the immediate effect of the failure, rather than the ultimate one.
- bunderbunder 8y ago
ams6110 8y ago
Maybe I'm a worse than average coder, but I find it nearly impossible to write a "unit" of code without also writing one or two bugs along the way. So whether you do your testing ad hoc as you develop, or write unit tests, you do have to spend time in testing each unit of code you write. It doesn't seem to me that formalizing this into a repeatable unit test is adding a lot of real extra work.
- Retric 8y ago
  I would suggest trying to write correct code without unit tests. They fill up mental capacity making it harder to think about then write correct code in the first place. Keeping below 1 bug per 100 lines of code is viable simply by being careful and thinking things through. That's a long way from perfection, but it really helps.
  - verinus 8y ago
    THen why not think "in-code"? I mean thinking through all aspects of edge cases and the like would likely result in some written documentation- why not do it in code then?
    - Retric 8y ago
      You can only focus on a relatively small number of different things at the exact same time. 1 3 2 5 9 2 8 7 vs. 1 3 2 5 9 2 8 7 Memorizing the first sequence is trivial, but dealing with each half independently is much less mental effort. Now each idea can be more complex than just a number, but even still you make fewer mistakes by removing mental overhead. PS: Now, their are a lot of tricks on how you can get better at this stuff. But if the parent poster tends to write bugs in most functions then simplifying things may help.
  - blktiger 8y ago
    I find that writing unit tests helps me think about the code so that I write better code. Plus it generally forces me to make the code more modular than I might otherwise. YMMV
eeZah7Ux 8y ago
> Units covered 100% This metric means very little. It does not measure the extent of code path coverage and much more.
- rhencke 8y ago
  There is only one thing I have found code coverage is good for. Code coverage can tell you what code is not tested at all. Now, this is very useful. But: Code coverage can't tell you what code _is tested_. Code coverage can't tell you how _well_ code is tested.
  - ravenstine 8y ago
    It's also a good way to find parts of the code that are not being used and are worth deleting.
- mosselman 8y ago
  Yes that is the essence of what I was saying.
thephyber 8y ago
I have found unit tests are useful for the original developer to very explicitly describe what test cases the code was written for (including parameter data types). The lack of unit tests for specific types of use cases can also signal to the inheriting developer what the code wasn't originally expected to do (which can signal performance limitations).

Klathmon 8y ago

It's a trade off of when you want to spend the time. Sure, it's less efficient to write unit tests ahead of time for every case, but in a lot of cases fixing a bug faster when it's discovered is much more important than even double the hours spent during less "pressing" times.

skohan 8y ago
But I think the argument that the article is making is that it's not only the case that you're moving the time spent fixing bugs, (i.e. a greedy vs lazy strategy), the issue is also that the baseline maintenance cost of the code base is increased by orders of magnitude when unit tests are considered. I tend to agree - in my personal experience in organizations with strict TDD culture, a perverse incentive often emerges to preserve existing flawed architecture over obviously better solutions just because it's so painful to deal with all the tests. One of software development's most powerful properties is the ability to iterate quickly: it's foolish to prioritize dogmatic beliefs about testing over that quality.
- Klathmon 8y ago
  Yeah I absolutely agree that in many cases tests become more of burden than a help, but I think that isn't a problem with testing but with it's application. It's just an extension of your code, if you are going to throw some code away to change an interface, then throw the tests away too. If you are afraid to do that because of the time spent, then you probably spent too much time on writing tests. While I am dogmatic about tests, I also believe that around 50% code coverage is normally enough in most application codebases. Cover the important parts, the parts that are hard to test manually, the "core" pieces that lots of other areas rely on, and some tests for bugs that you want to prevent happening again. If you want to quickly iterate, go for it! You shouldn't have all that many tests in the parts you are changing frequently. But to change a core aspect of the codebase, or a really complicated aspect of it, then the extra work of rewriting the tests shouldn't be all that bad. I'd shorten your last statement. It's foolish to prioritize dogmatic beliefs. Everything has a time and a place, and moderation is key.

pron 8y ago

But the goal of tests isn't just to find bugs in newly-written code. It's to defend against regressions, and a regression is even harder to localize, given the team may not even be well-familiar with the failing code.

bluGill 8y ago
This is both correct and incorrect. Correct that the goal is to prevent regressions. I claim (I don't know how to study this) 80% of your tests will never fail and so they could safely be deleted - but I have no insight into which tests will fail so I say keep them all. Incorrect because in fact it isn't hard to localize failures: it is something in the code you just touched!
- abecedarius 8y ago
  I don't understand how you could live the experience that the problem is always with the just-edited code. The closest I can come is supposing that you've always worked with thoroughly unit-tested code (and correct, well-documented libraries, etc.).
  - bluGill 8y ago
    I'm lucky enough that the code base I work on was a "big rewrite" not long enough with the goal of having everything tested. Also we are an embedded system where we know exactly which version of each library to support and upgrading any library is in itself a big deal done as a separate exercise.
- pron 8y ago
  > it is something in the code you just touched! Yes, but you also need to localize the effect of the bug to know why is it that the code you changed broke the program (and remember that we're talking about a case where the rest of the program is not familiar to you enough). Good unit tests can help you find the immediate effect of the failure, rather than the ultimate one.
bunderbunder 8y ago

ams6110 8y ago

Maybe I'm a worse than average coder, but I find it nearly impossible to write a "unit" of code without also writing one or two bugs along the way. So whether you do your testing ad hoc as you develop, or write unit tests, you do have to spend time in testing each unit of code you write. It doesn't seem to me that formalizing this into a repeatable unit test is adding a lot of real extra work.

Retric 8y ago
I would suggest trying to write correct code without unit tests. They fill up mental capacity making it harder to think about then write correct code in the first place. Keeping below 1 bug per 100 lines of code is viable simply by being careful and thinking things through. That's a long way from perfection, but it really helps.
- verinus 8y ago
  THen why not think "in-code"? I mean thinking through all aspects of edge cases and the like would likely result in some written documentation- why not do it in code then?
  - Retric 8y ago
    You can only focus on a relatively small number of different things at the exact same time. 1 3 2 5 9 2 8 7 vs. 1 3 2 5 9 2 8 7 Memorizing the first sequence is trivial, but dealing with each half independently is much less mental effort. Now each idea can be more complex than just a number, but even still you make fewer mistakes by removing mental overhead. PS: Now, their are a lot of tricks on how you can get better at this stuff. But if the parent poster tends to write bugs in most functions then simplifying things may help.
- blktiger 8y ago
  I find that writing unit tests helps me think about the code so that I write better code. Plus it generally forces me to make the code more modular than I might otherwise. YMMV

eeZah7Ux 8y ago

> Units covered 100% This metric means very little. It does not measure the extent of code path coverage and much more.

rhencke 8y ago
There is only one thing I have found code coverage is good for. Code coverage can tell you what code is not tested at all. Now, this is very useful. But: Code coverage can't tell you what code _is tested_. Code coverage can't tell you how _well_ code is tested.
- ravenstine 8y ago
  It's also a good way to find parts of the code that are not being used and are worth deleting.
mosselman 8y ago
Yes that is the essence of what I was saying.

thephyber 8y ago

I have found unit tests are useful for the original developer to very explicitly describe what test cases the code was written for (including parameter data types). The lack of unit tests for specific types of use cases can also signal to the inheriting developer what the code wasn't originally expected to do (which can signal performance limitations).