3 ms·

Disclaimer: unit-test proponent hypothesizing. > But taking the data at face value, the trend lines speak clearly. More test methods in codebases predict more

Disclaimer: unit-test proponent hypothesizing. > But taking the data at face value, the trend lines speak clearly. More test methods in codebases predict more cyclomatic complexity per method and more lines of code per method My initial reaction to this is that having more tests means your code accommodates more and increasingly complex cases and edge-cases in general, because you’re not going to write 10 test-cases testing the same logic (or combination of logic). That’s going to mean more code and more complex code either you like it or not. The projects with fewer test-cases may not handle as many edge-cases, and maybe they don’t have this complexity (and the associated tests) because they don’t need it yet? If so, tests themselves are not bad, but may just be indicative of the real world cost of real world complexity?

milesvp 9y ago
Quite possibly. I will assert, though, from my experience, and the wisdom I've garnered from talking to people around unit testing, that as code assypmtotically approach 100% code coverage, the complexity of said code will tend to go up exponetially (or worse). I don't have significant proof other than to say that things like DI and IoC start to get used in places that make little sense, which means complexity that wouldn't exist without tests. I also have anecdotes of defensive code I've written, with code branches that 'will never execute'. Branches that are there in case I missed something subtle, that simply log an error, or otherwise throw some exception. I can verify the code works before walling it off, but if you ask me to unit test those lines of code, now I need ways to set the internals of a function externally, or otherwise add code to allow variables to be set in a way that shouldn't be possible. This leads to complexity. And if you asked me to test it, I'd just quietly delete the defensive code rather than poison internal state variables.
- josteink 9y ago
  > as code assypmtotically approach 100% code coverage, the complexity of said code will tend to go up exponetially (or worse). I think attempting to reach 100% coverage is a good direction to steer against while you have low coverage, but not worth it as a an ultimate goal. As you get closer (even as as TDD-proponent), I'm pretty sure the costs outweighs the gains, and you get problems like the ones you describe here. Where a "reasonable" cutoff point may be... Now that's a harder discussion :)
  - bonesss 9y ago
    > Where a "reasonable" cutoff point may be... Now that's a harder discussion :) I think this is something that should really be reflected in your systems overriding architecture... A 100% tested persistent ignorant Domain library is A Good Thing. But your Infrastructure library, or your build scripts, or things that only really make sense as integration tests? Only as much as makes sense. While it's an individual heuristic: if I can see myself arguing for more than 30 minutes why it's meaningless to test something then I have no problem letting it slide ;)
Izkata 9y ago
I'm a heavy proponent of writing tests - even if it's not full-on TDD - at work, to the sometimes annoyance of my co-workers, and my reaction before even reading the results was that for those two, his prediction was wrong. It seems obvious to me: If you have (good) automated tests for correctness, a method can handle more complexity without the programmer needing to handle all paths in their head at once. The tests should let them know if they made a mistake somewhere. On the other hand, if no such tests exist, those methods would be more likely to be written as smaller methods that call each other (maybe one "driver" and a couple "helpers"), so the programmer doesn't need to hold the whole thing in their head at once. The odd part is that the second way can often be better for clarity/maintainability, since those helpers can be reused and the driver can be written at a higher level. But it seems like automated tests give people an out where they don't have to think about that anymore.
- bonesss 9y ago
  Just to expound on your hypothesis (from another OOP TDD-ish proponent): Google defines Cyclomatic Complexity as "a software metric... [that] is a quantitative measure of the number of linearly independent paths through a program's source code." It seems to me that swapping out a dependency, abstracting it, and then feeding multiple variants of the same dependency to some code would greatly increase the number of linearly independent paths through the code. These kinds of refactorings/improvements also tend to greatly increase maintainability, clarity, flexibility, and your programs featureset... By way of example: taking some DB insert code and updating it write to a simplified interface and then injecting components to handle writing to the DB, a local file for testing, AWS S3, and such... A well-factored, clean, reusable, persistence ignorant solution will have a higher cyclomatic complexity than a hard-coded, raw, INSERT. I know which one I want to maintain, though ;)

milesvp 9y ago

Quite possibly. I will assert, though, from my experience, and the wisdom I've garnered from talking to people around unit testing, that as code assypmtotically approach 100% code coverage, the complexity of said code will tend to go up exponetially (or worse). I don't have significant proof other than to say that things like DI and IoC start to get used in places that make little sense, which means complexity that wouldn't exist without tests. I also have anecdotes of defensive code I've written, with code branches that 'will never execute'. Branches that are there in case I missed something subtle, that simply log an error, or otherwise throw some exception. I can verify the code works before walling it off, but if you ask me to unit test those lines of code, now I need ways to set the internals of a function externally, or otherwise add code to allow variables to be set in a way that shouldn't be possible. This leads to complexity. And if you asked me to test it, I'd just quietly delete the defensive code rather than poison internal state variables.

josteink 9y ago
> as code assypmtotically approach 100% code coverage, the complexity of said code will tend to go up exponetially (or worse). I think attempting to reach 100% coverage is a good direction to steer against while you have low coverage, but not worth it as a an ultimate goal. As you get closer (even as as TDD-proponent), I'm pretty sure the costs outweighs the gains, and you get problems like the ones you describe here. Where a "reasonable" cutoff point may be... Now that's a harder discussion :)
- bonesss 9y ago
  > Where a "reasonable" cutoff point may be... Now that's a harder discussion :) I think this is something that should really be reflected in your systems overriding architecture... A 100% tested persistent ignorant Domain library is A Good Thing. But your Infrastructure library, or your build scripts, or things that only really make sense as integration tests? Only as much as makes sense. While it's an individual heuristic: if I can see myself arguing for more than 30 minutes why it's meaningless to test something then I have no problem letting it slide ;)

Izkata 9y ago

I'm a heavy proponent of writing tests - even if it's not full-on TDD - at work, to the sometimes annoyance of my co-workers, and my reaction before even reading the results was that for those two, his prediction was wrong. It seems obvious to me: If you have (good) automated tests for correctness, a method can handle more complexity without the programmer needing to handle all paths in their head at once. The tests should let them know if they made a mistake somewhere. On the other hand, if no such tests exist, those methods would be more likely to be written as smaller methods that call each other (maybe one "driver" and a couple "helpers"), so the programmer doesn't need to hold the whole thing in their head at once. The odd part is that the second way can often be better for clarity/maintainability, since those helpers can be reused and the driver can be written at a higher level. But it seems like automated tests give people an out where they don't have to think about that anymore.

bonesss 9y ago
Just to expound on your hypothesis (from another OOP TDD-ish proponent): Google defines Cyclomatic Complexity as "a software metric... [that] is a quantitative measure of the number of linearly independent paths through a program's source code." It seems to me that swapping out a dependency, abstracting it, and then feeding multiple variants of the same dependency to some code would greatly increase the number of linearly independent paths through the code. These kinds of refactorings/improvements also tend to greatly increase maintainability, clarity, flexibility, and your programs featureset... By way of example: taking some DB insert code and updating it write to a simplified interface and then injecting components to handle writing to the DB, a local file for testing, AWS S3, and such... A well-factored, clean, reusable, persistence ignorant solution will have a higher cyclomatic complexity than a hard-coded, raw, INSERT. I know which one I want to maintain, though ;)