3 ms·

Test-a-single-function style, what most people default to when they hear "unit test", is probably the thing that most leads into that situation. I've been occa

by Izkata 3y ago

Test-a-single-function style, what most people default to when they hear "unit test", is probably the thing that most leads into that situation. I've been occasionally pushing coworkers to restructure code and tests into a higher level style, defining an API boundary and calling it from the rest of the code (as if you're writing a library except it's inside your codebase instead of installed), then writing tests to that API. That does make for some easily refactorable code where the tests don't need to change at all. That pseudo-library style is kind of what "unit test" originally meant: a business unit, not a code unit. Examples simplified it too much and people copied the style instead of the substance, and the original meaning was lost. For another example, some of our recent projects have been on updating daemons, those have also been really good candidates for this since there are clear entry and exit points.

maw 3y ago
> That pseudo-library style is kind of what "unit test" originally meant: a business unit, not a code unit. Examples simplified it too much and people copied the style instead of the substance, and the original meaning was lost. Very interesting. It sounds a bit like what happened with the term "Hungarian notation".
GuB-42 3y ago
But you probably won't get 99% coverage doing that. Some conditions are just too difficult to get from a higher level, and you still need to take them in consideration: hardware-related problems, race conditions, etc... I don't consider it a problem, but it is to those who want to see big numbers.
- yellowapple 3y ago
  I feel like if you're unable to get 99% coverage with that then there's probably some dead code that needs pruned. As for hardware-related problems and race conditions, testing at a higher level of abstraction seems like it'd help more than it'd hurt - in the former case ensuring graceful handling as part of the tests, and in the latter case making the race conditions more likely to hit (and hopefully fix).
  - lll-o-lll 3y ago
    I see the OP as talking about “integration” style tests (say testing against an api) and not being able to capture all of the OS and similar edge case errors in the test. I do think this is a hard problem, particularly if you don’t design for it from the outset. It’s a bit of the old classic “reading from the file never returns an ‘access denied’ error, until it does”. There’s ways and means, but let’s not pretend this is simple.
- jval43 3y ago
  You don't need 99% coverage. You just need enough coverage to be able to refactor the thing without breaking it.
  - shrimp_emoji 3y ago
    I bet the reason they aim for 99% coverage is because they're using Python on JavaScript for some complex software, which means they've roped themselves into being their own compiler. Without all that test coverage, all the bugs are surprise runtime bugs, and the tests take the "surprise" part out.
mythhabit 3y ago
I believe that the original definition, way back before most of us were born, was a test of the smallest possible unit, i.e a single function ect. At least according to Wikipedia, in 1969, it was defined at unit tests, component tests and integration tests. In todays development world, the unittest is primarily a developer tool to help speed up development. One should not be afraid to delete unittests if they are doing refactorings. But the long term value is always in the integration tests.
- bazoom42 3y ago
  Not the smallest unit, just an internal unit or component smaller than the whole of the system. The important thing is not the size of the code affected by the test, the important thing is that a test should verify a single requirement isolated from other requirements. I believe the original distinction between unit test and integration test was that integration test was for when parts developed in isolation by seperate teams was finally integrated. But this tended to be a disaster, which is why continous integration was adopted.
  - mythhabit 3y ago
    Just going of the Wikipedia page on unit testing, in the 1956 paper on programming for the SAGE project, refers to testing as: Parameter testing, assembly testing, shake down testing. In the context, parameter testing is testing of individual sub routines (functions/methods). From there the term unit test was used in the Mercury System as unit testing of "individual programs outside and inside the system". I suppose here, the ethos is the same. Obviously, when reading the rest of the papers, they are clear on the fact, that it is the specifications that the programmer have developed for, that should be tested. That was at the time, synonyms with individual sub routines, and as such it was both the smallest unit and a logical testable unit at the same time. Since then, we've come a long way with programming smaller units with better composability. I'm not sure I agree that the original meaning of a unit test was lost. Perhaps, only one part of the definition was carried over to modern development practices. In any case, I always stand by the fact that long term value is in tests of the "API". Everything else is implementation details.

by Izkata 3y ago

maw 3y ago

> That pseudo-library style is kind of what "unit test" originally meant: a business unit, not a code unit. Examples simplified it too much and people copied the style instead of the substance, and the original meaning was lost. Very interesting. It sounds a bit like what happened with the term "Hungarian notation".

GuB-42 3y ago

But you probably won't get 99% coverage doing that. Some conditions are just too difficult to get from a higher level, and you still need to take them in consideration: hardware-related problems, race conditions, etc... I don't consider it a problem, but it is to those who want to see big numbers.

yellowapple 3y ago
I feel like if you're unable to get 99% coverage with that then there's probably some dead code that needs pruned. As for hardware-related problems and race conditions, testing at a higher level of abstraction seems like it'd help more than it'd hurt - in the former case ensuring graceful handling as part of the tests, and in the latter case making the race conditions more likely to hit (and hopefully fix).
- lll-o-lll 3y ago
  I see the OP as talking about “integration” style tests (say testing against an api) and not being able to capture all of the OS and similar edge case errors in the test. I do think this is a hard problem, particularly if you don’t design for it from the outset. It’s a bit of the old classic “reading from the file never returns an ‘access denied’ error, until it does”. There’s ways and means, but let’s not pretend this is simple.
jval43 3y ago
You don't need 99% coverage. You just need enough coverage to be able to refactor the thing without breaking it.
- shrimp_emoji 3y ago
  I bet the reason they aim for 99% coverage is because they're using Python on JavaScript for some complex software, which means they've roped themselves into being their own compiler. Without all that test coverage, all the bugs are surprise runtime bugs, and the tests take the "surprise" part out.

mythhabit 3y ago

I believe that the original definition, way back before most of us were born, was a test of the smallest possible unit, i.e a single function ect. At least according to Wikipedia, in 1969, it was defined at unit tests, component tests and integration tests. In todays development world, the unittest is primarily a developer tool to help speed up development. One should not be afraid to delete unittests if they are doing refactorings. But the long term value is always in the integration tests.

bazoom42 3y ago
Not the smallest unit, just an internal unit or component smaller than the whole of the system. The important thing is not the size of the code affected by the test, the important thing is that a test should verify a single requirement isolated from other requirements. I believe the original distinction between unit test and integration test was that integration test was for when parts developed in isolation by seperate teams was finally integrated. But this tended to be a disaster, which is why continous integration was adopted.
- mythhabit 3y ago
  Just going of the Wikipedia page on unit testing, in the 1956 paper on programming for the SAGE project, refers to testing as: Parameter testing, assembly testing, shake down testing. In the context, parameter testing is testing of individual sub routines (functions/methods). From there the term unit test was used in the Mercury System as unit testing of "individual programs outside and inside the system". I suppose here, the ethos is the same. Obviously, when reading the rest of the papers, they are clear on the fact, that it is the specifications that the programmer have developed for, that should be tested. That was at the time, synonyms with individual sub routines, and as such it was both the smallest unit and a logical testable unit at the same time. Since then, we've come a long way with programming smaller units with better composability. I'm not sure I agree that the original meaning of a unit test was lost. Perhaps, only one part of the definition was carried over to modern development practices. In any case, I always stand by the fact that long term value is in tests of the "API". Everything else is implementation details.