3 ms·

Tests written after the feature is implemented can be very useful to keep an eye on regression in the case of teams where multiple developers are checking in ch

Tests written after the feature is implemented can be very useful to keep an eye on regression in the case of teams where multiple developers are checking in changes throughout the day and there is a continuous integration build and test system present. At two companies and across three teams where I've worked as a developer responsible for automation I have found that for integration and system type tests writing tests after feature implementation and properly reporting results following high frequency checkins has been allowed me to have a significant impact on the way we work to the point where some of the leads and management are actually taking testing more seriously than they used to. It's still hard to get rid of the idea that tests are secondary but they're not so eager to cut them as a matter of saving time as before after seeing how they allow us to identify problems faster, and therefore fix them faster. In this article the author looks he's talking from the perspective of mostly a single developer or contractor but for bigger teams tests written after feature implementation are still important so I would say based on my experience don't discount them. That said, his approach to writing tests is something I agree with, rather than rigidly sticking to a particular methodology he is pragmatic and it's better than doing nothing. Also he doesn't mention this but solid reporting is so important. I am involved with Tesults (https://www.tesults.com https://www.tesults.com) which is a test results reporting application designed for teams of 10 or more to integrate into their build/test system. If you work for a team of this size and are serious about automated testing check it out. Send me an email if you're thinking of introducing it to your team and have any questions or feedback.

aisofteng 10y ago
I've mentioned this in a couple other comments here, but it bears repeating here because this comment comes so close to what I feel is truly the most important principle, but falls just short. Yes, you can construct a regression test suite out of tests written after implementation, and this test suite will hopefully catch incorrect or incomplete alterations made to the software later. However, the _reason_ it can do that is because these tests will have been written, ideally consciously _because_ of this underlying reason, with the original design of the software in mind; they will be essentially a written record of what the software was _intended_ to do and was designed to do. If one of these tests fails, it means that a design element has been incorrectly or incompletely altered; a well-written unit test of this sort will have an easy to follow flow which will, upon failure, clearly communicate to the programmer which original design element he has not correctly altered and under which specific conditions the error actually occurs. Writing good tests after implementation can indeed encode a hugely significant amount of information. (Of course, it should also test intended behavior with respect to a wide range of both good or bad inputs, but that is beside the point I want to drive home here.) Well enough written unit tests can be used as documentation of an application's design! [N.B. The above thesis really only holds if the authors of the original code write unit tests, ideally as they develop while any design considerations are fresh in memory. As discussed in another comment, sending other engineers back later to try to figure out what the design was and then unit test their interpretation of they think it must have been will almost definitely not give anywhere near the same quality.]
- shshhdhs 10y ago
  >> Well enough written unit tests can be used as documentation of an application's design! That sounds really cool. Is there an open source project that you respect that has achieved this, that you can point me to? I'd like to look at it.
  - cjcenizal 10y ago
    Have a look at the unit tests for the components and services in https://github.com/smaato/ui-framework https://github.com/smaato/ui-framework. We designed the test suites to explore and explain how the interface of each module works. So reading the tests is one way to get a feel for how each module is designed to be used.
  - mannykannot 10y ago
    While a test suite gives you a collection of examples of use, it doesn't tell you the intended purpose and motivation for the software, why it is designed the way it is, provide any argument for correctness or best-, typical- and worst-case performance, or discuss what trade-offs were made, and why. It is often hard to divine the underlying rules from a set of examples, and they don't go out of their way to point out which are the corner cases. There is a great deal of information in the design of any non-trivial piece of software that is not efficiently expounded by tests.

aisofteng 10y ago

I've mentioned this in a couple other comments here, but it bears repeating here because this comment comes so close to what I feel is truly the most important principle, but falls just short. Yes, you can construct a regression test suite out of tests written after implementation, and this test suite will hopefully catch incorrect or incomplete alterations made to the software later. However, the _reason_ it can do that is because these tests will have been written, ideally consciously _because_ of this underlying reason, with the original design of the software in mind; they will be essentially a written record of what the software was _intended_ to do and was designed to do. If one of these tests fails, it means that a design element has been incorrectly or incompletely altered; a well-written unit test of this sort will have an easy to follow flow which will, upon failure, clearly communicate to the programmer which original design element he has not correctly altered and under which specific conditions the error actually occurs. Writing good tests after implementation can indeed encode a hugely significant amount of information. (Of course, it should also test intended behavior with respect to a wide range of both good or bad inputs, but that is beside the point I want to drive home here.) Well enough written unit tests can be used as documentation of an application's design! [N.B. The above thesis really only holds if the authors of the original code write unit tests, ideally as they develop while any design considerations are fresh in memory. As discussed in another comment, sending other engineers back later to try to figure out what the design was and then unit test their interpretation of they think it must have been will almost definitely not give anywhere near the same quality.]

shshhdhs 10y ago
>> Well enough written unit tests can be used as documentation of an application's design! That sounds really cool. Is there an open source project that you respect that has achieved this, that you can point me to? I'd like to look at it.
- cjcenizal 10y ago
  Have a look at the unit tests for the components and services in https://github.com/smaato/ui-framework https://github.com/smaato/ui-framework. We designed the test suites to explore and explain how the interface of each module works. So reading the tests is one way to get a feel for how each module is designed to be used.
- mannykannot 10y ago
  While a test suite gives you a collection of examples of use, it doesn't tell you the intended purpose and motivation for the software, why it is designed the way it is, provide any argument for correctness or best-, typical- and worst-case performance, or discuss what trade-offs were made, and why. It is often hard to divine the underlying rules from a set of examples, and they don't go out of their way to point out which are the corner cases. There is a great deal of information in the design of any non-trivial piece of software that is not efficiently expounded by tests.