4 ms·

The article top recommendations are strong: * Keep system level integration tests [for up to a year]. * Keep unit tests that test key algorithms for which the

by pacala 10y ago

The article top recommendations are strong: * Keep system level integration tests [for up to a year]. * Keep unit tests that test key algorithms for which there is a broad, formal, independent oracle of correctness, and for which there is ascribable business value. In my experience, an overarching test layering strategy works best in providing maximum coverage for minimum friction: Add tests bottom-up, at each layer testing the core functionality of that layer, using directly the layers beneath. The most valuable tests are the system-level tests, which, hopefully, exercise a large swath of the underlying codebase. This reduces to the author's recommendation for most cases. Some people are proud to ditch UnitTesting[TM] to avoid the unit test cargo cult, in particular, the proliferation of "decoupled" tests via overuse of mocking libraries. There is very little value in "unit tests" that reproduce the code flow via mocking asserts, alas a lot of codebases that are heavy on unit testing degenerate into gratuitous mock fests, which become incredibly onerous to work with.

je42 10y ago
No. It should have been. * Keep system level integration until requirement for system level integration change * Keep unittests. And increase code coverage. Any code that is not executed in a unittest will eventually break. Especially in interpreted languages. * Obviously code coverage is not enough. But in my experience code coverage is minimum requirement. Not something to be happy if reached. * The biz cases also need to be tested, which is usually done by a combination of unittest, integration system, system tests and end 2 end tests.
- imtringued 10y ago
  And now the burden of changing tests prevents any meaningful refactoring.
- ryanbrunner 10y ago
  Honest question. What's the value of a test like this? class MyObjectBuilder def create(object) if validator.valid?(object) factory.save(object) else raise "invalid" end end end def object_builder_test_success object_builder.mock(validator, mock_validator) object_builder.mock(factory, mock_factory) expect_method_call(validator.validate).return(true) expect_method_call(factory.save).return(true) object_builder.create(object) end def object_builder_test_failure object_builder.mock(validator, mock_validator) object_builder.mock(factory, mock_factory) expect_method_call(validator.validate).return(false) assert_raise(object_builder.create(object)) end This test is both extremely typical for strong advocates of TDD and "testable" designs, and also almost completely useless in my mind. The test is literally a more pained version of the implementation of the method (not to mention that there's typically a lot more ceremony around setting up mocks than my pseudocode indicates) It adds value in the sense that if someone randomly types garbage into that file it will break, but it acts as a barrier to refactoring or business requirements change, as pretty much nothing but the exact implementation of the method will satisfy the test, and offers no documentation benefit over the code itself.
  - billsix 10y ago
    Lol I was on a Ruby project like this, I said these tests are tautologies since we're verbosely rewriting the implementation as a test. My yearly review from that project? "Bill does not understand unit testing" Nobody gets fired for writing too many tests.
  - gwright 10y ago
    There is no value in a test like this. This is basically a test of a precondition (object must be valid). In fact there is not much value in the entire if statement since it is also testing a precondition. That leaves: def create(object) factory.save(object) end which is just simple delegation. Perhaps just inline the call to create and get rid of the entire class, method, and tests. It is a waste of time testing for a pre-condition. Ensuring a pre-condition is the responsibility of the caller and tests in that part of the code, or integration tests, should be responsible for identifying problems in that regard.

by pacala 10y ago

je42 10y ago

No. It should have been. * Keep system level integration until requirement for system level integration change * Keep unittests. And increase code coverage. Any code that is not executed in a unittest will eventually break. Especially in interpreted languages. * Obviously code coverage is not enough. But in my experience code coverage is minimum requirement. Not something to be happy if reached. * The biz cases also need to be tested, which is usually done by a combination of unittest, integration system, system tests and end 2 end tests.

imtringued 10y ago
And now the burden of changing tests prevents any meaningful refactoring.
ryanbrunner 10y ago
Honest question. What's the value of a test like this? class MyObjectBuilder def create(object) if validator.valid?(object) factory.save(object) else raise "invalid" end end end def object_builder_test_success object_builder.mock(validator, mock_validator) object_builder.mock(factory, mock_factory) expect_method_call(validator.validate).return(true) expect_method_call(factory.save).return(true) object_builder.create(object) end def object_builder_test_failure object_builder.mock(validator, mock_validator) object_builder.mock(factory, mock_factory) expect_method_call(validator.validate).return(false) assert_raise(object_builder.create(object)) end This test is both extremely typical for strong advocates of TDD and "testable" designs, and also almost completely useless in my mind. The test is literally a more pained version of the implementation of the method (not to mention that there's typically a lot more ceremony around setting up mocks than my pseudocode indicates) It adds value in the sense that if someone randomly types garbage into that file it will break, but it acts as a barrier to refactoring or business requirements change, as pretty much nothing but the exact implementation of the method will satisfy the test, and offers no documentation benefit over the code itself.
- billsix 10y ago
  Lol I was on a Ruby project like this, I said these tests are tautologies since we're verbosely rewriting the implementation as a test. My yearly review from that project? "Bill does not understand unit testing" Nobody gets fired for writing too many tests.
- gwright 10y ago
  There is no value in a test like this. This is basically a test of a precondition (object must be valid). In fact there is not much value in the entire if statement since it is also testing a precondition. That leaves: def create(object) factory.save(object) end which is just simple delegation. Perhaps just inline the call to create and get rid of the entire class, method, and tests. It is a waste of time testing for a pre-condition. Ensuring a pre-condition is the responsibility of the caller and tests in that part of the code, or integration tests, should be responsible for identifying problems in that regard.