3 ms·

Abandoning unit tests was a thing many companies were proud to tell me while I was interviewing earlier this year. I always thought that was ridiculous - but i

by vdnkh 10y ago

Abandoning unit tests was a thing many companies were proud to tell me while I was interviewing earlier this year. I always thought that was ridiculous - but i suppose their products didn't have so many users, or thought they could tolerate some bad behavior. I'm happy I ended up at a place where we're big on tests and even have SDETs embedded in within our team. Besides being useful, I often use a unit test as my main method of completing a feature. I also believe it's a code smell when it's hard to write a test. Maybe it's a regional thing - I've heard it said that here in nyc were much more strict with testing (a carryover from financial roots).

brightball 10y ago
I'm all for testing until it gets in the way of a proper solution. I've spent more time fixing tests because we changed something on purpose than I ever have because they caught something we missed. I've also spent time working with developers who cared more about their tests than their code or customers. Like anything else, all things in moderation.
- usrusr 10y ago
  > I've spent more time fixing tests because we changed something on purpose than I ever have because they caught something we missed. That's why I love the "dump results into a single large string and compare" style of testing. Declaring a given output as known good (until proven otherwise) is a bit of a gamble, because yes, you do think harder when you are writing individual asserts. But when something changes it is very easy to tell a bad, unexpected change in the output from a good one and replacing the previous known good copy with the new one is just a few keystrokes away. A comfortable anything-to-JSON tool and a good diff viewer should be part of any testing arsenal.
pacala 10y ago
The article top recommendations are strong: * Keep system level integration tests [for up to a year]. * Keep unit tests that test key algorithms for which there is a broad, formal, independent oracle of correctness, and for which there is ascribable business value. In my experience, an overarching test layering strategy works best in providing maximum coverage for minimum friction: Add tests bottom-up, at each layer testing the core functionality of that layer, using directly the layers beneath. The most valuable tests are the system-level tests, which, hopefully, exercise a large swath of the underlying codebase. This reduces to the author's recommendation for most cases. Some people are proud to ditch UnitTesting[TM] to avoid the unit test cargo cult, in particular, the proliferation of "decoupled" tests via overuse of mocking libraries. There is very little value in "unit tests" that reproduce the code flow via mocking asserts, alas a lot of codebases that are heavy on unit testing degenerate into gratuitous mock fests, which become incredibly onerous to work with.
- je42 10y ago
  No. It should have been. * Keep system level integration until requirement for system level integration change * Keep unittests. And increase code coverage. Any code that is not executed in a unittest will eventually break. Especially in interpreted languages. * Obviously code coverage is not enough. But in my experience code coverage is minimum requirement. Not something to be happy if reached. * The biz cases also need to be tested, which is usually done by a combination of unittest, integration system, system tests and end 2 end tests.
  - imtringued 10y ago
    And now the burden of changing tests prevents any meaningful refactoring.
  - ryanbrunner 10y ago
    Honest question. What's the value of a test like this? class MyObjectBuilder def create(object) if validator.valid?(object) factory.save(object) else raise "invalid" end end end def object_builder_test_success object_builder.mock(validator, mock_validator) object_builder.mock(factory, mock_factory) expect_method_call(validator.validate).return(true) expect_method_call(factory.save).return(true) object_builder.create(object) end def object_builder_test_failure object_builder.mock(validator, mock_validator) object_builder.mock(factory, mock_factory) expect_method_call(validator.validate).return(false) assert_raise(object_builder.create(object)) end This test is both extremely typical for strong advocates of TDD and "testable" designs, and also almost completely useless in my mind. The test is literally a more pained version of the implementation of the method (not to mention that there's typically a lot more ceremony around setting up mocks than my pseudocode indicates) It adds value in the sense that if someone randomly types garbage into that file it will break, but it acts as a barrier to refactoring or business requirements change, as pretty much nothing but the exact implementation of the method will satisfy the test, and offers no documentation benefit over the code itself.
    - 10y ago
dimino 10y ago
When your cost to fix a bug is very low, maybe you don't need unit tests. Every middle manager on the planet thinks their product/system/whatever is the most critical part of the business, and any errors are unrecoverable and must be avoided, regardless of cost. People tend to forget that testing comes with cost, and pretending like there are zero benefits to not performing certain types of testing is just asking for your competitors to outpace you. If you build your system the right way, bugs simply can't kill you.
- deleted 10y ago
  [deleted]
blub 10y ago
The author doesn't argue that unit tests should be abandoned, so I don't see the point of your comment. It's also contradictory to say that "features" can be tested as a single unit, aka function or class.

by vdnkh 10y ago

brightball 10y ago

I'm all for testing until it gets in the way of a proper solution. I've spent more time fixing tests because we changed something on purpose than I ever have because they caught something we missed. I've also spent time working with developers who cared more about their tests than their code or customers. Like anything else, all things in moderation.

usrusr 10y ago
> I've spent more time fixing tests because we changed something on purpose than I ever have because they caught something we missed. That's why I love the "dump results into a single large string and compare" style of testing. Declaring a given output as known good (until proven otherwise) is a bit of a gamble, because yes, you do think harder when you are writing individual asserts. But when something changes it is very easy to tell a bad, unexpected change in the output from a good one and replacing the previous known good copy with the new one is just a few keystrokes away. A comfortable anything-to-JSON tool and a good diff viewer should be part of any testing arsenal.

pacala 10y ago

The article top recommendations are strong: * Keep system level integration tests [for up to a year]. * Keep unit tests that test key algorithms for which there is a broad, formal, independent oracle of correctness, and for which there is ascribable business value. In my experience, an overarching test layering strategy works best in providing maximum coverage for minimum friction: Add tests bottom-up, at each layer testing the core functionality of that layer, using directly the layers beneath. The most valuable tests are the system-level tests, which, hopefully, exercise a large swath of the underlying codebase. This reduces to the author's recommendation for most cases. Some people are proud to ditch UnitTesting[TM] to avoid the unit test cargo cult, in particular, the proliferation of "decoupled" tests via overuse of mocking libraries. There is very little value in "unit tests" that reproduce the code flow via mocking asserts, alas a lot of codebases that are heavy on unit testing degenerate into gratuitous mock fests, which become incredibly onerous to work with.

je42 10y ago
No. It should have been. * Keep system level integration until requirement for system level integration change * Keep unittests. And increase code coverage. Any code that is not executed in a unittest will eventually break. Especially in interpreted languages. * Obviously code coverage is not enough. But in my experience code coverage is minimum requirement. Not something to be happy if reached. * The biz cases also need to be tested, which is usually done by a combination of unittest, integration system, system tests and end 2 end tests.
- imtringued 10y ago
  And now the burden of changing tests prevents any meaningful refactoring.
- ryanbrunner 10y ago
  Honest question. What's the value of a test like this? class MyObjectBuilder def create(object) if validator.valid?(object) factory.save(object) else raise "invalid" end end end def object_builder_test_success object_builder.mock(validator, mock_validator) object_builder.mock(factory, mock_factory) expect_method_call(validator.validate).return(true) expect_method_call(factory.save).return(true) object_builder.create(object) end def object_builder_test_failure object_builder.mock(validator, mock_validator) object_builder.mock(factory, mock_factory) expect_method_call(validator.validate).return(false) assert_raise(object_builder.create(object)) end This test is both extremely typical for strong advocates of TDD and "testable" designs, and also almost completely useless in my mind. The test is literally a more pained version of the implementation of the method (not to mention that there's typically a lot more ceremony around setting up mocks than my pseudocode indicates) It adds value in the sense that if someone randomly types garbage into that file it will break, but it acts as a barrier to refactoring or business requirements change, as pretty much nothing but the exact implementation of the method will satisfy the test, and offers no documentation benefit over the code itself.
  - 10y ago

dimino 10y ago

When your cost to fix a bug is very low, maybe you don't need unit tests. Every middle manager on the planet thinks their product/system/whatever is the most critical part of the business, and any errors are unrecoverable and must be avoided, regardless of cost. People tend to forget that testing comes with cost, and pretending like there are zero benefits to not performing certain types of testing is just asking for your competitors to outpace you. If you build your system the right way, bugs simply can't kill you.

deleted 10y ago
[deleted]

blub 10y ago

The author doesn't argue that unit tests should be abandoned, so I don't see the point of your comment. It's also contradictory to say that "features" can be tested as a single unit, aka function or class.