3 ms·
I have seen statements like 'even bad tests are better than no tests'. I feel that for unit tests, 'no tests are better than bad tests'. Failure in unit tests o
by simula67 8y ago
I have seen statements like 'even bad tests are better than no tests'. I feel that for unit tests, 'no tests are better than bad tests'. Failure in unit tests often break CI and I have seen many tests which tests the structure of the code and not it's intent. For example, I have seen some tests where the product code is putting some element to a list and the test code checks if the element is present in the list. I can understand why people do it ( they are trying to improve "code coverage" ), however if tomorrow you refactor the code to not put that element in the list, but still accomplish the same thing, the test would break. Often, this breaks CI also and now you are sitting late at office trying to figure out what went wrong so you can push your change into the release. This is valuable time that could be spent on new features, fixing bugs or just going home and relaxing. End to end tests ( or integration tests ) are better since they are real user interactions.
If a test fails and it breaks CI, there should be a problem in the product. Move everything else to some "additional testing" bucket and not block developers.
- irundebian 8y agoI think that's not right way to test units. You should test the interface resp. the right execution of the specification of the unit and not testing implementation details. That means you're either testing the return value or testing the inputs which were modified by the unit. If the behavior of a unit changes in such a way that it does something else with its inputs, it's a specification change which requires other test cases.
- nradov 8y agoThis is why you need coding standards for ensuring good quality unit tests, and enforce those standards through rigorous code reviews. Developers shouldn't be able to check in crappy unit tests on the release branch.
- nimblegorilla 8y agoIn an ideal world developers never push something with broken tests to CI. When developers don't run the tests locally it seems like a sign that there are too many bloated integration tests and not enough fast unit tests.