4 ms·

At my current company we have pretty high test coverage. The test suite also happens to be full of mocks plus some cargoculted antipatterns that make many tests

by mkl95 3y ago

At my current company we have pretty high test coverage. The test suite also happens to be full of mocks plus some cargoculted antipatterns that make many tests useless. The engineers who actually test their stuff can be counted with one hand.

stouset 3y ago
I see this so often. Mocks and stubs get used all over the place because nobody understands what it means to write code that’s easily testable. They’re great when used correctly (e.g., remote services or inherently stateful APIs like time). But they almost never are. You end up with tests that ensure one and only one thing: the code is written the way it’s currently written. Tests should do two things: find unexpected out-of-spec behavior, and prevent regressions during the course of editing and refactoring. These overly-mocked tests by definition can’t do the first one and they actively inhibit the second. They have negative value insofar as they constantly trigger failure while making completely benign edits.
- codethatwerks 3y ago
  [flagged]
- keybored 3y ago
  It feels like - First write the needed change - Now assert that we just wrote the needed change
rhdunn 3y ago
This is why I tend to lean toward the approach of only using mocks, stubs, etc. when absolutely needed. That tends to be at the places where the code is interacting with external components like databases or web services than with components that are within the current layer, such as connecting controllers and services. It's also why I don't like the philosophy of unit testing being about only testing a specific class. -- You end up in situations where you convince yourself you have to mock the helper classes it uses, so end up with disconnected pointless tests. Instead, each test should be testing as much real code as possible.
- ThunderSizzle 3y ago
  The best definition for "unit" in unit testing is the test itself. The test is the unit - as in, each test is testing a unit of code and doesn't interfere with other tests or rely on the external environment. Too many times have I been told a class or a method in a class is a unit. It makes no sense and is the reason for excessive mocking, because they end up trying to isolate that class as an arbitrary unit.
  - lamontcg 3y ago
    Yeah that's the biggest issue in testing software. In a perfect world someone would stop and refactor the entire codebase to make every object perfectly unit testable. In the real world, you have to pick between expanding the system under test (SUT) to include multiple real collaborator objects or creating a violent mess of mocks.
    - pydry 3y ago
      It's not the right thing to do in a perfect world either. I've written quite a lot of software without any dependency injection with 0 unit tests that is fully covered by reliable, more than fast enough integration tests. Some people think that if I did dependency injection so I could unit test all of that same code, that code would be "better" in some indefinable way. In reality the code would just be longer (dependency injection increases SLOC). All other things being equal - longer = worse. DI-all-the-things is a dogma hangover from the late 1990s when integration testing and end to end testing tooling was all poor, computers were all slow and people tended to write more of their own algorithms (rather than importing them). DI is now something that should be approached on a case by case basis.
      - lamontcg 3y ago
        Yep, and I've seen code shipped that passed all its unit tests but the interface between them was subtly wrong so it violently failed to work at all after being shipped. Had to write functional tests to make sure it all worked together.

by mkl95 3y ago

stouset 3y ago

I see this so often. Mocks and stubs get used all over the place because nobody understands what it means to write code that’s easily testable. They’re great when used correctly (e.g., remote services or inherently stateful APIs like time). But they almost never are. You end up with tests that ensure one and only one thing: the code is written the way it’s currently written. Tests should do two things: find unexpected out-of-spec behavior, and prevent regressions during the course of editing and refactoring. These overly-mocked tests by definition can’t do the first one and they actively inhibit the second. They have negative value insofar as they constantly trigger failure while making completely benign edits.

codethatwerks 3y ago
[flagged]
keybored 3y ago
It feels like - First write the needed change - Now assert that we just wrote the needed change

rhdunn 3y ago

This is why I tend to lean toward the approach of only using mocks, stubs, etc. when absolutely needed. That tends to be at the places where the code is interacting with external components like databases or web services than with components that are within the current layer, such as connecting controllers and services. It's also why I don't like the philosophy of unit testing being about only testing a specific class. -- You end up in situations where you convince yourself you have to mock the helper classes it uses, so end up with disconnected pointless tests. Instead, each test should be testing as much real code as possible.

ThunderSizzle 3y ago
The best definition for "unit" in unit testing is the test itself. The test is the unit - as in, each test is testing a unit of code and doesn't interfere with other tests or rely on the external environment. Too many times have I been told a class or a method in a class is a unit. It makes no sense and is the reason for excessive mocking, because they end up trying to isolate that class as an arbitrary unit.
- lamontcg 3y ago
  Yeah that's the biggest issue in testing software. In a perfect world someone would stop and refactor the entire codebase to make every object perfectly unit testable. In the real world, you have to pick between expanding the system under test (SUT) to include multiple real collaborator objects or creating a violent mess of mocks.
  - pydry 3y ago
    It's not the right thing to do in a perfect world either. I've written quite a lot of software without any dependency injection with 0 unit tests that is fully covered by reliable, more than fast enough integration tests. Some people think that if I did dependency injection so I could unit test all of that same code, that code would be "better" in some indefinable way. In reality the code would just be longer (dependency injection increases SLOC). All other things being equal - longer = worse. DI-all-the-things is a dogma hangover from the late 1990s when integration testing and end to end testing tooling was all poor, computers were all slow and people tended to write more of their own algorithms (rather than importing them). DI is now something that should be approached on a case by case basis.
    - lamontcg 3y ago
      Yep, and I've seen code shipped that passed all its unit tests but the interface between them was subtly wrong so it violently failed to work at all after being shipped. Had to write functional tests to make sure it all worked together.