4 ms·

I keep running into the exact inverse: people who seem to believe they only need unit tests (i.e. no functional/integration tests). It's insane. If I had to pic

I keep running into the exact inverse: people who seem to believe they only need unit tests (i.e. no functional/integration tests). It's insane. If I had to pick one, I'd go with the more "live" multi-component tests and ditch the unit tests. Unit tests still have value, as a lightweight way to catch dumb errors in future refactors. That's fantastic. Love it. As soon as they cease being lightweight (and the norm seems to be more effort spent creating mocks/stubs than the actual tests) then they fail even in that purpose and just become another impediment to useful work. I'm all for unit tests as part of a full testing meal, but a whole plate full of nothing but unit tests makes me want to throw up.

mattferderer 7y ago
I agree. As always, "It Depends". Use the best tool (type of test) to document & test the business or technical rules being requested. Tests help teams (and future-self of the individual who wrote the code) understand why certain code was implemented & what purpose it is meant to solve. As the business changes the rules they want implemented, having well written out tests seem to help significantly with handling those rules. It also helps when you want to refactor for the sake of code understanding & readability. It is often the case where both developers & those requesting the changes forget why some of the rules were put in place to begin with.
kraftman 7y ago
I think mocks in unit tests are a code smell, that indicates that the business logic has not been correctly isolated from the rest of the code, or that the code shouldn't be unit tested at all, and should instead be component/integration tested.
- closeparen 7y ago
  Most architectural guidance encourages separation of concerns into layers (handler, controller, gateway, etc). Each layer consumes the next. You will be hard pressed to unit test one layer without injecting a mock of the layer below. When the layer is essentially pro forma and has no logic of its own, this is excruciating, but the line coverage number rules all.
  - kraftman 7y ago
    Yeah, I think this separation of concern misses a step. Let's say you have controller -> Business logic layer -> Data access layer. The controller takes in some input, validates it, passes it to the business logic layer. The business logic layer retrieves some data from the DAL, does some logic, maybe writes data back to the DAL, and passes the data back up to the controller. The problem is that the business logic layer is usually doing two things at the same time, in the same function: the 'plumbing' (calling other things and passing data round), and the actual business logic. When you try and define a boundary around the code, it gets messy, because what you want to test, the business logic, is mixed in with the plumbing so you have to use either dependency injection or mocking to force a boundary. This makes your tests more brittle as changes in things you don't want to test (your dependencies) affect your tests. If you pull out the business logic into its own modules, and instead have a function that orchestrates the business logic (i.e. tell the database what data you want, tell the business logic functions to perform an operation on the data), you end up with a nice clean boundary around your business logic, which can be unit tested easily without mocks. You then test the 'plumbing' part by using a component test that checks that the inputs and outputs of the component as a whole (business logic + plumbing + data access) are correct.
    - closeparen 7y ago
      I think what you’re describing is what we would call a controller, and management / senior ICs would still demand 90% line coverage by unit tests.
toasterlovin 7y ago
The way I've started to think about testing is like this: you should test your interfaces, because the interface is what matters. If your code is a web app, then you should be testing the user interfaces and workflows that users interact with throughout the app. If your code is a library, then you should be testing the classes and functions that users will be interacting with when they use your library. In my opinion, everything else is an implementation detail. But, I think this distinction gets blurrier the larger your organization. While you may have intended some class in your web app as an implementation detail, the larger your organization is, the more likely it is that somebody will start using that class for some other purpose. And thus and implementation detail becomes an interface that other people depend on.
dnautics 7y ago
I'd say that when I don't know where to start (because there are so many ways one could start building something) outside-in TDD is very useful. (About 40-50% of my code is outside-in TDD).
paranoidrobot 7y ago
> As soon as they cease being lightweight (and the norm seems to be more effort spent creating mocks/stubs than the actual tests) then they fail even in that purpose and just become another impediment to useful work. Agreed. I think that in many cases it is a signal that your code is too complex, and may be violating the Single-Responsiblity principle.
closeparen 7y ago
Now mix in VP-level accountability for unit test coverage KPIs, coupled with piss poor integration test infrastructure, and you have the uphill battle I fight daily.

mattferderer 7y ago

I agree. As always, "It Depends". Use the best tool (type of test) to document & test the business or technical rules being requested. Tests help teams (and future-self of the individual who wrote the code) understand why certain code was implemented & what purpose it is meant to solve. As the business changes the rules they want implemented, having well written out tests seem to help significantly with handling those rules. It also helps when you want to refactor for the sake of code understanding & readability. It is often the case where both developers & those requesting the changes forget why some of the rules were put in place to begin with.

kraftman 7y ago

I think mocks in unit tests are a code smell, that indicates that the business logic has not been correctly isolated from the rest of the code, or that the code shouldn't be unit tested at all, and should instead be component/integration tested.

closeparen 7y ago
Most architectural guidance encourages separation of concerns into layers (handler, controller, gateway, etc). Each layer consumes the next. You will be hard pressed to unit test one layer without injecting a mock of the layer below. When the layer is essentially pro forma and has no logic of its own, this is excruciating, but the line coverage number rules all.
- kraftman 7y ago
  Yeah, I think this separation of concern misses a step. Let's say you have controller -> Business logic layer -> Data access layer. The controller takes in some input, validates it, passes it to the business logic layer. The business logic layer retrieves some data from the DAL, does some logic, maybe writes data back to the DAL, and passes the data back up to the controller. The problem is that the business logic layer is usually doing two things at the same time, in the same function: the 'plumbing' (calling other things and passing data round), and the actual business logic. When you try and define a boundary around the code, it gets messy, because what you want to test, the business logic, is mixed in with the plumbing so you have to use either dependency injection or mocking to force a boundary. This makes your tests more brittle as changes in things you don't want to test (your dependencies) affect your tests. If you pull out the business logic into its own modules, and instead have a function that orchestrates the business logic (i.e. tell the database what data you want, tell the business logic functions to perform an operation on the data), you end up with a nice clean boundary around your business logic, which can be unit tested easily without mocks. You then test the 'plumbing' part by using a component test that checks that the inputs and outputs of the component as a whole (business logic + plumbing + data access) are correct.
  - closeparen 7y ago
    I think what you’re describing is what we would call a controller, and management / senior ICs would still demand 90% line coverage by unit tests.

toasterlovin 7y ago

The way I've started to think about testing is like this: you should test your interfaces, because the interface is what matters. If your code is a web app, then you should be testing the user interfaces and workflows that users interact with throughout the app. If your code is a library, then you should be testing the classes and functions that users will be interacting with when they use your library. In my opinion, everything else is an implementation detail. But, I think this distinction gets blurrier the larger your organization. While you may have intended some class in your web app as an implementation detail, the larger your organization is, the more likely it is that somebody will start using that class for some other purpose. And thus and implementation detail becomes an interface that other people depend on.

dnautics 7y ago

I'd say that when I don't know where to start (because there are so many ways one could start building something) outside-in TDD is very useful. (About 40-50% of my code is outside-in TDD).

paranoidrobot 7y ago

> As soon as they cease being lightweight (and the norm seems to be more effort spent creating mocks/stubs than the actual tests) then they fail even in that purpose and just become another impediment to useful work. Agreed. I think that in many cases it is a signal that your code is too complex, and may be violating the Single-Responsiblity principle.

closeparen 7y ago

Now mix in VP-level accountability for unit test coverage KPIs, coupled with piss poor integration test infrastructure, and you have the uphill battle I fight daily.