4 ms·

The classic test rookie mindset is to test the functionality of the whole system, because that's what really matters. But in reality, unit testing every single

The classic test rookie mindset is to test the functionality of the whole system, because that's what really matters. But in reality, unit testing every single function and method is where the vast majority of the benefit lies. Details really matter. It took me some time to learn this, even after being told. It's the same for most people. This little post will probably convince no one. But maybe remember it when you finally get there yourself :)

Izkata 3y ago
> every single function and method Very much no, that's the bad kind of unit test that locks your code into a specific structure and makes it a pain to update because you also have to change all the related tests even if the actual interface used by the rest of the codebase didn't change. I would call this the rookie mistake of someone new to unit tests. You want to encapsulate your code with some sort of interface that matches the problem space, then test to that interface. How its internals are broken down don't matter: it could be one big function, it could be a dozen functions, it could be a class, it as long as the inputs and outputs match what the test is looking for you can refactor and add/remove features without having to spend extra time changing the tests. Makes it much less of a pain to work with in general. One way of looking at it I've used before with coworkers: For this new feature you're writing, imagine a library for it already exists. What is the simplest and most straightforward way to use that library? That's your interface, the thing you expose to the world and what you run your tests against. This is what unit testing originally meant: semantic units, not code units. It's like app Hungarian notation vs system Hungarian notation, the original idea got overtaken by people who didn't understand the idea and only mimicked the surface level appearance.
troupo 3y ago
> But in reality, unit testing every single function and method is where the vast majority of the benefit lies. Details really matter. To me, this is actual rookie mentality. You end up testing the same thing multiple times over different lines of code, mocking and providing various sets of testing data... When you could just test specified and/or observable behaviour of your system, and achieve the exactly same result with fewer tests.
- BurningFrog 3y ago
  Well, I certainly don't end up doing that. There are many ways of doing things, and I guess we do unit tests differently. > When you could just test specified and/or observable behaviour of your system, and achieve the exactly same result with fewer tests. In my experience, it turns out to be very difficult to test a specific behaviour 5-10 layers deep from an external interface. Also, when one of those intermediate layers changes, you tend to have to rewrite many of those tests.
  - troupo 3y ago
    > Well, I certainly don't end up doing that. How else are you "unit testing every single function and method"? > it turns out to be very difficult to test a specific behaviour 5-10 layers deep from an external interface. If you don't test it, how do you know your system works for that specific behaviour? Just because you've tested every single function and method in isolation doesn't mean they actually work with each other, or produce the responses the way you need them to. > Also, when one of those intermediate layers changes, you tend to have to rewrite many of those tests. As you should. Otherwise how do you know that your system still works?
    - BurningFrog 3y ago
      > How else are you "unit testing every single function and method"? There are many ways to write unit tests, as well as writing code that is easy to test. I don't know how my ways differs from yours, but I don't have much of the problems you mention. > Otherwise how do you know that your system still works? We do have some integration test, of course. But it's a small part of the total test suite.
      - troupo 3y ago
        > I don't know how my ways differs from yours, but I don't have much of the problems you mention. You haven't answered "How else are you "unit testing every single function and method"?" Given a medium-sized project and at least a passing and a failing test case for each function and method, you end up if not with hundreds, but with dozens of tests largely doing the same thing. > We do have some integration test, of course. But it's a small part of the total test suite. So what does your test suite contain? Lots of unit tests for each method and function. What else?
BeetleB 3y ago
I'll mirror a sibling comment. For me your mentality is the rookie mentality. I too once believed in strict unit tests, as well as a strict differentiation between them and other kinds of tests (end to end, integration, etc). Then I joined a project where they were just starting to add tests to an existing project, and the lead developer was adamant on the following philosophy: "Virtually all tests will run the full program, and we'll mock out whatever slows the tests down (e.g. network access, etc)". I whined but I had to comply. After a year of this, I give him credit for changing my mindset. The majority of bugs we found simply would not have been found with unit tests. On the flip side, almost none of the unit test failures were false alarms (function signature changed, etc). Since then, I've dropped categorizing tests as unit tests vs other types of tests. Examine the project at hand, and write automated tests - preferably fast ones. Focus on testing features, and not functions.

Izkata 3y ago

> every single function and method Very much no, that's the bad kind of unit test that locks your code into a specific structure and makes it a pain to update because you also have to change all the related tests even if the actual interface used by the rest of the codebase didn't change. I would call this the rookie mistake of someone new to unit tests. You want to encapsulate your code with some sort of interface that matches the problem space, then test to that interface. How its internals are broken down don't matter: it could be one big function, it could be a dozen functions, it could be a class, it as long as the inputs and outputs match what the test is looking for you can refactor and add/remove features without having to spend extra time changing the tests. Makes it much less of a pain to work with in general. One way of looking at it I've used before with coworkers: For this new feature you're writing, imagine a library for it already exists. What is the simplest and most straightforward way to use that library? That's your interface, the thing you expose to the world and what you run your tests against. This is what unit testing originally meant: semantic units, not code units. It's like app Hungarian notation vs system Hungarian notation, the original idea got overtaken by people who didn't understand the idea and only mimicked the surface level appearance.

troupo 3y ago

> But in reality, unit testing every single function and method is where the vast majority of the benefit lies. Details really matter. To me, this is actual rookie mentality. You end up testing the same thing multiple times over different lines of code, mocking and providing various sets of testing data... When you could just test specified and/or observable behaviour of your system, and achieve the exactly same result with fewer tests.

BurningFrog 3y ago
Well, I certainly don't end up doing that. There are many ways of doing things, and I guess we do unit tests differently. > When you could just test specified and/or observable behaviour of your system, and achieve the exactly same result with fewer tests. In my experience, it turns out to be very difficult to test a specific behaviour 5-10 layers deep from an external interface. Also, when one of those intermediate layers changes, you tend to have to rewrite many of those tests.
- troupo 3y ago
  > Well, I certainly don't end up doing that. How else are you "unit testing every single function and method"? > it turns out to be very difficult to test a specific behaviour 5-10 layers deep from an external interface. If you don't test it, how do you know your system works for that specific behaviour? Just because you've tested every single function and method in isolation doesn't mean they actually work with each other, or produce the responses the way you need them to. > Also, when one of those intermediate layers changes, you tend to have to rewrite many of those tests. As you should. Otherwise how do you know that your system still works?
  - BurningFrog 3y ago
    > How else are you "unit testing every single function and method"? There are many ways to write unit tests, as well as writing code that is easy to test. I don't know how my ways differs from yours, but I don't have much of the problems you mention. > Otherwise how do you know that your system still works? We do have some integration test, of course. But it's a small part of the total test suite.
    - troupo 3y ago
      > I don't know how my ways differs from yours, but I don't have much of the problems you mention. You haven't answered "How else are you "unit testing every single function and method"?" Given a medium-sized project and at least a passing and a failing test case for each function and method, you end up if not with hundreds, but with dozens of tests largely doing the same thing. > We do have some integration test, of course. But it's a small part of the total test suite. So what does your test suite contain? Lots of unit tests for each method and function. What else?

BeetleB 3y ago

I'll mirror a sibling comment. For me your mentality is the rookie mentality. I too once believed in strict unit tests, as well as a strict differentiation between them and other kinds of tests (end to end, integration, etc). Then I joined a project where they were just starting to add tests to an existing project, and the lead developer was adamant on the following philosophy: "Virtually all tests will run the full program, and we'll mock out whatever slows the tests down (e.g. network access, etc)". I whined but I had to comply. After a year of this, I give him credit for changing my mindset. The majority of bugs we found simply would not have been found with unit tests. On the flip side, almost none of the unit test failures were false alarms (function signature changed, etc). Since then, I've dropped categorizing tests as unit tests vs other types of tests. Examine the project at hand, and write automated tests - preferably fast ones. Focus on testing features, and not functions.