4 ms·

> And they can be more brittle than your actual code. The brittlest tests I've ever seen have been mock-heavy unit tests. Some of them amount to little more t

> And they can be more brittle than your actual code. The brittlest tests I've ever seen have been mock-heavy unit tests. Some of them amount to little more than file-change detectors. Those tests can fail massively even when everything still actually works. They cry wolf so often the team ends up trained to unthinkingly change them so they "go green." > But I think I get what he's getting at in terms of – ideally – wanting to minimize those kinds of tests by transforming previously unreliable parts of the system into something we can use "as a given" (like integer arithmetic). A noble goal, but practically not a lot different from saying we should aim for formally verified software. After all, it's what we have for computer arithmetic example: http://www.cl.cam.ac.uk/~jrh13/slides/jnao-02jun06/slides.pdf http://www.cl.cam.ac.uk/~jrh13/slides/jnao-02jun06/slides.pd... (2006): > Intel uses formal verification quite extensively, e.g. ... verification of...floating-point unit[s]...

Blazespinnaker 9y ago
My experience with unit tests is that they force a developer to think through their logic. So the tests themselves,once passing don’t help that much (though a little), but the act of writing them did have high ROI. The coverage metrics also help management understand what is baked.
bonesss 9y ago
> The brittlest tests I've ever seen have been mock-heavy unit tests Not to knock mocking in general, but the need to replace multiple things at once to test one thing is just a big design smell, and I agree: their usefulness tends to be very limited in terms of logic exercised. Call me a fuddy-dud, but I'm generally working with dependancy injection when doing OOP so stubbing out app layers for test contexts feels kinda natural, and I prefer to make my test mocks the old fashioned way: manually using inheritance. It keeps the pain high if I'm doing something wrong, keeps the tests focused, and means I'm hammering the objects design from the POV of an inheritor/reuser from the get-go. I've noticed that the additions necessary to make a manual mock tend to be a good predictor of future changes required in the code-base, while framework based mocks tend to spend a lot of time re-declaring things we already know.

Blazespinnaker 9y ago

My experience with unit tests is that they force a developer to think through their logic. So the tests themselves,once passing don’t help that much (though a little), but the act of writing them did have high ROI. The coverage metrics also help management understand what is baked.

bonesss 9y ago

> The brittlest tests I've ever seen have been mock-heavy unit tests Not to knock mocking in general, but the need to replace multiple things at once to test one thing is just a big design smell, and I agree: their usefulness tends to be very limited in terms of logic exercised. Call me a fuddy-dud, but I'm generally working with dependancy injection when doing OOP so stubbing out app layers for test contexts feels kinda natural, and I prefer to make my test mocks the old fashioned way: manually using inheritance. It keeps the pain high if I'm doing something wrong, keeps the tests focused, and means I'm hammering the objects design from the POV of an inheritor/reuser from the get-go. I've noticed that the additions necessary to make a manual mock tend to be a good predictor of future changes required in the code-base, while framework based mocks tend to spend a lot of time re-declaring things we already know.