4 ms·

“Don’t mock what you don't own” in 5 minutes (2022)

42 points by JNRowe 1y ago

gnabgib 1y ago
(2022)
fellatio 1y ago
Polite way here is similar to inversion of control. It's a good idea but you need a real thing test somewhere but that can be running the program in its entirety, or using the facade end to end against the real interface for a spot check.
mpweiher 1y ago
Why I don't mock (2014): https://news.ycombinator.com/item?id=7809402 https://news.ycombinator.com/item?id=7809402 "Well, it's impolite, isn't it?"
AugustoCAS 1y ago
Something that I find amusing in the Java community is that a good number of senior developers, with anything from 5-20 years of experience, who do 'tdd' have never heard of the concept of test doubles and religiously think that a class must be tested in complete isolation mocking everything else. The saddest one I saw was a team trying to do functional programming (with Spring). The tech lead was a bit flummoxed when I asked why mocks are not used in functional languages and continued to think that 'mocking functions' is the correct way to do TDD.
- esailija 1y ago
  Tests knowing about implementation details and testing the implementation details (which is the case 99.999% of the time if you use mocks) is more common than not. Even when the main value of automated testing is being able to change those very implementation details, that you now cannot do. A whole bunch of work spent for no benefit or negative benefit is pretty common.
- 0x445442 1y ago
  I use Java and Spring extensively. If I am in a lead role and have say over the code base I won't allow mocking frameworks to be used in tests. If you want a good way to shine a light on poorly structured code, disallow mocking frameworks. Java added the Funcional interface in v8 making it quite easy to code to an interface and yet in Spring everyone's go to is just to slap a @Component on a concrete class.
eximius 1y ago
And better yet, follow the tangential advice and use high fidelity fakes (I usually don't quite go all the way to verified fakes/contract tests, but they're a good idea if you have the time). Real object where practical, fakes otherwise, mocks only as required for exceptional circumstances. Works great in interconnected monorepos where you can provide high fidelity fakes of your services for other integrating teams to use in their tests. Often our service fakes are literally just the real service wrapped with an injected fake DB/store.
- sparsely 1y ago
  Fakes are better than mocks if only for the 100x superior debugging experience.
lmm 1y ago
Ok but the test is now not covering the parts that are actually going to break. And if you're only testing the parts above the nice business-friendly interface, you probably don't need mocks at all.
strken 1y ago
I'm confused. If I'm allowed to mock what I don't own, but only if it has a good API that's well-adapted to my use case, then "don't mock what you don't own" must have nothing to do with ownership. It sounds like it's more about wrapping awkward APIs instead of calling them directly. Yes, this is good advice! But it only tangentially has anything to do with mocking.
molf 1y ago
I’m not sure this is good advice. I prefer to test as much of the stack as possible. The most common mistake I see these days is people testing too much in isolation, which leads to a false sense of safety. If you care about being alerted when your dependencies break, writing only the kind of tests described in the article is risky. You’ve removed those dependencies from your test suite. If a minor library update changes `.json()` to `.parse(format="json")`, and you assumed they followed semver but they didn’t: you’ll find out after deployment. Ah, but you use static typing? Great! That’ll catch some API changes. But if you discover an API changed without warning (because you thought nobody would ever do that) you’re on your own again. I suggest using a nice HTTP recording/replay library for your tests so you can adapt easily (without making live HTTP calls in your tests, which would be way too flaky, even if feasible). I stopped worrying long ago about what is or isn’t “real” unit testing. I test as much of the software stack as I can. If a test covers too many abstraction layers at once, I split it into lower- and higher-level cases. These days, I prefer fewer “poorly” factored tests that cover many real layers of the code over countless razor-thin unit tests that only check whether a loop was implemented correctly. While risking that the whole system doesn’t work together. Because by the time you get to write your system/integration/whatever tests, you’re already exhausted from writing and refactoring all those near-pointless micro-tests.
- strken 1y ago
  My take on unit tests is that they happen "automatically" for the things that actually need unit testing. Consider a load shedder, a parser, or an implementation of an algorithm. Most devs will naturally write a unit test for anything with complicated logic just because it's easier to test complicated things in isolation.
- hynek 1y ago
  > I’m not sure this is good advice. I prefer to test as much of the stack as possible. The most common mistake I see these days is people testing too much in isolation, which leads to a false sense of safety. You make it sounds as if the article would argue for test isolation which it emphatically doesn't. It in fact even links out to the Mock Hell talk. Every mock makes the test suite less meaningful and the question the article is trying to answer is how to minimize the damage the mocks do to your software if you actually need them.
hamstergene 1y ago
It would be great if every complex or volatile third-party library came with its own mock and an abstract interface to which they both conform. Including pieces of language’s standard library itself. It makes so much more sense this way: - users don’t have to duplicate effort of writing more or less same mock/stub of the same library - mock is forced to be up to date, because the library’s maintainers probably pay more attention to their release notes than you do - no one knows the best choice of mocking approach/DSL than the library’s authors - the adoption of inversion of control / dependency injection and comprehensive testing could be so much wider if writing a test didn’t require first solving an extra problem if how to mock something with complicated API and if it worth solving in the first place
- rileymichael 1y ago
  this is what we do internally at $dayJob. the gradle java-test-fixtures plugin (https://docs.gradle.org/current/userguide/java_testing.html#sec:java_test_fixtures https://docs.gradle.org/current/userguide/java_testing.html#...) creates a dedicated source set and wires up artifact publishing with a `test-fixtures` classifier. consuming a libraries fixtures is as simple as: // pull in some-library implementation("some-library-coordinates") // use its provided fixtures in tests testImplementation(testFixtures("some-library-coordinates")) wish it was more common in the ecosystem, but it seems most folks aren't aware of it.
ramchip 1y ago
This is very similar to Jose Valim's "Mocks and explicit contracts" from 2015, down to using a Twitter client as example. https://dashbit.co/blog/mocks-and-explicit-contracts https://dashbit.co/blog/mocks-and-explicit-contracts
hmmokidk 1y ago
I love Stripe for creating a CLI testing thing that even allows you to test webhooks.

42 points by JNRowe 1y ago

(2022)

Polite way here is similar to inversion of control. It's a good idea but you need a real thing test somewhere but that can be running the program in its entirety, or using the facade end to end against the real interface for a spot check.

mpweiher 1y ago

Why I don't mock (2014): https://news.ycombinator.com/item?id=7809402 https://news.ycombinator.com/item?id=7809402 "Well, it's impolite, isn't it?"

AugustoCAS 1y ago

Something that I find amusing in the Java community is that a good number of senior developers, with anything from 5-20 years of experience, who do 'tdd' have never heard of the concept of test doubles and religiously think that a class must be tested in complete isolation mocking everything else. The saddest one I saw was a team trying to do functional programming (with Spring). The tech lead was a bit flummoxed when I asked why mocks are not used in functional languages and continued to think that 'mocking functions' is the correct way to do TDD.

esailija 1y ago
Tests knowing about implementation details and testing the implementation details (which is the case 99.999% of the time if you use mocks) is more common than not. Even when the main value of automated testing is being able to change those very implementation details, that you now cannot do. A whole bunch of work spent for no benefit or negative benefit is pretty common.
0x445442 1y ago
I use Java and Spring extensively. If I am in a lead role and have say over the code base I won't allow mocking frameworks to be used in tests. If you want a good way to shine a light on poorly structured code, disallow mocking frameworks. Java added the Funcional interface in v8 making it quite easy to code to an interface and yet in Spring everyone's go to is just to slap a @Component on a concrete class.

eximius 1y ago

And better yet, follow the tangential advice and use high fidelity fakes (I usually don't quite go all the way to verified fakes/contract tests, but they're a good idea if you have the time). Real object where practical, fakes otherwise, mocks only as required for exceptional circumstances. Works great in interconnected monorepos where you can provide high fidelity fakes of your services for other integrating teams to use in their tests. Often our service fakes are literally just the real service wrapped with an injected fake DB/store.

sparsely 1y ago
Fakes are better than mocks if only for the 100x superior debugging experience.

lmm 1y ago

Ok but the test is now not covering the parts that are actually going to break. And if you're only testing the parts above the nice business-friendly interface, you probably don't need mocks at all.

strken 1y ago

I'm confused. If I'm allowed to mock what I don't own, but only if it has a good API that's well-adapted to my use case, then "don't mock what you don't own" must have nothing to do with ownership. It sounds like it's more about wrapping awkward APIs instead of calling them directly. Yes, this is good advice! But it only tangentially has anything to do with mocking.

molf 1y ago

I’m not sure this is good advice. I prefer to test as much of the stack as possible. The most common mistake I see these days is people testing too much in isolation, which leads to a false sense of safety. If you care about being alerted when your dependencies break, writing only the kind of tests described in the article is risky. You’ve removed those dependencies from your test suite. If a minor library update changes `.json()` to `.parse(format="json")`, and you assumed they followed semver but they didn’t: you’ll find out after deployment. Ah, but you use static typing? Great! That’ll catch some API changes. But if you discover an API changed without warning (because you thought nobody would ever do that) you’re on your own again. I suggest using a nice HTTP recording/replay library for your tests so you can adapt easily (without making live HTTP calls in your tests, which would be way too flaky, even if feasible). I stopped worrying long ago about what is or isn’t “real” unit testing. I test as much of the software stack as I can. If a test covers too many abstraction layers at once, I split it into lower- and higher-level cases. These days, I prefer fewer “poorly” factored tests that cover many real layers of the code over countless razor-thin unit tests that only check whether a loop was implemented correctly. While risking that the whole system doesn’t work together. Because by the time you get to write your system/integration/whatever tests, you’re already exhausted from writing and refactoring all those near-pointless micro-tests.

strken 1y ago
My take on unit tests is that they happen "automatically" for the things that actually need unit testing. Consider a load shedder, a parser, or an implementation of an algorithm. Most devs will naturally write a unit test for anything with complicated logic just because it's easier to test complicated things in isolation.
hynek 1y ago
> I’m not sure this is good advice. I prefer to test as much of the stack as possible. The most common mistake I see these days is people testing too much in isolation, which leads to a false sense of safety. You make it sounds as if the article would argue for test isolation which it emphatically doesn't. It in fact even links out to the Mock Hell talk. Every mock makes the test suite less meaningful and the question the article is trying to answer is how to minimize the damage the mocks do to your software if you actually need them.

hamstergene 1y ago

It would be great if every complex or volatile third-party library came with its own mock and an abstract interface to which they both conform. Including pieces of language’s standard library itself. It makes so much more sense this way: - users don’t have to duplicate effort of writing more or less same mock/stub of the same library - mock is forced to be up to date, because the library’s maintainers probably pay more attention to their release notes than you do - no one knows the best choice of mocking approach/DSL than the library’s authors - the adoption of inversion of control / dependency injection and comprehensive testing could be so much wider if writing a test didn’t require first solving an extra problem if how to mock something with complicated API and if it worth solving in the first place

rileymichael 1y ago
this is what we do internally at $dayJob. the gradle java-test-fixtures plugin (https://docs.gradle.org/current/userguide/java_testing.html#sec:java_test_fixtures https://docs.gradle.org/current/userguide/java_testing.html#...) creates a dedicated source set and wires up artifact publishing with a `test-fixtures` classifier. consuming a libraries fixtures is as simple as: // pull in some-library implementation("some-library-coordinates") // use its provided fixtures in tests testImplementation(testFixtures("some-library-coordinates")) wish it was more common in the ecosystem, but it seems most folks aren't aware of it.

ramchip 1y ago

This is very similar to Jose Valim's "Mocks and explicit contracts" from 2015, down to using a Twitter client as example. https://dashbit.co/blog/mocks-and-explicit-contracts https://dashbit.co/blog/mocks-and-explicit-contracts

hmmokidk 1y ago

I love Stripe for creating a CLI testing thing that even allows you to test webhooks.