8 ms·

Agreed with the articles, mocks are very tied to implementation details. I almost always prefer state-based testing: https://martinfowler.com/articles/mocksAre

Agreed with the articles, mocks are very tied to implementation details. I almost always prefer state-based testing: https://martinfowler.com/articles/mocksArentStubs.html https://martinfowler.com/articles/mocksArentStubs.html As "observable state goes from a to b" is much closer to a business/functional requirement that will still/always be true, regardless of refactorings. Refactoring in codebases with state-based tests is a pleasure; in codebases with mock-based tests it's tedious, constantly updating tests when no semantic behavior was supposed to change. Also, mocking via module hacks like in the article (and in the JS world) is scary; modules are basically global variables so it's a very coarse grained slice point. Dependency injection is almost always better.

squirrelicus 7y ago
Even better than "observable state from a to b" is "a transform of value from a to b", no inspection and whitebox needed :D
- switchbak 7y ago
  Absolutely. The more functional we can make the code, typically the easier it is to test it based only on inputs and outputs. Much of what I do is try to come up with architectures that model the logic in this way, and are still legible to my co workers.
hinkley 7y ago
Dependency injection leads to elaborate, brittle fixtures that become more elaborate and arcane over time. And the sunk cost fallacy has people working for hours (and I’ve witnessed pairs spend two days, that’s over three man days!) trying to maintain them. I use mocks to arrange state A without going through all the business rules involved in creating state A. But you have to expose the states to do it, which can make for pretty descriptive code but is outside of some people’s experience and so they resist it. Earlier tests have already verified all routes to State A, the next batch of test now takes it as a given. This controls test explosion by utilizing transitivity. A->B B->C implies A->C. Tons of unit tests for A->B and B->C and then you only need a couple of functional tests (say, one negative and positive) test of A->C. You’re just checking the plumbing isn’t broken. Otherwise, you get Cartesian products. You end up with elaborate, (often custom), mocks that couple all of the tests together in hard to maintain ways. You end up with tests that accidentally test the mocks/fixtures instead of the code. Some of my current coworkers do this too. I don’t know where this pattern comes from. It’s often easier to replace their two page fixtures with two or three lines of mocks per test. It’s often almost the same amount of code, but so what, each test can be read. Each test can be run individually, and behavioral changes to the code due to urges affect mostly the tests you would expect. Those tests can be fixed, rewritten, or just removed as they disagree with the new requirements. The absolute worst is fixtures with asynchronous code. Those break constantly, and often invisibly. What’s the point of a fire alarm if the damned thing doesn’t work? It’s almost worse than nothing at all.
- stephen 7y ago
  > Dependency injection leads to elaborate, brittle fixtures I believe that can happen, but personally (N=1) haven't seen it. If anything, (well-done) DI is supposed to prevent that, b/c fixtures are isolated instead of depending on random global behavior. That said, I also don't like Spring/Guice/Dagger auto-wiring DI (somewhat guessing that is what caused your headaches), and instead just create an "AppContext" type with all of the applications "singletons" and pass that around. Granted, it's still global-ish, so maybe I'm cheating, but its nicer IMO than module hacks. > I use mocks to arrange state A without going through > all the business rules involved in creating state A I like tests being able to immediately jump to state A, but fwiw don't see why mocks would be needed to do so. I do agree re avoiding test explosion/transitivity, and a few functional happy/sad plumbing tests, but again seems orthogonal to state/mock. > You end up with elaborate, (often custom), mocks that > couple all of the tests together in hard to maintain ways Totally agreed. Another -1 for mocks. :-) I.e. with state-based you should be able to test "end-to-end" (with state-based / in-memory stubs for your input/output data stores/etc.) without any of "oh right, copy/paste these 5 lines of 'when method X return result Y' for ~2-5 some mocked-out calls). (And, to be pedantic, if you mean something other than 'when method X return result Y' for the term 'mock', then we're probably talking about different things.)
  - squiggleblaz 7y ago
    > That said, I also don't like Spring/Guice/Dagger auto-wiring DI (somewhat guessing that is what caused your headaches), and instead just create an "AppContext" type with all of the applications "singletons" and pass that around. > Granted, it's still global-ish, so maybe I'm cheating, but its nicer IMO than module hacks. I agree with you. I would like better support for (a) intersection types and (b) excluding global names. With intersection types, you avoid the globalish nature, since you can say "this method takes a PersistentUserStore", "this method takes a LoggerInterface", and "the parent method takes a PersistentUserStore&LoggerInterface" which can be passed to both. Lo and behold, the fact that it's globalish disappears since at any given point, you only Unfortunately, as with all things, PhpStorm does this exactly wrong - so that an intersection type is inspected as tho it were a union type and it gives me no particular benefit. Jetbrains seems less interested in helping me avoid bugs and focuses on helping me avoid reading the php.net manual. I appreciate it when I don't need to check out the manual, but I need to avoid bugs. As for excluding global names - if there were some way to make it illegal to say `new Foo`, except in the top of the code, for certain values of "Foo", it would be a serious help in keeping my tests of business logic free of implementation details. I'm not entirely sure how to restrict those "certain values of Foo" though.
aasasd 7y ago
> As "observable state goes from a to b" You might also notice that this is equivalent to a function that accepts a and returns b. Which lets one use equivalence partitioning to plan the tests: https://en.wikipedia.org/wiki/Equivalence_partitioning https://en.wikipedia.org/wiki/Equivalence_partitioning Except that with singletons, ‘context objects’ or similar quasi-global state, the effect might be non-isolated, and effects may be silently introduced that the calling code doesn't know about.
moron4hire 7y ago
A situation in which I've used mocking well: testing parsing of data coming over a serial port. I have had several situations to need to parse common data formats coming from hardware over a wire and mocking that data with recorded streams is super helpful for making sure your parser is working right.
- arantius 7y ago
  It sounds to me like you've got two things: some hardware code, and some data code. You're not testing the hardware code, you are testing the data code. If you split them into two places, you could get the same effect, with no mocks. And be more clear about what parts are tested and what aren't.
  - moron4hire 7y ago
    That is literally what I said I was doing.

squirrelicus 7y ago

Even better than "observable state from a to b" is "a transform of value from a to b", no inspection and whitebox needed :D

switchbak 7y ago
Absolutely. The more functional we can make the code, typically the easier it is to test it based only on inputs and outputs. Much of what I do is try to come up with architectures that model the logic in this way, and are still legible to my co workers.

hinkley 7y ago

Dependency injection leads to elaborate, brittle fixtures that become more elaborate and arcane over time. And the sunk cost fallacy has people working for hours (and I’ve witnessed pairs spend two days, that’s over three man days!) trying to maintain them. I use mocks to arrange state A without going through all the business rules involved in creating state A. But you have to expose the states to do it, which can make for pretty descriptive code but is outside of some people’s experience and so they resist it. Earlier tests have already verified all routes to State A, the next batch of test now takes it as a given. This controls test explosion by utilizing transitivity. A->B B->C implies A->C. Tons of unit tests for A->B and B->C and then you only need a couple of functional tests (say, one negative and positive) test of A->C. You’re just checking the plumbing isn’t broken. Otherwise, you get Cartesian products. You end up with elaborate, (often custom), mocks that couple all of the tests together in hard to maintain ways. You end up with tests that accidentally test the mocks/fixtures instead of the code. Some of my current coworkers do this too. I don’t know where this pattern comes from. It’s often easier to replace their two page fixtures with two or three lines of mocks per test. It’s often almost the same amount of code, but so what, each test can be read. Each test can be run individually, and behavioral changes to the code due to urges affect mostly the tests you would expect. Those tests can be fixed, rewritten, or just removed as they disagree with the new requirements. The absolute worst is fixtures with asynchronous code. Those break constantly, and often invisibly. What’s the point of a fire alarm if the damned thing doesn’t work? It’s almost worse than nothing at all.

stephen 7y ago
> Dependency injection leads to elaborate, brittle fixtures I believe that can happen, but personally (N=1) haven't seen it. If anything, (well-done) DI is supposed to prevent that, b/c fixtures are isolated instead of depending on random global behavior. That said, I also don't like Spring/Guice/Dagger auto-wiring DI (somewhat guessing that is what caused your headaches), and instead just create an "AppContext" type with all of the applications "singletons" and pass that around. Granted, it's still global-ish, so maybe I'm cheating, but its nicer IMO than module hacks. > I use mocks to arrange state A without going through > all the business rules involved in creating state A I like tests being able to immediately jump to state A, but fwiw don't see why mocks would be needed to do so. I do agree re avoiding test explosion/transitivity, and a few functional happy/sad plumbing tests, but again seems orthogonal to state/mock. > You end up with elaborate, (often custom), mocks that > couple all of the tests together in hard to maintain ways Totally agreed. Another -1 for mocks. :-) I.e. with state-based you should be able to test "end-to-end" (with state-based / in-memory stubs for your input/output data stores/etc.) without any of "oh right, copy/paste these 5 lines of 'when method X return result Y' for ~2-5 some mocked-out calls). (And, to be pedantic, if you mean something other than 'when method X return result Y' for the term 'mock', then we're probably talking about different things.)
- squiggleblaz 7y ago
  > That said, I also don't like Spring/Guice/Dagger auto-wiring DI (somewhat guessing that is what caused your headaches), and instead just create an "AppContext" type with all of the applications "singletons" and pass that around. > Granted, it's still global-ish, so maybe I'm cheating, but its nicer IMO than module hacks. I agree with you. I would like better support for (a) intersection types and (b) excluding global names. With intersection types, you avoid the globalish nature, since you can say "this method takes a PersistentUserStore", "this method takes a LoggerInterface", and "the parent method takes a PersistentUserStore&LoggerInterface" which can be passed to both. Lo and behold, the fact that it's globalish disappears since at any given point, you only Unfortunately, as with all things, PhpStorm does this exactly wrong - so that an intersection type is inspected as tho it were a union type and it gives me no particular benefit. Jetbrains seems less interested in helping me avoid bugs and focuses on helping me avoid reading the php.net manual. I appreciate it when I don't need to check out the manual, but I need to avoid bugs. As for excluding global names - if there were some way to make it illegal to say `new Foo`, except in the top of the code, for certain values of "Foo", it would be a serious help in keeping my tests of business logic free of implementation details. I'm not entirely sure how to restrict those "certain values of Foo" though.

aasasd 7y ago

> As "observable state goes from a to b" You might also notice that this is equivalent to a function that accepts a and returns b. Which lets one use equivalence partitioning to plan the tests: https://en.wikipedia.org/wiki/Equivalence_partitioning https://en.wikipedia.org/wiki/Equivalence_partitioning Except that with singletons, ‘context objects’ or similar quasi-global state, the effect might be non-isolated, and effects may be silently introduced that the calling code doesn't know about.

moron4hire 7y ago

A situation in which I've used mocking well: testing parsing of data coming over a serial port. I have had several situations to need to parse common data formats coming from hardware over a wire and mocking that data with recorded streams is super helpful for making sure your parser is working right.

arantius 7y ago
It sounds to me like you've got two things: some hardware code, and some data code. You're not testing the hardware code, you are testing the data code. If you split them into two places, you could get the same effect, with no mocks. And be more clear about what parts are tested and what aren't.
- moron4hire 7y ago
  That is literally what I said I was doing.