4 ms·

I’ve read this article twice now and it feels like confused word soup with no clear statements of the intended differences, specifically none of the statements

by mlthoughts2018 6y ago

I’ve read this article twice now and it feels like confused word soup with no clear statements of the intended differences, specifically none of the statements about the order of state vs behavior validation are intelligible to me. I have read them multiple times and just feel (a) frustrated and (b) a mock and a stub are really the same idea with nothing more than occasional conventional differences that amount to semantic hair-splitting with no material consequence or need for articles to be written about the platonic difference.

kyrra 6y ago
My take... Stubs just return preprogrammed data. Mocks do the above, but can provide their own assertions about call order or number of times called.
- MaxBarraclough 6y ago
  That seems consistent with the article. Fowler doesn't explicit state that mocks are a kind of stub, but it seems to follow from his definitions.
Benjammer 6y ago
What do you mean by “order” in the sentence “... the order of state vs. behavior validation ...”? State validation means checking a stub dependency of the SUT to make sure it’s in the right state. Behavior validation means you check a mock dependency to make sure the right functions were called the right number of times, and in the right order. A stub is just a thing you can substitute for the dependency that knows how to tell the test it’s state, and a mock is a substitute for the dependency that knows how to tell it’s behavior, like which functions are called and how many times. In terms of them being the same thing, they are both types of Test Doubles, so they both fall into a higher level category of “a substitute for a dependency of the system under test (SUT)”. The difference is how they’re used.
- hyko 6y ago
  I guess in the example given you could also validate behaviour by using a Spy Warehouse (a capturing stub). The theoretical distinction with Mocks is that they contain their own assertions. The practical distinction (when using mocking frameworks) is often that they allow testing a SUT that depends on concrete classes, without subclassing them.
danidiaz 6y ago
The same "final state" of a fake can potentially be achieved by many different patterns of interaction between the system under test and the fake. We don't really care about those patterns of interaction as long as they end up doing the right thing: allowing the SUT to affect his "outside world" as intended. Making assertions about the exact pattern of interaction (that is: using mocks) is too high granularity, and will make the tests break on valid refactorings. Because we don't care about the pattern itself—it's just an implementation detail—but about what it achieves.
mnsc 6y ago
Worth noting is that this article is almost 15 years old so the "need" for this article probably was a bit different back then. That said, much of this still is valuable to think about and this article, to me, is not just word soup but I'll give you that it's not very focused. And I get confused since "mockist tdd" isn't really talked about explicility anymore. But I would argue the concept is very much alive as a natural part of behavior-driven development which encourages outside-in development with a focus on writing acceptance test first. So my question is if Fowlers worry from the past, that using mocks and asserting behaviour is focusing too much on the implementation. But like Fowler I haven't done BDD "for real" so I haven't seen how it plays out in the long term.
spyrefused 6y ago
I found quite useful an example I read in the book "The Art of Unit Testing" by Roy Osherove, it was something like this (I don't have it at hand, I repeat it from memory): Suppose you have to test the irrigation system of a tree in your garden, you want to test how many times and how much water is used, you have two ways to do it: 1) State-based: run the system for a few hours, during which it should water the tree several times. At the end of that time, you check the state of the watered tree: Is the soil moist enough, is the tree well, are the leaves green, etc. (Obviously this is just an example as this test would be complex in real life, but it is to get the idea across). 2) Interaction: We place a flow meter, a device that registers how much water is flowing through the device and at what times. At the end of the day, check that the device has been called the correct number of times, with the correct amount of water each time, and we don't worry about checking the status of the tree.
finnthehuman 6y ago
>I’ve read this article twice now and it feels like confused word soup Yeah, that'll happen with a lot of the testing "literature" from the 2005ish to 2015ish era. It was a popular topic at the time, and one I was searching for answers on too. But I was working at a company that was ahead of the curve on testing and I got so frustrated at publicly available resources on testing. During that time and still to this day, when I read about testing online I kinda feel like I'm drunk. I have an idea of what's going on, things sorta make sense but not really. I'm only understanding bits and pieces. There are unnecessary and over-complicated details on some things, suspiciously light on meaty topics. There was so much drivel written about testing back then that I still get angry remembering that people would say with a straight face "you don't need documentation, tests are your documentation."
arwhatever 6y ago
Might I recommend this overview? https://enterprisecraftsmanship.com/posts/when-to-mock/ https://enterprisecraftsmanship.com/posts/when-to-mock/ I more understand the Martin Fowler article quite well, but only after struggling with it and revisiting it (sporadically) over several years, and I really think this by Mr, Khorikov gives a much clearer, more straightforward overview.

kyrra 6y ago

My take... Stubs just return preprogrammed data. Mocks do the above, but can provide their own assertions about call order or number of times called.

MaxBarraclough 6y ago
That seems consistent with the article. Fowler doesn't explicit state that mocks are a kind of stub, but it seems to follow from his definitions.

Benjammer 6y ago

What do you mean by “order” in the sentence “... the order of state vs. behavior validation ...”? State validation means checking a stub dependency of the SUT to make sure it’s in the right state. Behavior validation means you check a mock dependency to make sure the right functions were called the right number of times, and in the right order. A stub is just a thing you can substitute for the dependency that knows how to tell the test it’s state, and a mock is a substitute for the dependency that knows how to tell it’s behavior, like which functions are called and how many times. In terms of them being the same thing, they are both types of Test Doubles, so they both fall into a higher level category of “a substitute for a dependency of the system under test (SUT)”. The difference is how they’re used.

hyko 6y ago
I guess in the example given you could also validate behaviour by using a Spy Warehouse (a capturing stub). The theoretical distinction with Mocks is that they contain their own assertions. The practical distinction (when using mocking frameworks) is often that they allow testing a SUT that depends on concrete classes, without subclassing them.

danidiaz 6y ago

The same "final state" of a fake can potentially be achieved by many different patterns of interaction between the system under test and the fake. We don't really care about those patterns of interaction as long as they end up doing the right thing: allowing the SUT to affect his "outside world" as intended. Making assertions about the exact pattern of interaction (that is: using mocks) is too high granularity, and will make the tests break on valid refactorings. Because we don't care about the pattern itself—it's just an implementation detail—but about what it achieves.

mnsc 6y ago

Worth noting is that this article is almost 15 years old so the "need" for this article probably was a bit different back then. That said, much of this still is valuable to think about and this article, to me, is not just word soup but I'll give you that it's not very focused. And I get confused since "mockist tdd" isn't really talked about explicility anymore. But I would argue the concept is very much alive as a natural part of behavior-driven development which encourages outside-in development with a focus on writing acceptance test first. So my question is if Fowlers worry from the past, that using mocks and asserting behaviour is focusing too much on the implementation. But like Fowler I haven't done BDD "for real" so I haven't seen how it plays out in the long term.

spyrefused 6y ago

I found quite useful an example I read in the book "The Art of Unit Testing" by Roy Osherove, it was something like this (I don't have it at hand, I repeat it from memory): Suppose you have to test the irrigation system of a tree in your garden, you want to test how many times and how much water is used, you have two ways to do it: 1) State-based: run the system for a few hours, during which it should water the tree several times. At the end of that time, you check the state of the watered tree: Is the soil moist enough, is the tree well, are the leaves green, etc. (Obviously this is just an example as this test would be complex in real life, but it is to get the idea across). 2) Interaction: We place a flow meter, a device that registers how much water is flowing through the device and at what times. At the end of the day, check that the device has been called the correct number of times, with the correct amount of water each time, and we don't worry about checking the status of the tree.

finnthehuman 6y ago

>I’ve read this article twice now and it feels like confused word soup Yeah, that'll happen with a lot of the testing "literature" from the 2005ish to 2015ish era. It was a popular topic at the time, and one I was searching for answers on too. But I was working at a company that was ahead of the curve on testing and I got so frustrated at publicly available resources on testing. During that time and still to this day, when I read about testing online I kinda feel like I'm drunk. I have an idea of what's going on, things sorta make sense but not really. I'm only understanding bits and pieces. There are unnecessary and over-complicated details on some things, suspiciously light on meaty topics. There was so much drivel written about testing back then that I still get angry remembering that people would say with a straight face "you don't need documentation, tests are your documentation."

arwhatever 6y ago

Might I recommend this overview? https://enterprisecraftsmanship.com/posts/when-to-mock/ https://enterprisecraftsmanship.com/posts/when-to-mock/ I more understand the Martin Fowler article quite well, but only after struggling with it and revisiting it (sporadically) over several years, and I really think this by Mr, Khorikov gives a much clearer, more straightforward overview.