7 ms·

There are objects that demand to be interacted with; there are objects that allow only certain predefined interactions; there are objects that allow any interac

There are objects that demand to be interacted with; there are objects that allow only certain predefined interactions; there are objects that allow any interaction and record them all for later inspection; and there are objects that are simple implementations of interfaces whose "real" implementation is more complex. Each of these object types exists in practice, and each is clearly an example of some larger category. One group of people says "these should all be called mocks". Another group says "these are mocks, stubs, spies, and fakes, respectively, and collectively these are called test doubles". You can prefer the simplicity of calling them all "mocks" (simplicity is nice) but that leaves you with no way to distinguish between the types. These categories seem like hair-splitting if you've written relatively few tests using test doubles. Once you've written a few thousand, you start to wish for even finer-grained distinctions. This is how basically everything works as you get deeper into the details: you need words for the fine-grained distinctions. As far as I know, there has never been a competing set of terms. The only major positions are "we should use [the words I mentioned above]", which is mostly favored by those who have used a lot of doubles; and "we should not use separate words at all", mostly favored by those who haven't used a lot of doubles.

searls 10y ago
As a person who co-owns a company named Test Double, you'll never guess, dear reader, where I come down on this issue. Here is a GitHub wiki where I break them down a decent amount: https://github.com/testdouble/contributing-tests/wiki/Test-Double https://github.com/testdouble/contributing-tests/wiki/Test-D... I've also written a JavaScript library called testdouble.js that has pretty detailed docs on their background purpose: https://github.com/testdouble/testdouble.js/blob/master/docs/2-howto-purpose.md https://github.com/testdouble/testdouble.js/blob/master/docs... For what it's worth, Gerard Meszaros coined the term "test double" in his fantastic book (which most of you have never heard of, much less read) called xUnit Patterns. This is relevant to Gary's point, because like most things that require expertise specific terms only become useful as you grow close to the work. Most people don't use test doubles to facilitate isolated test-driven development, so most people won't come across the terms (or their sources). I can't claim to be an expert in much, but—for reasons that evade my understanding—I do claim to be one in test double science™. What I can say is it can be tiresome when an expert has to constantly engage with relative novices who lack a curiosity with really engaging on the topic. Over the years, I've grown comfortable with the fact that most people—by which I mean, almost _all_ people—who want to talk about test doubles actually just want to fake something out in a test that's causing them pain. This is a perfectly fine thing to want to do, but it's orthogonal to why nuanced test double types & subtypes and jargon was ever necessary, which is mostly steeped in very rigorous schools of test-driven development. There's no wrong way to do it. If you only write relatively "integrated" tests, nomenclature like "mock that out" won't cause you much harm but for the occasional eyeroll from people like me. But if you're interested in using tests as a sounding board to improve the design of your systems, they're one toolset for one style of approach (of many) that can help you in that endeavor—and which require the kind of specificity that Gary described above.
- dpark 10y ago
  My feeling after reading your doc is that "mock" is an ill-defined term and asserting the specific meaning here is just prescriptive linguism. Not only does your own doc agree that in common usage, "mock" is a generic term for any test double, your docs also assert that a "partial mock" is "any actual object which has been wrapped or changed to provide artificial responses to some methods but not others". So by their "precise" definitions, a "partial mock" is actually not a partial "mock". Come on.
tatterdemalion 10y ago
One of the things that programmers do that I find most frustrating is proclaim that the definition of words that they use is the objectively correct definition, and frame their entire discussion as an argument that other people aren't using words correctly. Pick any term and the kind of argument comes to mind - "type," "object," "metaprogramming," "dependency injection," "design pattern," etc. This article has a lot of good content about testing strategies, as well as an introduction to some of the jargon used in a certain testing discourse, but the argument is framed as telling you that mocks aren't stubs, and you, the reader, have been using these words incorrectly. But the reader has only been using these words differently. This is a very common pattern of behavior among programmers and I think it is worthwhile to push back against it, separately from acknowledging the useful information in this article. (And the grandparent comment you replied to also did some of this as well, of course!)
- gary_bernhardt 10y ago
  I agree with you in general. In this case there's only one coherent taxonomy in widespread use so I think it's reasonable to say "just use it". It's unfortunate that people who aren't already experts on a topic can't tell whether that kind of decree represents consensus or someone's idiosyncratic opinion (and there's plenty of both in general). Framing is a different thing, though, and I can't comment there since I haven't re-read the whole article recently.
dpark 10y ago
> One group of people says "these should all be called mocks". Another group says "these are mocks, stubs, spies, and fakes, respectively, and collectively these are called test doubles". You can prefer the simplicity of calling them all "mocks" (simplicity is nice) but that leaves you with no way to distinguish between the types. I'm not saying that those should all be called mocks. I'm saying that are all called mocks by many developers already. There are frameworks that call themselves things like Moq and EasyMock that support multiple of the paradigms you describe. Picking "objects that demand to be interacted with" as the only real "mocks" feels arbitrary and frankly wrong given that the term "mock" has exactly nothing to do with requiring tests to cause certain method calls with certain params. I'm totally fine with saying that we should recognize the difference between types of doubles and give them different names. I'm not on board with saying that suddenly a "method" is a function with no side effects as if that's some standard definition. > and "we should not use separate words at all", mostly favored by those who haven't used a lot of doubles. This is not subtle. And asserting that every with experience agrees with you does not make it true. I have written enough tests to have a legitimate opinion on this. I have also written test code filled with "mocks" that predates this article.
- gary_bernhardt 10y ago
  I didn't say that you are wrong. I didn't say "that everyone with experience agrees with [me]". I specifically used the words "mostly favored by" twice. Mostly does not mean "everyone". It's fine if you want to use different words. But you either have words for the different types or not. And if you have separate words, you either use the five existing ones or try to push five new ones. I think it's significant that you objected to my claim about the popular taxonomy without suggesting five alternative terms with even a fraction of the adoption. (I've built an entire career partly on popularizing this stuff and I've never even encountered a competing set of terms!)
  - dpark 10y ago
    My comment at the bottom was pointing out that you basically called people who don't agree with you amateurs. Yes, you qualified it with "most" but the sentiment remains. This sort of response is especially unhelpful when the "amateurs" being dismissed are not laypeople but professionals in the same field. It's like a group of heart surgeons asserting that arteries are very specifically the blood vessels that carry blood from the heart to the lungs and then calling general practitioners inexperienced for not getting on board with the retroactive redefinition. I also don't believe it's reasonable to assert that I have to provide new terminology in order to point out that this definition is fundamentally problematic and does not match the common usage. But if you want a word, call it a preset (because all the interactions are defined in advance) or a mine field (because if you step in the wrong place, it blows up your test, and because your grandkids will still be dealing with the mess).
Rob_van_Hoose 10y ago
I think you hit the nail on the head -- "as you get deeper into the details: you need words for the fine-grained distinctions." Indeed, nomenclature gives us clarity of concept and that precision yields efficiency of thought and communication. Are the names arbitrary? I certainly thought so when I started. How did mock objects became expectation testing instruments and stubs didn't? How exactly is a stubbed class not a fake? Or would we use the word fake only for something like an in memory database? Aren't all test doubles fakes? Except, if a spy just wraps the object class, but allows calls through, then is it a double at all, or would it just be a probe? So are partial mocks and proxies liars? Regardless of whether the names are arbitrary, it's useful to have them. It sure beats "double type A" and "Type B double". "Perhaps you should use the Second Double Form, the Third Form isn't really necessary." Ugh. Perhaps there isn't a competing name set because it's hard enough getting everyone to agree these are distinct concepts. Maybe that would be easier with better names. After all, if there's popular rejection of the distinctions until the underlying concepts are understood, then the names aren't leading us to water on their own merit. It seems, also, the more I hear the generation before mine talk about doubles, the more I find they use them very sparingly. The Beck's and Fowler's seem laid back about the nomenclature in favor of shared processual understanding.
jbrains 10y ago
This happens all the time. We have a vague notion of a thing, so we call it "X". Later, as we learn more, we realize that we can think of X as a category of things, and that one of those things, annoyingly, should probably be called "X". Now we have a problem. Some people will call the thing X and some people will continue to call the category X, because they read our blog posts from five years ago, because they were so popular, because we are so awesome. This is a sign of a good thing, even though it causes confusion. Now we have to roam the countryside saying "I know we say 'X' to mean this general thing, but we also say 'X' to mean that more specific thing. I'm really sorry. It's my fault. I recommend that you say 'Y' to refer to the category, but be prepared to read 'X' on the web. It is what it is." It is what it is.

searls 10y ago

As a person who co-owns a company named Test Double, you'll never guess, dear reader, where I come down on this issue. Here is a GitHub wiki where I break them down a decent amount: https://github.com/testdouble/contributing-tests/wiki/Test-Double https://github.com/testdouble/contributing-tests/wiki/Test-D... I've also written a JavaScript library called testdouble.js that has pretty detailed docs on their background purpose: https://github.com/testdouble/testdouble.js/blob/master/docs/2-howto-purpose.md https://github.com/testdouble/testdouble.js/blob/master/docs... For what it's worth, Gerard Meszaros coined the term "test double" in his fantastic book (which most of you have never heard of, much less read) called xUnit Patterns. This is relevant to Gary's point, because like most things that require expertise specific terms only become useful as you grow close to the work. Most people don't use test doubles to facilitate isolated test-driven development, so most people won't come across the terms (or their sources). I can't claim to be an expert in much, but—for reasons that evade my understanding—I do claim to be one in test double science™. What I can say is it can be tiresome when an expert has to constantly engage with relative novices who lack a curiosity with really engaging on the topic. Over the years, I've grown comfortable with the fact that most people—by which I mean, almost _all_ people—who want to talk about test doubles actually just want to fake something out in a test that's causing them pain. This is a perfectly fine thing to want to do, but it's orthogonal to why nuanced test double types & subtypes and jargon was ever necessary, which is mostly steeped in very rigorous schools of test-driven development. There's no wrong way to do it. If you only write relatively "integrated" tests, nomenclature like "mock that out" won't cause you much harm but for the occasional eyeroll from people like me. But if you're interested in using tests as a sounding board to improve the design of your systems, they're one toolset for one style of approach (of many) that can help you in that endeavor—and which require the kind of specificity that Gary described above.

dpark 10y ago
My feeling after reading your doc is that "mock" is an ill-defined term and asserting the specific meaning here is just prescriptive linguism. Not only does your own doc agree that in common usage, "mock" is a generic term for any test double, your docs also assert that a "partial mock" is "any actual object which has been wrapped or changed to provide artificial responses to some methods but not others". So by their "precise" definitions, a "partial mock" is actually not a partial "mock". Come on.

tatterdemalion 10y ago

One of the things that programmers do that I find most frustrating is proclaim that the definition of words that they use is the objectively correct definition, and frame their entire discussion as an argument that other people aren't using words correctly. Pick any term and the kind of argument comes to mind - "type," "object," "metaprogramming," "dependency injection," "design pattern," etc. This article has a lot of good content about testing strategies, as well as an introduction to some of the jargon used in a certain testing discourse, but the argument is framed as telling you that mocks aren't stubs, and you, the reader, have been using these words incorrectly. But the reader has only been using these words differently. This is a very common pattern of behavior among programmers and I think it is worthwhile to push back against it, separately from acknowledging the useful information in this article. (And the grandparent comment you replied to also did some of this as well, of course!)

gary_bernhardt 10y ago
I agree with you in general. In this case there's only one coherent taxonomy in widespread use so I think it's reasonable to say "just use it". It's unfortunate that people who aren't already experts on a topic can't tell whether that kind of decree represents consensus or someone's idiosyncratic opinion (and there's plenty of both in general). Framing is a different thing, though, and I can't comment there since I haven't re-read the whole article recently.

dpark 10y ago

> One group of people says "these should all be called mocks". Another group says "these are mocks, stubs, spies, and fakes, respectively, and collectively these are called test doubles". You can prefer the simplicity of calling them all "mocks" (simplicity is nice) but that leaves you with no way to distinguish between the types. I'm not saying that those should all be called mocks. I'm saying that are all called mocks by many developers already. There are frameworks that call themselves things like Moq and EasyMock that support multiple of the paradigms you describe. Picking "objects that demand to be interacted with" as the only real "mocks" feels arbitrary and frankly wrong given that the term "mock" has exactly nothing to do with requiring tests to cause certain method calls with certain params. I'm totally fine with saying that we should recognize the difference between types of doubles and give them different names. I'm not on board with saying that suddenly a "method" is a function with no side effects as if that's some standard definition. > and "we should not use separate words at all", mostly favored by those who haven't used a lot of doubles. This is not subtle. And asserting that every with experience agrees with you does not make it true. I have written enough tests to have a legitimate opinion on this. I have also written test code filled with "mocks" that predates this article.

gary_bernhardt 10y ago
I didn't say that you are wrong. I didn't say "that everyone with experience agrees with [me]". I specifically used the words "mostly favored by" twice. Mostly does not mean "everyone". It's fine if you want to use different words. But you either have words for the different types or not. And if you have separate words, you either use the five existing ones or try to push five new ones. I think it's significant that you objected to my claim about the popular taxonomy without suggesting five alternative terms with even a fraction of the adoption. (I've built an entire career partly on popularizing this stuff and I've never even encountered a competing set of terms!)
- dpark 10y ago
  My comment at the bottom was pointing out that you basically called people who don't agree with you amateurs. Yes, you qualified it with "most" but the sentiment remains. This sort of response is especially unhelpful when the "amateurs" being dismissed are not laypeople but professionals in the same field. It's like a group of heart surgeons asserting that arteries are very specifically the blood vessels that carry blood from the heart to the lungs and then calling general practitioners inexperienced for not getting on board with the retroactive redefinition. I also don't believe it's reasonable to assert that I have to provide new terminology in order to point out that this definition is fundamentally problematic and does not match the common usage. But if you want a word, call it a preset (because all the interactions are defined in advance) or a mine field (because if you step in the wrong place, it blows up your test, and because your grandkids will still be dealing with the mess).

Rob_van_Hoose 10y ago

I think you hit the nail on the head -- "as you get deeper into the details: you need words for the fine-grained distinctions." Indeed, nomenclature gives us clarity of concept and that precision yields efficiency of thought and communication. Are the names arbitrary? I certainly thought so when I started. How did mock objects became expectation testing instruments and stubs didn't? How exactly is a stubbed class not a fake? Or would we use the word fake only for something like an in memory database? Aren't all test doubles fakes? Except, if a spy just wraps the object class, but allows calls through, then is it a double at all, or would it just be a probe? So are partial mocks and proxies liars? Regardless of whether the names are arbitrary, it's useful to have them. It sure beats "double type A" and "Type B double". "Perhaps you should use the Second Double Form, the Third Form isn't really necessary." Ugh. Perhaps there isn't a competing name set because it's hard enough getting everyone to agree these are distinct concepts. Maybe that would be easier with better names. After all, if there's popular rejection of the distinctions until the underlying concepts are understood, then the names aren't leading us to water on their own merit. It seems, also, the more I hear the generation before mine talk about doubles, the more I find they use them very sparingly. The Beck's and Fowler's seem laid back about the nomenclature in favor of shared processual understanding.

jbrains 10y ago

This happens all the time. We have a vague notion of a thing, so we call it "X". Later, as we learn more, we realize that we can think of X as a category of things, and that one of those things, annoyingly, should probably be called "X". Now we have a problem. Some people will call the thing X and some people will continue to call the category X, because they read our blog posts from five years ago, because they were so popular, because we are so awesome. This is a sign of a good thing, even though it causes confusion. Now we have to roam the countryside saying "I know we say 'X' to mean this general thing, but we also say 'X' to mean that more specific thing. I'm really sorry. It's my fault. I recommend that you say 'Y' to refer to the category, but be prepared to read 'X' on the web. It is what it is." It is what it is.