5 ms·

I feel like this distinction is arbitrary and unintuitive. A mock is literally just something that mimics something else. All "doubles" can be considered mocks.

by dpark 10y ago

I feel like this distinction is arbitrary and unintuitive. A mock is literally just something that mimics something else. All "doubles" can be considered mocks. Claiming that mocks insist on call verification seems like a very forced categorization. If there's value in this distinction, maybe we need new terminology instead of asserting that everyone who's conflated "stubs" and "mocks" for two decades is wrong. As a side note, verifying call patterns makes for terrible, brittle tests that cost far too much to maintain. This pattern is great for enforcing call patterns that truly that matter for, e.g, performance reasons (no unexpected cascade of DB calls), but is typically just a giant pain that provides little to no value, because instead of testing that the code does the right thing, you're testing that it doesn't change.

aikah 10y ago
I think the difference is mocks carry tests while Stubs are dumb fakes. I think the problem is the choice of words. Maybe mocks should actually be called "spies" instead and stubs "fakes".
- dpark 10y ago
  That terminology makes much more sense, especially the "spies" part. Personally, I tend to think of stubs as being super basic. A stub class for me returns default values or throws NotImplemented or something as simple as possible. It doesn't maintain meaningful state. Mocks for me are generally simplified implementations like an in-memory store that replaces a DB. Frameworks like Moq have muddied my definition though because they call things mocks when they only return canned data. Building a real mock (fake) with real state using Moq tends to be a pain.
- gary_bernhardt 10y ago
  Spies are a different thing: they allow any interaction, recording it all so that you can inspect it layer. Python's "mock" library, for example, is mostly a spy library.
  - dpark 10y ago
    > Python's "mock" library, for example, is mostly a spy library. It's almost as if the experts don't all agree with this terminology. Edit: This was a bit too snarky, but truthfully, I don't see evidence in the field that these definitions are widely used. I see evidence that an very small crowd is using these terms and asserting their correctness. Meanwhile the rest of the industry thinks anything they use as a "double" in a test is a "mock". Even the people writing widely-used mocking libraries aren't using this terminology consistently.
gary_bernhardt 10y ago
There are objects that demand to be interacted with; there are objects that allow only certain predefined interactions; there are objects that allow any interaction and record them all for later inspection; and there are objects that are simple implementations of interfaces whose "real" implementation is more complex. Each of these object types exists in practice, and each is clearly an example of some larger category. One group of people says "these should all be called mocks". Another group says "these are mocks, stubs, spies, and fakes, respectively, and collectively these are called test doubles". You can prefer the simplicity of calling them all "mocks" (simplicity is nice) but that leaves you with no way to distinguish between the types. These categories seem like hair-splitting if you've written relatively few tests using test doubles. Once you've written a few thousand, you start to wish for even finer-grained distinctions. This is how basically everything works as you get deeper into the details: you need words for the fine-grained distinctions. As far as I know, there has never been a competing set of terms. The only major positions are "we should use [the words I mentioned above]", which is mostly favored by those who have used a lot of doubles; and "we should not use separate words at all", mostly favored by those who haven't used a lot of doubles.
- searls 10y ago
  As a person who co-owns a company named Test Double, you'll never guess, dear reader, where I come down on this issue. Here is a GitHub wiki where I break them down a decent amount: https://github.com/testdouble/contributing-tests/wiki/Test-Double https://github.com/testdouble/contributing-tests/wiki/Test-D... I've also written a JavaScript library called testdouble.js that has pretty detailed docs on their background purpose: https://github.com/testdouble/testdouble.js/blob/master/docs/2-howto-purpose.md https://github.com/testdouble/testdouble.js/blob/master/docs... For what it's worth, Gerard Meszaros coined the term "test double" in his fantastic book (which most of you have never heard of, much less read) called xUnit Patterns. This is relevant to Gary's point, because like most things that require expertise specific terms only become useful as you grow close to the work. Most people don't use test doubles to facilitate isolated test-driven development, so most people won't come across the terms (or their sources). I can't claim to be an expert in much, but—for reasons that evade my understanding—I do claim to be one in test double science™. What I can say is it can be tiresome when an expert has to constantly engage with relative novices who lack a curiosity with really engaging on the topic. Over the years, I've grown comfortable with the fact that most people—by which I mean, almost _all_ people—who want to talk about test doubles actually just want to fake something out in a test that's causing them pain. This is a perfectly fine thing to want to do, but it's orthogonal to why nuanced test double types & subtypes and jargon was ever necessary, which is mostly steeped in very rigorous schools of test-driven development. There's no wrong way to do it. If you only write relatively "integrated" tests, nomenclature like "mock that out" won't cause you much harm but for the occasional eyeroll from people like me. But if you're interested in using tests as a sounding board to improve the design of your systems, they're one toolset for one style of approach (of many) that can help you in that endeavor—and which require the kind of specificity that Gary described above.
MichaelMoser123 10y ago
Also most mocking libraries require that tested/mocked classes must be derived from an interface, when an interface is not required this adds overhead just for the purpose of testing, this is bad. Correction: googletest/googlemock (in c++) can create mocks of non virtual classes, they use scripts to generate the mock code. Still you need a special test build to link with the mocked classes, in java/.net you dont have this problem because of reflection.

by dpark 10y ago

aikah 10y ago

I think the difference is mocks carry tests while Stubs are dumb fakes. I think the problem is the choice of words. Maybe mocks should actually be called "spies" instead and stubs "fakes".

dpark 10y ago
That terminology makes much more sense, especially the "spies" part. Personally, I tend to think of stubs as being super basic. A stub class for me returns default values or throws NotImplemented or something as simple as possible. It doesn't maintain meaningful state. Mocks for me are generally simplified implementations like an in-memory store that replaces a DB. Frameworks like Moq have muddied my definition though because they call things mocks when they only return canned data. Building a real mock (fake) with real state using Moq tends to be a pain.
gary_bernhardt 10y ago
Spies are a different thing: they allow any interaction, recording it all so that you can inspect it layer. Python's "mock" library, for example, is mostly a spy library.
- dpark 10y ago
  > Python's "mock" library, for example, is mostly a spy library. It's almost as if the experts don't all agree with this terminology. Edit: This was a bit too snarky, but truthfully, I don't see evidence in the field that these definitions are widely used. I see evidence that an very small crowd is using these terms and asserting their correctness. Meanwhile the rest of the industry thinks anything they use as a "double" in a test is a "mock". Even the people writing widely-used mocking libraries aren't using this terminology consistently.

gary_bernhardt 10y ago

There are objects that demand to be interacted with; there are objects that allow only certain predefined interactions; there are objects that allow any interaction and record them all for later inspection; and there are objects that are simple implementations of interfaces whose "real" implementation is more complex. Each of these object types exists in practice, and each is clearly an example of some larger category. One group of people says "these should all be called mocks". Another group says "these are mocks, stubs, spies, and fakes, respectively, and collectively these are called test doubles". You can prefer the simplicity of calling them all "mocks" (simplicity is nice) but that leaves you with no way to distinguish between the types. These categories seem like hair-splitting if you've written relatively few tests using test doubles. Once you've written a few thousand, you start to wish for even finer-grained distinctions. This is how basically everything works as you get deeper into the details: you need words for the fine-grained distinctions. As far as I know, there has never been a competing set of terms. The only major positions are "we should use [the words I mentioned above]", which is mostly favored by those who have used a lot of doubles; and "we should not use separate words at all", mostly favored by those who haven't used a lot of doubles.

searls 10y ago
As a person who co-owns a company named Test Double, you'll never guess, dear reader, where I come down on this issue. Here is a GitHub wiki where I break them down a decent amount: https://github.com/testdouble/contributing-tests/wiki/Test-Double https://github.com/testdouble/contributing-tests/wiki/Test-D... I've also written a JavaScript library called testdouble.js that has pretty detailed docs on their background purpose: https://github.com/testdouble/testdouble.js/blob/master/docs/2-howto-purpose.md https://github.com/testdouble/testdouble.js/blob/master/docs... For what it's worth, Gerard Meszaros coined the term "test double" in his fantastic book (which most of you have never heard of, much less read) called xUnit Patterns. This is relevant to Gary's point, because like most things that require expertise specific terms only become useful as you grow close to the work. Most people don't use test doubles to facilitate isolated test-driven development, so most people won't come across the terms (or their sources). I can't claim to be an expert in much, but—for reasons that evade my understanding—I do claim to be one in test double science™. What I can say is it can be tiresome when an expert has to constantly engage with relative novices who lack a curiosity with really engaging on the topic. Over the years, I've grown comfortable with the fact that most people—by which I mean, almost _all_ people—who want to talk about test doubles actually just want to fake something out in a test that's causing them pain. This is a perfectly fine thing to want to do, but it's orthogonal to why nuanced test double types & subtypes and jargon was ever necessary, which is mostly steeped in very rigorous schools of test-driven development. There's no wrong way to do it. If you only write relatively "integrated" tests, nomenclature like "mock that out" won't cause you much harm but for the occasional eyeroll from people like me. But if you're interested in using tests as a sounding board to improve the design of your systems, they're one toolset for one style of approach (of many) that can help you in that endeavor—and which require the kind of specificity that Gary described above.

MichaelMoser123 10y ago

Also most mocking libraries require that tested/mocked classes must be derived from an interface, when an interface is not required this adds overhead just for the purpose of testing, this is bad. Correction: googletest/googlemock (in c++) can create mocks of non virtual classes, they use scripts to generate the mock code. Still you need a special test build to link with the mocked classes, in java/.net you dont have this problem because of reflection.