5 ms·

Can you expand a bit on the distinction you're making between mocks and stubs, and why you believe stubs are both underused and useful? Perhaps I'm doing it wr

Can you expand a bit on the distinction you're making between mocks and stubs, and why you believe stubs are both underused and useful? Perhaps I'm doing it wrong, but I've found in my experience that bugs I miss in testing increase proportionally with my use of mocks or stubs. Invariably, when I'm forced to stub something and replicate its behavior I miss something subtle that comes back to bite me.

latch 15y ago
I think there's 2 problem. First, people rely too heavily on mocks/stubs/fakes/(whatever you want to call them). This has gotten better in the past 4 or so years (in my mind, largely because of Rails and it being "acceptable" to hit a DB in a test - though it might actually predate Rails). I think this problem is pretty straightforward to understand (again, especially when you look at the Rails way to test a DB interaction versus a more "traditional" way). However, there are instances where mocks/fakes/stubs are important. Some outside dependencies might not be accessible during testing, might not be predictable/deterministic, might be too slow or might simply involve too much setup. Also, solely relying on "real" implementations can also make your tests too brittle. Why should the logic for can_legally_drink? break if the DB column is changed from DOB to dateofbirth (ok, that's an extreme and poor example, but you get the idea, hopefully). Anyways, assuming you agree that sometimes a fake is simply better, you get into mocks vs stubs. To me a stub is dumb and a mock is strict. Stubs also automatically reply with canned answers, mocks don't. A mock is used to assert that a certain expected call was made. A stub is used just to get your test to move over a line of code. The problem is that people use mocks over and over again. Re-specifying the same expectations in every test..making X tests break when you change the behavior of the interaction, versus just 1 test. As semi-pseudocode (and yes, it might be better to just hit the DB in this case): def self.login_user(name, password) user = Store.get_user_by_name(name) return user.nil? || user.password != password ? nil : user end To me, you kinda wanna check that the above code properly interacts with someStore. This is when you use a mock: it "gets the user from the store" do Store.should_receive(:get_user_by_name).with('leto') User.login_user ('leto', 'ghanima') end You also want to make sure the password matching works. This is where people go wrong. They'll use a mock (strict) again..which'll just repeat the above code...except this test has really nothing to do with how store...we just want to get over the line of code: it "returns nil if the passwords don't match" do Store.stub!(:get_user_by_name).and_return(User.new) User.login_user('leto', 'ghanima').should be_nil end If you are familiar with jMock, it's kinda the difference between allowing and oneOf. oneOf is very strict, allowing isn't. You really should use allowing whenever you aren't explicitly testing the interaction (and you shouldn't explicitly test the interaction in more than 1 test (dedicated to testing said interaction)). Some framework even return smart canned answers. So instead of returning nil, they'll default to a new instance of the type, or an empty array, or some default scalar value (like an empty string). This is particularly useful when you just want to get passed a null reference (which is exceedingly common). Does that make sense?
- jsdalton 15y ago
  Thanks, it makes perfect sense and is pretty much why I virtually never use mocks. In fact, my personal philosophy is at odds with most advice I have seen around unit testing, in that I try to avoid the use of even stubs wherever possible. In other words, I try to make my unit tests as much like integration tests as possible. Obvious exceptions are things like third-party APIs, which must be stubbed, but everything else I try to get close to production code.
- astral303 15y ago
  I've had similar experience with strict mocks. However, mocks don't need to be strict. In Java, for example, the current mocking tool of choice is Mockito, which returns "nice" mocks that return nulls, empty collections and such. I can't seem to find a blog post discussing the merits, but IME strict mocking leads to a lot of tests breaking just because you added one more piece of functionality to a method. Now, this causes you to edit all sorts of existing tests to cause them to ignore a call to collaborator. While you can extract common expectations and such to eliminate duplication, you still end up with many tests failing due to one small change in behavior. Normally your tests tend to assert one or two behaviors per test. With strict mocking, your tests become awkwardly cluttered by setting up expectations/interactions for things that are unrelated to what they are asserting. With nice mocking, it's much easier to get away with only the expectations/interactions that are directly relevant to what a single test is asserting. This also means that you get more fine-grained failures, which speeds up diagnostics.

latch 15y ago

I think there's 2 problem. First, people rely too heavily on mocks/stubs/fakes/(whatever you want to call them). This has gotten better in the past 4 or so years (in my mind, largely because of Rails and it being "acceptable" to hit a DB in a test - though it might actually predate Rails). I think this problem is pretty straightforward to understand (again, especially when you look at the Rails way to test a DB interaction versus a more "traditional" way). However, there are instances where mocks/fakes/stubs are important. Some outside dependencies might not be accessible during testing, might not be predictable/deterministic, might be too slow or might simply involve too much setup. Also, solely relying on "real" implementations can also make your tests too brittle. Why should the logic for can_legally_drink? break if the DB column is changed from DOB to dateofbirth (ok, that's an extreme and poor example, but you get the idea, hopefully). Anyways, assuming you agree that sometimes a fake is simply better, you get into mocks vs stubs. To me a stub is dumb and a mock is strict. Stubs also automatically reply with canned answers, mocks don't. A mock is used to assert that a certain expected call was made. A stub is used just to get your test to move over a line of code. The problem is that people use mocks over and over again. Re-specifying the same expectations in every test..making X tests break when you change the behavior of the interaction, versus just 1 test. As semi-pseudocode (and yes, it might be better to just hit the DB in this case): def self.login_user(name, password) user = Store.get_user_by_name(name) return user.nil? || user.password != password ? nil : user end To me, you kinda wanna check that the above code properly interacts with someStore. This is when you use a mock: it "gets the user from the store" do Store.should_receive(:get_user_by_name).with('leto') User.login_user ('leto', 'ghanima') end You also want to make sure the password matching works. This is where people go wrong. They'll use a mock (strict) again..which'll just repeat the above code...except this test has really nothing to do with how store...we just want to get over the line of code: it "returns nil if the passwords don't match" do Store.stub!(:get_user_by_name).and_return(User.new) User.login_user('leto', 'ghanima').should be_nil end If you are familiar with jMock, it's kinda the difference between allowing and oneOf. oneOf is very strict, allowing isn't. You really should use allowing whenever you aren't explicitly testing the interaction (and you shouldn't explicitly test the interaction in more than 1 test (dedicated to testing said interaction)). Some framework even return smart canned answers. So instead of returning nil, they'll default to a new instance of the type, or an empty array, or some default scalar value (like an empty string). This is particularly useful when you just want to get passed a null reference (which is exceedingly common). Does that make sense?

jsdalton 15y ago
Thanks, it makes perfect sense and is pretty much why I virtually never use mocks. In fact, my personal philosophy is at odds with most advice I have seen around unit testing, in that I try to avoid the use of even stubs wherever possible. In other words, I try to make my unit tests as much like integration tests as possible. Obvious exceptions are things like third-party APIs, which must be stubbed, but everything else I try to get close to production code.
astral303 15y ago
I've had similar experience with strict mocks. However, mocks don't need to be strict. In Java, for example, the current mocking tool of choice is Mockito, which returns "nice" mocks that return nulls, empty collections and such. I can't seem to find a blog post discussing the merits, but IME strict mocking leads to a lot of tests breaking just because you added one more piece of functionality to a method. Now, this causes you to edit all sorts of existing tests to cause them to ignore a call to collaborator. While you can extract common expectations and such to eliminate duplication, you still end up with many tests failing due to one small change in behavior. Normally your tests tend to assert one or two behaviors per test. With strict mocking, your tests become awkwardly cluttered by setting up expectations/interactions for things that are unrelated to what they are asserting. With nice mocking, it's much easier to get away with only the expectations/interactions that are directly relevant to what a single test is asserting. This also means that you get more fine-grained failures, which speeds up diagnostics.