3 ms·

I might-should've rephrased my original post to suggest that mocks for non-code-you-wrote deps are a waste, but being able to inject different implementations o

I might-should've rephrased my original post to suggest that mocks for non-code-you-wrote deps are a waste, but being able to inject different implementations of code you wrote for the sake of testing still has value. If you want to test how your code behaves when getting/setting data from a database, you fundamentally cannot mock the database. If you want to test how your code forms arguments to send to a dependency, you probably can, as long as you do both. If you're just testing that your args are correct without actually using them, you'll never really know. A specific example stands out in my mind when I watched a TDD-minded developer suggest that their code was complete because they injected a mocked version of DynamoDB into a service and then verified their tests ran. They committed and pushed without actually testing against DynamoDB. When tasked with _actually running it_, they found that there was a fundamental flaw in the way that developer perceived Dynamo's behavior, which obviously meant their mock was totally useless. If you want to test that you're pulling the right ID off an object to send to a database you can mock. If you want to test how your code actually behaves with a database, including failure scenarios and invalid arguments, you cannot. As usual, solutions are much more nuanced than I previously stated. I've moved away from mocks, not banished them entirely.

chihuahua 6y ago
Yes, there's a pretty fundamental problem: does your mock match the behavior of the real component? You can never be sure, unless you write extensive tests that compare your mock to the real component. And then those tests need the real component. But mocks could be useful for testing that your code correctly handles errors that are returned by an external component.
matthias509 6y ago
I’ve seen this over and over. The engineer building the mock is the same that wrote the code, so the same incorrect assumptions get baked into both. This is what makes mocks of limited utility. In the end, they don’t prove the code works and they are expensive to create and maintain.
- bird_monster 6y ago
  Yep. I kinda also think that the dev that wrote the code shouldn't be the dev that writes any of the tests, but that's much more of a philosophically sound but in-practice poor idea that I think about sometimes. I think it has an array of benefits that go totally unrealized if a person writes both. Things like, if your code is too complex for another engineer to write tests for, it's too complex for your team to maintain. If code standards aren't consistent enough in your codebase that other engineers feel the need to refactor/redo blocks of code in the name of style/readability/whatever, your codebase's style isn't automated well enough. Things like that. I just think it's so much of a better strategy, unfortunately up front it seems like a big time sink for (at surface level) little game. I also refuse to measure test coverage in my codebases. "How frequently do bugs show up in production" and "how frequently are bugs fixed without adding tests" are metrics I find valuable but are underrepresented in the testing space. If bugs don't make it to prod very often, and when they do they are fixed, your testing strategy is probably sufficient. There's no reason to write thousands of null checks and formatting validators if you don't need to. Tests require as much or more maintenance as code. There's no reason to write more of them than you need.
  - raducu 6y ago
    Some things can be mocked quite successfuly -- like ORM stuff. You can also write an integration test for the mocked component, let the rest of the code blaze away with the mocked version, it is usually a very good compromise.

chihuahua 6y ago

Yes, there's a pretty fundamental problem: does your mock match the behavior of the real component? You can never be sure, unless you write extensive tests that compare your mock to the real component. And then those tests need the real component. But mocks could be useful for testing that your code correctly handles errors that are returned by an external component.

matthias509 6y ago

I’ve seen this over and over. The engineer building the mock is the same that wrote the code, so the same incorrect assumptions get baked into both. This is what makes mocks of limited utility. In the end, they don’t prove the code works and they are expensive to create and maintain.

bird_monster 6y ago
Yep. I kinda also think that the dev that wrote the code shouldn't be the dev that writes any of the tests, but that's much more of a philosophically sound but in-practice poor idea that I think about sometimes. I think it has an array of benefits that go totally unrealized if a person writes both. Things like, if your code is too complex for another engineer to write tests for, it's too complex for your team to maintain. If code standards aren't consistent enough in your codebase that other engineers feel the need to refactor/redo blocks of code in the name of style/readability/whatever, your codebase's style isn't automated well enough. Things like that. I just think it's so much of a better strategy, unfortunately up front it seems like a big time sink for (at surface level) little game. I also refuse to measure test coverage in my codebases. "How frequently do bugs show up in production" and "how frequently are bugs fixed without adding tests" are metrics I find valuable but are underrepresented in the testing space. If bugs don't make it to prod very often, and when they do they are fixed, your testing strategy is probably sufficient. There's no reason to write thousands of null checks and formatting validators if you don't need to. Tests require as much or more maintenance as code. There's no reason to write more of them than you need.
- raducu 6y ago
  Some things can be mocked quite successfuly -- like ORM stuff. You can also write an integration test for the mocked component, let the rest of the code blaze away with the mocked version, it is usually a very good compromise.