3 ms·
To play devil's advocate: What exactly is the difference between a Mock and a Fake? In one you store any side effecting values in memory with a predefined beha
by jackcviers3 6y ago
To play devil's advocate:
What exactly is the difference between a Mock and a Fake? In one you store any side effecting values in memory with a predefined behavior in a separate class. In the other, you define the predefined behavior in the test class by intercepting calls to external components. Your test coupling to internal implementation is just as high, or higher. Your Fake needs to do exactly what you expect in isolation of the class that depends upon it, or the test result won't work. In addition, without the reflection capability provided by many mocking frameworks, it is impossible to "Fake" implementations of certain library components via `sealed` or `final` modifiers on external library classes. It is also impossible to verify side effects without a method to expose those side effect actions in your Fake, which will force, at the very least, an extended declaration in your test class.
Integration testing is an essential part of the test pyramid. When your required interface for your injected mock/fake changes, your test is going to have to change. Your tests will be just as coupled to your implementation with a fake. The coupling is just hidden in your fake implementation.
In addition, often we are mocking huge interfaces with complex inner machinery that we don't fully understand.
The real issue with mocking and faking is the necessity to mock and fake to isolate tests due to poor interface design in the class under test, usually focused around side-effecting methods.
All side effecting methods must return some sum type that is not void that indicates success or failure of the side-effecting actions. All side effecting actions taken by the code must be apparent at the type level of this success or failure wrapper. Tests must use random, generated input parameters to verify the returns generated are correct. And as always, a type signifies for all X, then Y; while a test signifies for this x in X, then this y in Y. Types are enforced by the compiler or runtime, which (though they may have a bugs themselves) are never wrong as the interpreters of code. What you get is what you get, even if it doesn't match up with what you think a particular type expression should output.
Even with effect encoding and property-based generated testing and compiler/runtime proofs that your program interfaces are correct, it is highly unlikely that a person incapable of designing and implementing a correct implementation would be capable of designing and implementing a correct test for that implementation or type hierarchy. Even with effect encoding at the type level all the way down, integration testing between components against live infrastructure remains the best way of detecting bugs -- all your unit testing and design assumptions are exercised by exhaustive business case testing.
Mathematical proofs derived from the type system are very strong validators of design. But they can only verify that the assumptions and constraints the designer has created are correct. Only verification via integration and user testing can verify that those assumptions and constraints actually meet the business requirements of a given design.
All of this is to say, if you are focusing on using Fakes over Mocks and relying upon tests to ensure correct behavior, eventually you will fail. In other words, bugs will eventually manifest in any system in which automated integration and user testing that verifies exhaustively the business cases requested by the business user is not part of the release process.
Unit tests and types are measures of the correctness of a given design with respect to itself. Unit tests and types are not the measure of the reliability of a given design with respect to the use of software artifacts, and cannot be. Nor can coupled/uncoupled unit tests improve the ease of maintenance of a program. Types and unit tests are a useful design and maintenance tools that can make the intent of a design clearer to the reader of a piece of code, and can help identify times while implementing where you are violating the constraints you have set for yourself. But they cannot make refactoring or interface changes easier to encode without inflicting line churn cost in the tests. If an injection is exposed to the user of a class, it is exposed, and you must deal with it.