5 ms·
This was one of the fun things about working on Midori, the whole system was available to us, components all communicated through well defined interfaces, you r
by pacaro 9y ago
This was one of the fun things about working on Midori, the whole system was available to us, components all communicated through well defined interfaces, you really could (if necessary) mock the interfaces above and below and be sure that your logic was being exercised. The browser team, in particular, pushed hard to be close to 100% coverage. The overall number for the project was (IIRC) in the 85% range.
When I'm writing tests, I'm not so concerned about what the tool tells me about which lines are covered, I like to work through the mental exercise of knowing which basis paths are covered. If a function has too many for me to reason about, that is a problem in itself.
As an aside, while I'm rambling, all the examples in the article appeared to represent unnecessary abstraction, which is the opposite problem. If you have many methods in a class with only one basis path, what purpose does the class serve. These testing concerns may be the code smell that points to deeper problems
- cbanek 9y ago> As an aside, while I'm rambling, all the examples in the article appeared to represent unnecessary abstraction, which is the opposite problem. One other thing about code coverage, unit testing, and other testing fads is I think they actively affect the architecture, and usually in the overthinking it way. Instead of having one tight bit of procedural code (which may have some state, or some dependency calls), people split it up into multiple classes, and then test each bit. This allows them to use the class architecture to mock things, but really has just multiplied out the amount of code. And in the end, you're running tests over the golden path probably even less. It's even possible to have 100% of the code covered, and not run the golden path, because you're always mocking out at least one bit.
- sbov 9y agoIt's strange to me that we continue to make languages that force us to make certain architectural choices to help facilitate testing.
- cbanek 9y agoIndeed. It's almost as if testing should be built into the language itself. (I'm always a big fan of including self-test code in projects)
- vincnetas 9y agoWhat are the languages that have testing not as afterthought?
- pacaro 9y agoThis depends on how you think of things. Some languages (Eiffel?) have pre- and post-conditions, which do some of this job. Much of the boilerplate testing done for dynamic languages is done by the compiler for static languages. The borrow checker in Rust is doing the work that a test suite and/or static analysis tool would be doing for C/C++
- realusername 9y agoDefinitely Ruby in general, testing is a core of the language culture.
- thinkloop 9y ago> testing should be built into the language itself I wonder what that would look like...
- KallDrexx 9y agoI think there's an argument to be made that if the desire for testing is a main driver for your architecture then your tests are too granular, and you aren't testing any internal integration points. In my experience that means that your tests are so tied to your current architecture that you can't even refactor without having to rewrite tests.
- cowardlydragon 9y agomocking / interaction / expect breaks encapsulation to perform "testing". Thus it is often a test of the implementation's assumptions when first written, and even worse, when the code is maintained/edited, the test is merely changed to get it to pass, because unit tests with mocks are usually: 1) fragile to implementation 2) opaque as to intent Whereas input/output integration points are more reliable, transparent, and less fragile to implementation changes if the interface is maintained. However, if you must do mock-level interaction testing, Spock has made it almost palatable in Javaland. This is one area where functional fans get to make the imperative folks eat their lunch.
- YZF 9y agoExactly. I've seen tests where some code calls: printf("hello world"); And the test is: mock_printf(string) { if string != "hello world" then fail; } Which is basically just duplicating your code as tests.
- tracker1 9y agoI think that depends on the language/environment... for example, it's usually MUCH easier to get high coverage, and to do just enough mocking testing against Node-style modules in JS than logic in a typical C# or Java project. Usually because interfaces need to be clearly/well defined and overridden... In JS there are tools (rewire, proxyquire, etc) that can be used with tests in order to easily inject/replace dependencies without having to write code to support DI nearly as much. In fact, I'd say that there's usually no reason not to be very close to 100% coverage in JS projects.
- mdpopescu 9y ago> If you have many methods in a class with only one basis path, what purpose does the class serve. It ties together related algorithms. You want "addDays(n)", "addHours(n)", "addMinutes(n)" and so on to be in the same class, even if they're one-liners.
- pacaro 9y agoClearly there are good examples either way. I'm more interested in the idea that this strong mismatch between the complexity of the code and the complexity of its tests is, in itself, a code smell that may point to an issue that is nothing to do with TDD. I note that the examples you give have trivial test cases - presuming you don't care about overflow, and if you do, then those methods now have basis paths, whether explicit or implicit depends on the language.