3 ms·
The author suggests redirecting the scope of "TDD" to testing his app's public API endpoints, rather than internal functions. His logic is that public API endpo
by chatmasta 12y ago
The author suggests redirecting the scope of "TDD" to testing his app's public API endpoints, rather than internal functions. His logic is that public API endpoints represent the contracts for business logic with customers, and therefore are the highest level that can be accurately tested.
I am not convinced of this argument. Obviously, it's important to heavily test the endpoints of any API you make public. In fact, that's a situation where you want 100% coverage. However, testing the endpoints of an API does not allow you to forego testing its internals. At best, a passing suite of API endpoints should give you reasonable confidence that the internals are functioning properly.
An API is an abstraction over many moving parts. Any good API will consolidate thousands of lines of business logic into a few endpoints. There is a lot of room for error in the layer of abstraction between API and internal logic. It's entirely possible that a API endpoints could appear to be functioning properly, but actually be relying on broken internal code.
For example, consider a fruit basket API. You can insert fruit into the basket, and check what fruit is in the basket. A suite of tests for the API endpoints could insert fruit, and then check that it's there. In this case, the API is hiding a lot of internal logic. Storage mechanisms, data persistence, fault tolerance, and a slew of other logic decisions are completely opaque to the API consumer.
What if the internal code incorrectly stores the fruit in a temporary file? The API test will pass if it inserts fruit and then checks that it's there. But it is not going to check for that same fruit in an hour. What if it's gone?
Internal logic like data persistence is (rightfully) opaque to API consumers. That means that no testing of API endpoints can validate all internal logic. Therefore, you cannot ignore testing the internal logic in favor of only testing API endpoints.
- RussianCow 12y agoThis, a million times. If nothing else, testing internal logic allows you to more quickly do things like refactor and fix bugs, because you'll know exactly what part of the code is broken (otherwise you'll be shooting in the dark).
- lowbloodsugar 12y agoThey key problem is the number of combinations of inputs, outputs and state. And since its difficult to set up state correctly in such an api-based test, then the tests invariably end up using the api to initialize state. So that means we're dealing with the permutations of inputs, outputs and state, not merely the combinations. Basically anything more complicated than a "unit" is impossible to test for every permutation and as a result you get bugs when things happen in production that nobody anticipated a test for. In short, been there, done that. Yes, way easier to write less tests that don't test everything. Witnessed plenty of six figure+ bugs as result. Meanwhile, if refactoring means breaking a load of tests then you're doing it wrong. TDD is actually pretty hard to learn and it seems to me most people give up when the have to refactor code that they've tested internal details. Given their knowledge they conclude TDD is bogus, rather than wonder if perhaps their tests are bogus.
- rpedela 12y agoIn theory yes, in practice I mostly disagree. The ideal is to test everything, the public API and all the internals. But there are only so many hours in the day so if you have to choose (which most of us do) then the best place to put the majority of the testing effort is in the public API. The reality is that there will always be bugs...always. While 100% bug-free is a valiant goal, it is pretty much impossible to achieve in a complex system. However what is achievable (or more achievable at least) is being almost completely bug-free at the public API level. From a business point of view, this is the only thing that really matters because it is the only thing your users and customers care about. In your fruit basket example, there are two fixes for this. Either improve the public API tests to verify that an hour later the file is still there. Or you use a storage technology where you can assume it will work correctly (Postgres, S3, etc) and then have a code review. There is definitely an exception to "only need to test the public API". That is when a very complex component has to be built from scratch or almost scratch in order to support the public API. For example, I personally need to build a Lucene-based search server (Solr and ES don't fit my use case). While I shouldn't need unit tests for Lucene itself, I do need unit tests to verify the threading model, file structure, and couple other things I write from scratch are correct because the complexity is so high and I don't trust myself or anyone else to write it correct the first time.
- kibibu 12y ago> What if the internal code incorrectly stores the fruit in a temporary file? Genuinely curious - how would you write a lower-level unit test to account for this? If my "testAddFruit" method directly hits the temporally flimsy datastore, how is that any better than the API doing the same, albeit through multiple abstraction tiers?
- deleted 12y ago[deleted]