3 ms·

"Do you think it is a good idea?" Well, I've tried TDD several times myself, and as part of our local Python User's Group kata. I find it to be a frustrating t

by dalke 11y ago

"Do you think it is a good idea?" Well, I've tried TDD several times myself, and as part of our local Python User's Group kata. I find it to be a frustrating technique, but I can't deny there are many people who find it useful. I've watched a handful of video walkthroughs of TDD, and gotten frustrated at the lack of good testing or development in them. One example is Robert Martin's primes kata. His solution works, but realistically only for small numbers. He never specified an input domain, and during development never tried to identify ambiguities between the API 'promise' that the function would take int32, the expectation by most that it's fast to factor 2,147,483,647 (2^31-1), and the reality that his code would take a long time for that. You write "I think after-the-fact tests added after you already have decent coverage". I love code coverage as a way to identify missing tests. For me that's usually in error handling. Many of the TDD writings, going back to Kent Beck, say that TDD will naturally lead to high code coverage. I think that's wrong. I again thing many of the TDD writings ignore the problems that can come up during refactoring. I think Martin's TDD style works because the problem he uses are "simple" in an algorithm sense, which is why he can talk about a 'Transformation Priority Premise'. Martin says "Refactorings are simple operations that change the structure of code without changing its behavior", but as I wrote before, 'substitute algorithm' is not a simple operation. Then again, I think most businesses only need simple algorithms, even if the overall goal is quite complex. My complaint is about where TDD isn't enough. So if you also include coverage and after-the-fact tests, that would be far beyond what I've seen so far in TDD walkthroughs. That said, I don't think many people will watch a code walkthrough, so don't expect much fame or uptake from your series. That said, yes, I think it's a good idea. "but it does force you to design it in a unit-testable fashion" You might want to read http://www.rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf http://www.rbcs-us.com/documents/Why-Most-Unit-Testing-is-Wa... . There's a follow-up as well. He argues that most time should be spent on integration testing, and I agree. I think unit tests implicitly enforce an API on the code, even when there's no need for an API. It's a special purpose internal API designed for testing. Because they are internal functions, you should have the freedom to refactor them, but the unit test burden keeps you from doing so. (In one TDD kata we did, we got working code, then compared it to others who were there. Our code was several times larger because we had tests for multiple code layers, while they only had one layer. We could have removed the layers, but that would have called for rewriting most of the unit tests, and I think that inhibited our design sense.) In my own code, most of my tests are base on a unit-test framework but I almost only use the public API (for the library code) or a call to main() (for the command-line code). I feel this gives me more freedom to refactor without worrying about the extra work to change the unittests. Anyway, Coplien describes it much better than I, and my comments just now don't even apply to a "sort()" function as that would be a public API.

waterlink 11y ago
> His solution works, but realistically only for small numbers. He never specified an input domain, and during development never tried to identify ambiguities between the API 'promise' that the function would take int32, the expectation by most that it's fast to factor 2,147,483,647 (2^31-1), and the reality that his code would take a long time for that. I think in one of the videos I watched, he explicitly said, that this is lame implementation and can be optimized (for example by looping until square root of the number). The interesting point, is that one wouldn't have to change tests for that. One would want to write a benchmark for that, to use instead of test. As in: normal tests are tests for logic, while benchmarks are tests for performance. Additionally, it is always true, that premature optimization is not a good idea. If you have an evidence, that current algorithm is not fast enough (knowledge of your dataset sizes + understanding on what would the reasonable speed be + benchmark to verify all of that), then you optimize, without behavior change. > I think Martin's TDD style works because the problem he uses are "simple" in an algorithm sense, which is why he can talk about a 'Transformation Priority Premise'. Martin says "Refactorings are simple operations that change the structure of code without changing its behavior", but as I wrote before, 'substitute algorithm' is not a simple operation. Funny thing I have found about 'SubstituteAlgorithm' refactoring, is its example here: http://refactoring.com/catalog/substituteAlgorithm.html http://refactoring.com/catalog/substituteAlgorithm.html - it is not changing algorithm, it simply eliminates 'if' statements. That is something that I usually do naturally, and don't even consider underlying algorithm to be changed... So maybe, Martin Fowler, when coining this term, meant something different? > So if you also include coverage and after-the-fact tests, that would be far beyond what I've seen so far in TDD walkthroughs. It is definitely on my mind. Since that is what I usually do. It will be hard to demonstrate code coverage in Crystal, though, since I will have to write a library for that (there is none at the moment); so I might as well go for showing test coverage use in a different language for a change (like Clojure or Ruby). After the fact tests are definitely important too. > That said, I don't think many people will watch a code walkthrough, so don't expect much fame or uptake from your series. I know about that, that is why I am not only doing screencasts, but additionally, I am generally blogging, speaking at meetups (and hopefully in the future on conferences), simply speak to people and debate with them (I am ENTP after all), pair-program with random people, and I have in mind joining as a teacher/mentor one of the organization we have here in Berlin to teach beginners. > I think unit tests implicitly enforce an API on the code, even when there's no need for an API. It's a special purpose internal API designed for testing. Because they are internal functions, you should have the freedom to refactor them, but the unit test burden keeps you from doing so. "special purpose internal API designed for testing" is actually a smell to my knowledge. This emerges, when one tries to unit-test something that is not decoupled enough, or when one has too big objects/functions, read: violating one or many points of SOLID. That is why there is such a phrase going around: "Other word for 'testable' is 'decoupled'". I did see this happening a lot of times. Even I had to do such weird test-only APIs myself, when I had less experience. And yes, it was weird and painful to test. Now that I analyze my knowledge: indeed, the problem was is that the object/function under test was initially badly designed. If I were to go back to the past, I would tell myself: "You see here one method. I see here 4 objects.". Currently, I use that as an indicator - whenever I need to know something "intimate" about function or object, so that I need special API; means, that function or object under test is, most probably, should be split; and dependency injection should be used to combine the parts. This works both in OO and FP world for me: it is just a question of combining independent objects or independent functions together. Now that I think about it, all my objects/functions in the professional code I write tend to be public-API-ish. I never test private methods/functions. But I have to say, that my objects usually have only one or two public methods (and they are extremely small). > You might want to read http://www.rbcs-us.com/documents/Why-Most-Unit-Testing-is-Wa.. http://www.rbcs-us.com/documents/Why-Most-Unit-Testing-is-Wa.... . There's a follow-up as well. He argues that most time should be spent on integration testing, and I agree. Notion of Unit vs Integration goes on as long as I remember. And from one year to another, someone always coins "Unit tests are dead, long live Integration tests"; another year and someone says: "Integration tests are dead, long live Unit tests". It flips over and over again. What I think, is that both are important. Which is more important, depends on lots of factors. I am biased towards Unit testing, since I like being able to do some actual productive work on the airplane on a low-energy battery in my laptop, if I have to, and it will just simply work, because it is very well-tested. That is why, my code typically has 90-95% of unit tests and 5-10% for everything else. --- Thanks for such a conversation, you have really provided a lot of interesting insights for me! That is quite rare to find someone, who would do that.

by dalke 11y ago

waterlink 11y ago

> His solution works, but realistically only for small numbers. He never specified an input domain, and during development never tried to identify ambiguities between the API 'promise' that the function would take int32, the expectation by most that it's fast to factor 2,147,483,647 (2^31-1), and the reality that his code would take a long time for that. I think in one of the videos I watched, he explicitly said, that this is lame implementation and can be optimized (for example by looping until square root of the number). The interesting point, is that one wouldn't have to change tests for that. One would want to write a benchmark for that, to use instead of test. As in: normal tests are tests for logic, while benchmarks are tests for performance. Additionally, it is always true, that premature optimization is not a good idea. If you have an evidence, that current algorithm is not fast enough (knowledge of your dataset sizes + understanding on what would the reasonable speed be + benchmark to verify all of that), then you optimize, without behavior change. > I think Martin's TDD style works because the problem he uses are "simple" in an algorithm sense, which is why he can talk about a 'Transformation Priority Premise'. Martin says "Refactorings are simple operations that change the structure of code without changing its behavior", but as I wrote before, 'substitute algorithm' is not a simple operation. Funny thing I have found about 'SubstituteAlgorithm' refactoring, is its example here: http://refactoring.com/catalog/substituteAlgorithm.html http://refactoring.com/catalog/substituteAlgorithm.html - it is not changing algorithm, it simply eliminates 'if' statements. That is something that I usually do naturally, and don't even consider underlying algorithm to be changed... So maybe, Martin Fowler, when coining this term, meant something different? > So if you also include coverage and after-the-fact tests, that would be far beyond what I've seen so far in TDD walkthroughs. It is definitely on my mind. Since that is what I usually do. It will be hard to demonstrate code coverage in Crystal, though, since I will have to write a library for that (there is none at the moment); so I might as well go for showing test coverage use in a different language for a change (like Clojure or Ruby). After the fact tests are definitely important too. > That said, I don't think many people will watch a code walkthrough, so don't expect much fame or uptake from your series. I know about that, that is why I am not only doing screencasts, but additionally, I am generally blogging, speaking at meetups (and hopefully in the future on conferences), simply speak to people and debate with them (I am ENTP after all), pair-program with random people, and I have in mind joining as a teacher/mentor one of the organization we have here in Berlin to teach beginners. > I think unit tests implicitly enforce an API on the code, even when there's no need for an API. It's a special purpose internal API designed for testing. Because they are internal functions, you should have the freedom to refactor them, but the unit test burden keeps you from doing so. "special purpose internal API designed for testing" is actually a smell to my knowledge. This emerges, when one tries to unit-test something that is not decoupled enough, or when one has too big objects/functions, read: violating one or many points of SOLID. That is why there is such a phrase going around: "Other word for 'testable' is 'decoupled'". I did see this happening a lot of times. Even I had to do such weird test-only APIs myself, when I had less experience. And yes, it was weird and painful to test. Now that I analyze my knowledge: indeed, the problem was is that the object/function under test was initially badly designed. If I were to go back to the past, I would tell myself: "You see here one method. I see here 4 objects.". Currently, I use that as an indicator - whenever I need to know something "intimate" about function or object, so that I need special API; means, that function or object under test is, most probably, should be split; and dependency injection should be used to combine the parts. This works both in OO and FP world for me: it is just a question of combining independent objects or independent functions together. Now that I think about it, all my objects/functions in the professional code I write tend to be public-API-ish. I never test private methods/functions. But I have to say, that my objects usually have only one or two public methods (and they are extremely small). > You might want to read http://www.rbcs-us.com/documents/Why-Most-Unit-Testing-is-Wa.. http://www.rbcs-us.com/documents/Why-Most-Unit-Testing-is-Wa.... . There's a follow-up as well. He argues that most time should be spent on integration testing, and I agree. Notion of Unit vs Integration goes on as long as I remember. And from one year to another, someone always coins "Unit tests are dead, long live Integration tests"; another year and someone says: "Integration tests are dead, long live Unit tests". It flips over and over again. What I think, is that both are important. Which is more important, depends on lots of factors. I am biased towards Unit testing, since I like being able to do some actual productive work on the airplane on a low-energy battery in my laptop, if I have to, and it will just simply work, because it is very well-tested. That is why, my code typically has 90-95% of unit tests and 5-10% for everything else. --- Thanks for such a conversation, you have really provided a lot of interesting insights for me! That is quite rare to find someone, who would do that.