4 ms·

dalke, thank you so much for a such detailed feedback. I really like it. And I kind of agree with a lot of points. I will think how I can cover them in the near

dalke, thank you so much for a such detailed feedback. I really like it. And I kind of agree with a lot of points. I will think how I can cover them in the near future! > One of my issues with TDD is the 'refactor' step, which is supposed to take place during green. One of the refactor transformations is "Substitute Algorithm." I've always thought that was a huge loophole as the appropriate tests for a new algorithm can be quite different than the tests for the existing algorithm, but TDD says not to worry about that. I think in this case, I would write new test suite for new algorithm, and then replace both old test suite and old implementation in one step. In case, when tests suite can be shared, it makes sense to re-use it. > That issue is applicable here, as #3 really is a "Substitute Algorithm" refactor of #2. It could have started with the previous sort test cases, and shown how refactor the algorithm in the TDD framework rather than redo the code from scratch. So far I haven't actually seen any of the TDD example walkthroughs do it, at least with a non-trivial example like sorting, so showing this might also be enlightening or at least novel. I would like to research if that is possible. If it is, I will make an episode for that. > A test case goal should be to convince people that the code works. The test cases here aren't enough. For example, there is no test for when values are equal, and I don't recall the spec saying duplicate values were disallowed. This is important because some broken sort implementations might get into a loop for this case. Thank you for catching this, I wasn't focusing enough on this point. > If this were implemented in Python then there's a maximum recursion depth. I think the worst case for this quicksort implementation is if the input is sorted. So I would want to try sorting 100,000 sorted to feel more comfortable. This sort of error analysis must be part of TDD, but it doesn't really fit into the red/green test cycle because it's a test you write when you expect the code to pass, not fail. This can, actually, fit in red/green cycle. By writing a test that creates a worst case scenario of 100,000 sorted elements. And see it fail with stack overflow. And then fix it by randomizing element that is used for partitioning. WDYT? > Also, from an API perspective, if the input is the empty list then this code returns the original list, otherwise it returns a new list. I would prefer to always get a new list, so I have a better idea of ownership. Thanks for pointing this too. Next time I would add specific tests for that. In that case, it is reasonable to check it for empty, one-element and N-element array. --- Now that I think about that, I will make an episode #4 or #5 a re-visit of #3 - with all these problems pointed out and fixed ;)

dalke 11y ago
I'm relieved you didn't consider my comments to be harsh. It's tough to put together a screeencast, much less a series. You have my respect for doing so, and for choosing a simple but non-trivial problem like sorting that people use every day, instead of something like Fibonacci. > I think in this case, I would write new test suite I think that's reasonable. I bring this up to point out how the TDD formalism, as red-green-refactor, doesn't work. This means the "refactor" in TDD uses a subset of the available refactoring operations, which further means TDD cannot be the only thing which drives the design. (My usual example is going from naive string search to Knuth–Morris–Pratt. I think in the future I'll talking about moving from classic quicksort to Timsort. I've also used this to demonstrate that TDD is a form of white-box testing.) > I wasn't focusing enough on this point. One of my complaints about TDD is that it focuses on tests for the next small code transformation. This is very helpful for a lot of people! But maintaining that focus means that other aspects receive less attention. > And see it fail with stack overflow But what if it doesn't fail? Do you leave the test in? Or do you tweak the parameters until if fails? What if a recursive implementation never fails, perhaps because the compiler used tail recursion? How many non-failing tests do you add? Test case analysis is part of the traditional QA process, and I feel that too many TDD people reject it because they see after-the-fact tests as being counter to the TDD philosophy.
- waterlink 11y ago
  > It's tough to put together a screeencast, much less a series. You have my respect for doing so, and for choosing a simple but non-trivial problem like sorting that people use every day, instead of something like Fibonacci. Thanks! > > I think in this case, I would write new test suite > I think that's reasonable. I bring this up to point out how the TDD formalism, as red-green-refactor, doesn't work. This means the "refactor" in TDD uses a subset of the available refactoring operations, which further means TDD cannot be the only thing which drives the design. I believe, that is always the case with any best practice. You hit a point, where the problem you are facing requires something else, or combination of techniques, some analysis and thought. Reminds me about quadrants with "Known", "Knowable", "Complex" and "Chaotic". > .. TDD is a form of white-box testing .. I kind of agree on this one. In a lot of cases it is true. > > I wasn't focusing enough on this point. > One of my complaints about TDD is that it focuses on tests for the next small code transformation. This is very helpful for a lot of people! But maintaining that focus means that other aspects receive less attention. And that is exactly my intention: I want to help TDD spreading to the people (at least, the ones I have in reach here, in Berlin), who are not doing testing at all. (or doing it only from time to time). I want to infect them with a concept, and afterwards they will be on their own journey of discovery, what fits their tastes and their problems best; but they will no longer be able to write low-test-covered code. Do you think it is a good idea? Or maybe can be implemented in a better way? > > And see it fail with stack overflow > But what if it doesn't fail? Do you leave the test in? Or do you tweak the parameters until if fails? What if a recursive implementation never fails, perhaps because the compiler used tail recursion? How many non-failing tests do you add? I was playing around with that today. I have actually added (and left in) not only stack-overflow test, but performance test. I was increasing parameters, until reasonable, like 3 * 10 ^ 7 elements, which still can complete in reasonable time for running inside of the test suite. > Test case analysis is part of the traditional QA process, and I feel that too many TDD people reject it because they see after-the-fact tests as being counter to the TDD philosophy. I think after-the-fact tests added after you already have decent coverage is a good thing to do, I usually do that, especially in times, when the solution emerged too fast, and you didn't covered some edge cases; how you got to this coverage is a different question, be it a TDD, property-based testing, after-the-fact tests, all-tests-first, and/or anything else. After-the-fact tests help you validate your assumptions about your code you have just written, which allows to spot problems earlier and make code future-proof. My opinion on TDD philosophy is that it is something that doesn't drive complete design of the thing under implementation (nor it designed to do so, programmers are the ones who should do that), but it does force you to design it in a unit-testable fashion. Which makes adding after-the-fact tests really simple and enjoyable.

dalke 11y ago

I'm relieved you didn't consider my comments to be harsh. It's tough to put together a screeencast, much less a series. You have my respect for doing so, and for choosing a simple but non-trivial problem like sorting that people use every day, instead of something like Fibonacci. > I think in this case, I would write new test suite I think that's reasonable. I bring this up to point out how the TDD formalism, as red-green-refactor, doesn't work. This means the "refactor" in TDD uses a subset of the available refactoring operations, which further means TDD cannot be the only thing which drives the design. (My usual example is going from naive string search to Knuth–Morris–Pratt. I think in the future I'll talking about moving from classic quicksort to Timsort. I've also used this to demonstrate that TDD is a form of white-box testing.) > I wasn't focusing enough on this point. One of my complaints about TDD is that it focuses on tests for the next small code transformation. This is very helpful for a lot of people! But maintaining that focus means that other aspects receive less attention. > And see it fail with stack overflow But what if it doesn't fail? Do you leave the test in? Or do you tweak the parameters until if fails? What if a recursive implementation never fails, perhaps because the compiler used tail recursion? How many non-failing tests do you add? Test case analysis is part of the traditional QA process, and I feel that too many TDD people reject it because they see after-the-fact tests as being counter to the TDD philosophy.

waterlink 11y ago
> It's tough to put together a screeencast, much less a series. You have my respect for doing so, and for choosing a simple but non-trivial problem like sorting that people use every day, instead of something like Fibonacci. Thanks! > > I think in this case, I would write new test suite > I think that's reasonable. I bring this up to point out how the TDD formalism, as red-green-refactor, doesn't work. This means the "refactor" in TDD uses a subset of the available refactoring operations, which further means TDD cannot be the only thing which drives the design. I believe, that is always the case with any best practice. You hit a point, where the problem you are facing requires something else, or combination of techniques, some analysis and thought. Reminds me about quadrants with "Known", "Knowable", "Complex" and "Chaotic". > .. TDD is a form of white-box testing .. I kind of agree on this one. In a lot of cases it is true. > > I wasn't focusing enough on this point. > One of my complaints about TDD is that it focuses on tests for the next small code transformation. This is very helpful for a lot of people! But maintaining that focus means that other aspects receive less attention. And that is exactly my intention: I want to help TDD spreading to the people (at least, the ones I have in reach here, in Berlin), who are not doing testing at all. (or doing it only from time to time). I want to infect them with a concept, and afterwards they will be on their own journey of discovery, what fits their tastes and their problems best; but they will no longer be able to write low-test-covered code. Do you think it is a good idea? Or maybe can be implemented in a better way? > > And see it fail with stack overflow > But what if it doesn't fail? Do you leave the test in? Or do you tweak the parameters until if fails? What if a recursive implementation never fails, perhaps because the compiler used tail recursion? How many non-failing tests do you add? I was playing around with that today. I have actually added (and left in) not only stack-overflow test, but performance test. I was increasing parameters, until reasonable, like 3 * 10 ^ 7 elements, which still can complete in reasonable time for running inside of the test suite. > Test case analysis is part of the traditional QA process, and I feel that too many TDD people reject it because they see after-the-fact tests as being counter to the TDD philosophy. I think after-the-fact tests added after you already have decent coverage is a good thing to do, I usually do that, especially in times, when the solution emerged too fast, and you didn't covered some edge cases; how you got to this coverage is a different question, be it a TDD, property-based testing, after-the-fact tests, all-tests-first, and/or anything else. After-the-fact tests help you validate your assumptions about your code you have just written, which allows to spot problems earlier and make code future-proof. My opinion on TDD philosophy is that it is something that doesn't drive complete design of the thing under implementation (nor it designed to do so, programmers are the ones who should do that), but it does force you to design it in a unit-testable fashion. Which makes adding after-the-fact tests really simple and enjoyable.