5 ms·

If you don't understand the problem you're trying to solve well enough to write tests, you shouldn't start coding. Sometimes there are problems that are only r

If you don't understand the problem you're trying to solve well enough to write tests, you shouldn't start coding. Sometimes there are problems that are only really possible to appreciate after one starts coding. In many instances, I've found that attempting to build a prototype is the best way to understand the problem. In those instances, TDD is a huge mistake. It ties you to architectural decisions that you know you're going to change when you better understand the problem. Contrast Ron Jeffries' attempt to build a sudoku solver using TDD with Peter Norvig's attempt [1]. [1]: http://ravimohan.blogspot.com/2007/04/learning-from-sudoku-solvers.html http://ravimohan.blogspot.com/2007/04/learning-from-sudoku-s...

voodootrucker 8y ago
Peter Norvig's approach seems like the textbook way to TDD: 1. Find a large body of known good input/output pairs 2. create tests around that known good data 3. create an implementation that produces the desired output from the given input There's no reason Peter couldn't or shouldn't have started with the tests.
- quanticle 8y ago
  What you're describing is closer to what Ron Jeffries attempted than to what Peter Norvig did. It's Jeffries who started with input and output and then faffed about for five blog posts while attempting to figure out a way to get input to match output. What Norvig did was start by thinking about the representation of the problem and the constraints that the problem imposed. He then found a way to elegantly represent those constraints in code. And, only then did he write tests. Moreover, the process you describe is quite similar to the parody of "how to draw an owl" [1]. It completely glosses over the fact that "create an implementation that produces the desired output from the given input" is where 99.5% of the complexity lies. [1]: https://i.kym-cdn.com/photos/images/newsfeed/000/572/078/d6d.jpg https://i.kym-cdn.com/photos/images/newsfeed/000/572/078/d6d...
jacques_chester 8y ago
That a practice is not universally valuable is different from whether a practice is universally worthless. TDD has a sweet spot and by itself is not enough. "Reinventing computer science, one test at a time" is a joke I've heard at Pivotal about Pivotal. And it's sometimes true. There is a lot of valuable work in the TDD sweetspot. There is also a lot of work to be done on a whiteboard. Neither is a perfect substitute for the other. You profit from having both.
- quanticle 8y ago
  I'm not the one arguing that the practice is universally valuable. I'm responding to the grandparent's assertion that if you don't understand the problem well enough to write tests, you don't understand the problem well enough to write code. I'm suggesting that that's not always the case: oftentimes when encountering a new problem domain, it's easier to hammer out a crappy solution and watch it fail than it is to deliberately think through test case after test case. Moreover, writing test cases, in my experience, is often a form of fake productivity. It feels good. You're writing code. Hundreds and hundreds of lines of code. But you're not really making any headway on solving the problem.
  - jacques_chester 8y ago
    Sure, I think this is reasonable. Spiking and prototypes are legitimate. I've done both and I expect I'll do both again. I don't necessarily agree about the fake productivity. I don't think of SLOC as the metric of success. In fact, I rarely think about SLOC at all. What matters to me is proving to my own satisfaction that the software works, now and in the future.
    - quanticle 8y ago
      I think we agree more than we disagree. I'm totally in favor of having a comprehensive test suite that ensures that further changes to the application won't break already built features. I'm just of the opinion that unless you already know pretty much what the solution is going to look like it's not a good idea to build the test suite for the solution before you've built the solution itself. Building the test suite first locks you in to a particular way of organizing your code, and changing that organization is painful, because every time one of the externally facing APIs to one of your modules changes, you also end up having to change a bunch of tests. TDD works well for CRUD applications because there really isn't much new that's going on there. You're reading some data, validating it, manipulating it according to a fixed set of rules that you get from business analysis and saving it to a database. It's fairly straightforward to put tests around each of those stages. However, if you're making a program to solve a problem which is novel to you, it makes more sense to write the tests after you have a solution, since it's likely that your first few attempts to solve the problem will have to be heavily refactored before you have something that works in the way that you want.
mikekchar 8y ago
There are classes of problems for which TDD is quite difficult. My favourite example is fizz buzz. Asking someone to TDD fizz buzz in a job interview is cruel and unusual punishment because it is super hard to do. The reason it's hard for fizz buzz is because it is an infinite sequence. You end up with this choice of either testing some random examples, which may or may not help you with the actual problem (i.e., we can test fizz buzz of 1, 3, 5, and 15, but have we actually said anything about the other ones), or we end up writing tests that are exactly the same as the production code. A sudoku solver falls into this same category because constraints on the rules are global and the number of combinations are near infinite. So you either write some mostly useless smoke tests or you figure out all the constraints and compare the production code to the test code using the same implementation -- which is useless. This doesn't mean it is impossible to test drive these kinds of problems. It's not. But you have to take a very different approach than you would with a more normal problem. Jeffries's mistake was in not realising that his approach would not work. It is entirely possible that he did not realise that there were some problems for which this approach would not work until he tried it. While these kinds of problems show up frequently in some domains, then don't show up at all in others. He may have never run into it before. I highly recommend taking a look at this talk by Kevlin Henney (the TDD discussion starts at 29:15 but the whole talk is worth watching. If you are really pressed for time, his approach for testing fizz buzz starts at 37:40). https://www.youtube.com/watch?v=_M4o0ExLQCs https://www.youtube.com/watch?v=_M4o0ExLQCs I think the thing that's important to realise is that TDD is an approach that has benefits. The benefit of "I can turn my mind off and suddenly the answer will appear" is not one of the benefits. I don't think TDD makes coding easier per se. In fact, I find that junior developers have a lot harder time doing TDD than not doing TDD. It's an added constraint, so it shouldn't be surprising that it's harder. The question is whether or not the constraint leads to other benefits. One of the things I will quibble with you a bit is on whether or not TDD ties you into architectural decision. I do not agree that it should, but I do agree that a very large number of people who believe they are doing good TDD are doing exactly what you say. I have even seen people go off TDD for exactly this reason. However, I will argue that if you see this problem, it's the first clue that you are not doing TDD well. Should you avoid TDD if you aren't doing it well? That's a good question.

voodootrucker 8y ago

Peter Norvig's approach seems like the textbook way to TDD: 1. Find a large body of known good input/output pairs 2. create tests around that known good data 3. create an implementation that produces the desired output from the given input There's no reason Peter couldn't or shouldn't have started with the tests.

quanticle 8y ago
What you're describing is closer to what Ron Jeffries attempted than to what Peter Norvig did. It's Jeffries who started with input and output and then faffed about for five blog posts while attempting to figure out a way to get input to match output. What Norvig did was start by thinking about the representation of the problem and the constraints that the problem imposed. He then found a way to elegantly represent those constraints in code. And, only then did he write tests. Moreover, the process you describe is quite similar to the parody of "how to draw an owl" [1]. It completely glosses over the fact that "create an implementation that produces the desired output from the given input" is where 99.5% of the complexity lies. [1]: https://i.kym-cdn.com/photos/images/newsfeed/000/572/078/d6d.jpg https://i.kym-cdn.com/photos/images/newsfeed/000/572/078/d6d...

jacques_chester 8y ago

That a practice is not universally valuable is different from whether a practice is universally worthless. TDD has a sweet spot and by itself is not enough. "Reinventing computer science, one test at a time" is a joke I've heard at Pivotal about Pivotal. And it's sometimes true. There is a lot of valuable work in the TDD sweetspot. There is also a lot of work to be done on a whiteboard. Neither is a perfect substitute for the other. You profit from having both.

quanticle 8y ago
I'm not the one arguing that the practice is universally valuable. I'm responding to the grandparent's assertion that if you don't understand the problem well enough to write tests, you don't understand the problem well enough to write code. I'm suggesting that that's not always the case: oftentimes when encountering a new problem domain, it's easier to hammer out a crappy solution and watch it fail than it is to deliberately think through test case after test case. Moreover, writing test cases, in my experience, is often a form of fake productivity. It feels good. You're writing code. Hundreds and hundreds of lines of code. But you're not really making any headway on solving the problem.
- jacques_chester 8y ago
  Sure, I think this is reasonable. Spiking and prototypes are legitimate. I've done both and I expect I'll do both again. I don't necessarily agree about the fake productivity. I don't think of SLOC as the metric of success. In fact, I rarely think about SLOC at all. What matters to me is proving to my own satisfaction that the software works, now and in the future.
  - quanticle 8y ago
    I think we agree more than we disagree. I'm totally in favor of having a comprehensive test suite that ensures that further changes to the application won't break already built features. I'm just of the opinion that unless you already know pretty much what the solution is going to look like it's not a good idea to build the test suite for the solution before you've built the solution itself. Building the test suite first locks you in to a particular way of organizing your code, and changing that organization is painful, because every time one of the externally facing APIs to one of your modules changes, you also end up having to change a bunch of tests. TDD works well for CRUD applications because there really isn't much new that's going on there. You're reading some data, validating it, manipulating it according to a fixed set of rules that you get from business analysis and saving it to a database. It's fairly straightforward to put tests around each of those stages. However, if you're making a program to solve a problem which is novel to you, it makes more sense to write the tests after you have a solution, since it's likely that your first few attempts to solve the problem will have to be heavily refactored before you have something that works in the way that you want.

mikekchar 8y ago

There are classes of problems for which TDD is quite difficult. My favourite example is fizz buzz. Asking someone to TDD fizz buzz in a job interview is cruel and unusual punishment because it is super hard to do. The reason it's hard for fizz buzz is because it is an infinite sequence. You end up with this choice of either testing some random examples, which may or may not help you with the actual problem (i.e., we can test fizz buzz of 1, 3, 5, and 15, but have we actually said anything about the other ones), or we end up writing tests that are exactly the same as the production code. A sudoku solver falls into this same category because constraints on the rules are global and the number of combinations are near infinite. So you either write some mostly useless smoke tests or you figure out all the constraints and compare the production code to the test code using the same implementation -- which is useless. This doesn't mean it is impossible to test drive these kinds of problems. It's not. But you have to take a very different approach than you would with a more normal problem. Jeffries's mistake was in not realising that his approach would not work. It is entirely possible that he did not realise that there were some problems for which this approach would not work until he tried it. While these kinds of problems show up frequently in some domains, then don't show up at all in others. He may have never run into it before. I highly recommend taking a look at this talk by Kevlin Henney (the TDD discussion starts at 29:15 but the whole talk is worth watching. If you are really pressed for time, his approach for testing fizz buzz starts at 37:40). https://www.youtube.com/watch?v=_M4o0ExLQCs https://www.youtube.com/watch?v=_M4o0ExLQCs I think the thing that's important to realise is that TDD is an approach that has benefits. The benefit of "I can turn my mind off and suddenly the answer will appear" is not one of the benefits. I don't think TDD makes coding easier per se. In fact, I find that junior developers have a lot harder time doing TDD than not doing TDD. It's an added constraint, so it shouldn't be surprising that it's harder. The question is whether or not the constraint leads to other benefits. One of the things I will quibble with you a bit is on whether or not TDD ties you into architectural decision. I do not agree that it should, but I do agree that a very large number of people who believe they are doing good TDD are doing exactly what you say. I have even seen people go off TDD for exactly this reason. However, I will argue that if you see this problem, it's the first clue that you are not doing TDD well. Should you avoid TDD if you aren't doing it well? That's a good question.