6 ms·

I have to admit I've always been puzzled by the opposition to TDD. I don't see it as a novice thing, or a dogmatic thing. Yet many in this community see it as

by BrianHV 17y ago

I have to admit I've always been puzzled by the opposition to TDD. I don't see it as a novice thing, or a dogmatic thing. Yet many in this community see it as such, so I'm wondering what they're seeing that I'm not. To me, the fundamental benefit of TDD is that it proves that the code is in the state I think it's in before I make a change, and it proves that the change I made did what I expect. I'm not aware of any other technique that does that. And I find that to be compelling enough to use the technique routinely. It seems to me the article author's basic point is that TDD alone isn't sufficient to prove all the requirements are met. I agree with that, but I also don't think that's sufficient to dismiss the practice.

damienkatz 17y ago
> To me, the fundamental benefit of TDD is that it proves that the code is in the state I think it's in before I make a change, and it proves that the change I made did what I expect. That's called unit testing, and it's gets you to the same place. It's just TTD says when you must write the unit tests: before the code. As an experienced engineer, I love unit tests for the freedom it give me later when changing things, but TDD just feels all wrong during the early phases. It prevents me from seeing the larger picture and adds "inertia" to internal APIs that maybe later turn out to be inelegant. But that's just me, others are helped greatly by TDD in their coding. So my opposition to TDD is strictly personal. It's just not for me, but feel free to do it all you like.
- BrianHV 17y ago
  How do tests written after you've changed the code prove the state of the system prior to the change? Or put another way, what technique do you use to ensure the test is exercising what you think it is? I understand what you mean about inertia. If I'm not sure what I want an API to look like, I'll sometimes mess around with it without tests, then discard the work and rewrite with tests. I suppose post-hoc testing could be equivalent, but I find I usually write the code better the second time.
  - dalke 17y ago
    I think I pointed that out in my essay when I discussed how one limitation with TDD is that tests must fail, then change the code to pass the test. This doesn't let you add tests which are expected to pass but which are used to assess the validity of the code. One way is make the tests fail, such as by changing the comparison from the expected value to something else, or by forcing an exception at the end of the code. Once the test fails then you know the test is being executed against the expected code, so then change it to make it pass. This is done without modifying the code. It's similar to what you would have to do if you depend on a third-party package which might be buggy. Code coverage is another way to check if the test is actually exercising the code paths you think it should. And yes, in some cases I will insert failures into the code itself, to convince myself that the test is also working, but that's only one of several possibilities. While you make only a single iteration when developing APIs, I find that I many more than that to get the API to feel right, and that API development needs feedback based on implementation. I'll end up with a bunch of semi-tested but reasonable code, which I then test through unit tests, guided by my experience and assisted through code coverage (either manual or automated). It is much easier to make this code work right than it would be to start from scratch. I'm also not sure that I would say "post hoc", since it really is a mixture of different types and quality of tests all throughout the development process.
    - BrianHV 17y ago
      Thanks for this. I'm going to start paying attention to times when I want to change the API but feel inhibited by the tests. I haven't noticed it happening in the past, but it's possible I dismissed the idea of changing the API quickly enough that it just flew past me.
    - joevandyk 17y ago
      When the code needs additional functionality, then yes, I'd say that TDD indicates you should create a failing test case. But if you are adding additional test cases to prove that edge cases work correctly (and no code changes need to happen), then I don't see why you'd need to have a failing test case, and I don't know how you could argue for it.
- omouse 17y ago
  * I love unit tests for the freedom it give me later* I'm curious, what freedom is that? I'm not sure I've heard anyone else say that about unit-testing.
  - silentbicycle 17y ago
    If you have through test coverage, you can cut out a sloppy-but-functional prototype and replace it with a cleaner (or carefully optimized, etc.) version, and be reasonably confident that your code will still work. It gives you more freedom to replace parts of a project, rather than getting stuck maintaining horrible legacy code. Of course, there are a few big ifs there. Writing very thorough test coverage is not a trivial undertaking, and may not be the best use of your time. Also, this says nothing whatsover about when you write the tests. This matches my experience (legacy code, ugh), and I watched a Google tech talk in which the main SQLite author, Dr. D. Richard Hipp, spoke at length about how SQLite's extensive automated tests (http://sqlite.org/testing.html http://sqlite.org/testing.html) have given them room to change the implementation of major internal systems several times.
- cabalamat 17y ago
  Writing tests before the code can have advantages. Specifically, it forces you to look at what the code's external interface is, rather than how it will be implemented. This is important, because the simpler the interface is, the less cognitive load it puts on someone using it, even if that person is yourself. But I agree that being dogmatic about requiring all code to have tests written beforehand is silly. Being dogmatic about most things is silly.
michael_dorfman 17y ago
I'm not dismissing the practice, nor am I opposed to TDD. But your experience and mine are very different if you haven't encountered TDD zealots. In fact, as much as I hate to say it, I think zealotry is the hallmark of the TDD community, such as it is.
- abalashov 17y ago
  zealotry is the hallmark of the TDD community Yes.
pvg 17y ago
To me, the fundamental benefit of TDD is that it proves that the code is in the state I think it's in before I make a change, and it proves that the change I made did what I expect. I think this is the sort of statement that makes some of us foam at the mouth and gesticulate wildly at well-intentioned TDD proponents. TDD doesn't prove anything of the sort. It proves that the code passed your tests before you made the change and passed them after - which is a very, very different thing from proving the change you made did what you expect. I'm not aware of any other technique that does that. Because there isn't one, in the non-trivial case but there are many that try to help - from static typing to design-by-contract and of course a zillion other testing methodologies. In fact, chances are very good that what you do is not strictly unit testing. And that's fine and good. TDD didn't invent testing. http://blogs.msdn.com/cashto/archive/2009/03/31/it-s-ok-not-to-write-unit-tests.aspx http://blogs.msdn.com/cashto/archive/2009/03/31/it-s-ok-not-... Another rather irritating thing is the insistence that TDD is a design methodology. If you haven't yet, read the informative case of an XP adherent trying to write a sudoku solver with TDD. Here's a blog post that links to all the relevant articles - the XP'ers attempt, Peter Norvig's solution and a reference to Coders at Work where Norvig talks about it a bit. http://ravimohan.blogspot.com/2007/04/learning-from-sudoku-solvers.html http://ravimohan.blogspot.com/2007/04/learning-from-sudoku-s... Long story short, if you don't know how to solve a problem and reason about a problem TDD won't help you. Neither will Design Patterns, UML or any particular fashionable technique, of course.
- BrianHV 17y ago
  I think perhaps in my quest for concision I failed to communicate. I certainly didn't mean to imply that TDD could provide certainty for the entire system. But I have to differ with you in one key point. TDD specifically proves that the code does not pass your test before you made the change. The red bar is, in my mind, the most important part of the practice. Given that you have a failing test, you make a change, and then you have a passing test, what more do you feel is required for proof that the change did what you expect? It's possible that "proof" is too strong a word, but I find that process gives me sufficient confidence. (Side note: I'm honestly trying to figure out the best way to write code. I don't think TDD is the silver bullet, but I think it's a useful tool in the mechanical part of development.)
Retric 17y ago
it proves that the code is in the state I think it's in before I make a change, and it proves that the change I made did what I expect This advantage is less useful the better you know the system and the better you are as a programmer. One experienced programmer I know (30+years programming) said "When I know what to test I don't need the tests." When I started programming it seemed like I spent most of my time getting the code to do what I want, but that’s morphing so I spend more time trying to understand what I need the code to do. IMO, TDD is like using a good debugger, they can be awesome tools when you have a vague idea of what's going on, but once you really understand the code base you they slowly become less useful. PS: The guy I am talking about was one of those "old masters" you occasionally hear about. One of my favorite stories about him was: "He once spent a week cleaning up someone’s code and sent it off to testing with some misgivings, which it passed and then failed in production. Apparently the testers had learned to ignore his code as it was normally a waste of their time to look at it, and after a few years this ended up biting them is the ass." So clearly testing is useful, but you need to balance cost and benefit.
jasonwatkinspdx 17y ago
"I'm not aware of any other technique that does that." Well, as pointed out by others, TDD doesn't actually prove that. There are however methods that do, which fall under the general heading of formal methods or program proof. Read "A Discipline of Programming" by Dijkstra for a reasonable introduction. The techniques work, but are laborious. Modern theorem provers can help, but it still involves considerable effort. Which implies that such methods likely won't provide a return on investment unless the cost of errors in the program is extremely high (avionics, medical control). I prefer to think of TDD as similar to working a problem backwards and forwards in elementary algebra. You may still have gotten the answer wrong, but you'll have to have made a mistake twice, so it helps your odds quite a bit.
vikram 17y ago
Though I don't have any opposition to TDD as a technique, I have an issue when some developers believe that just because the code has tests it means that it works and if the code doesn't have tests it doesn't work.

by BrianHV 17y ago

damienkatz 17y ago

> To me, the fundamental benefit of TDD is that it proves that the code is in the state I think it's in before I make a change, and it proves that the change I made did what I expect. That's called unit testing, and it's gets you to the same place. It's just TTD says when you must write the unit tests: before the code. As an experienced engineer, I love unit tests for the freedom it give me later when changing things, but TDD just feels all wrong during the early phases. It prevents me from seeing the larger picture and adds "inertia" to internal APIs that maybe later turn out to be inelegant. But that's just me, others are helped greatly by TDD in their coding. So my opposition to TDD is strictly personal. It's just not for me, but feel free to do it all you like.

BrianHV 17y ago
How do tests written after you've changed the code prove the state of the system prior to the change? Or put another way, what technique do you use to ensure the test is exercising what you think it is? I understand what you mean about inertia. If I'm not sure what I want an API to look like, I'll sometimes mess around with it without tests, then discard the work and rewrite with tests. I suppose post-hoc testing could be equivalent, but I find I usually write the code better the second time.
- dalke 17y ago
  I think I pointed that out in my essay when I discussed how one limitation with TDD is that tests must fail, then change the code to pass the test. This doesn't let you add tests which are expected to pass but which are used to assess the validity of the code. One way is make the tests fail, such as by changing the comparison from the expected value to something else, or by forcing an exception at the end of the code. Once the test fails then you know the test is being executed against the expected code, so then change it to make it pass. This is done without modifying the code. It's similar to what you would have to do if you depend on a third-party package which might be buggy. Code coverage is another way to check if the test is actually exercising the code paths you think it should. And yes, in some cases I will insert failures into the code itself, to convince myself that the test is also working, but that's only one of several possibilities. While you make only a single iteration when developing APIs, I find that I many more than that to get the API to feel right, and that API development needs feedback based on implementation. I'll end up with a bunch of semi-tested but reasonable code, which I then test through unit tests, guided by my experience and assisted through code coverage (either manual or automated). It is much easier to make this code work right than it would be to start from scratch. I'm also not sure that I would say "post hoc", since it really is a mixture of different types and quality of tests all throughout the development process.
  - BrianHV 17y ago
    Thanks for this. I'm going to start paying attention to times when I want to change the API but feel inhibited by the tests. I haven't noticed it happening in the past, but it's possible I dismissed the idea of changing the API quickly enough that it just flew past me.
  - joevandyk 17y ago
    When the code needs additional functionality, then yes, I'd say that TDD indicates you should create a failing test case. But if you are adding additional test cases to prove that edge cases work correctly (and no code changes need to happen), then I don't see why you'd need to have a failing test case, and I don't know how you could argue for it.
omouse 17y ago
* I love unit tests for the freedom it give me later* I'm curious, what freedom is that? I'm not sure I've heard anyone else say that about unit-testing.
- silentbicycle 17y ago
  If you have through test coverage, you can cut out a sloppy-but-functional prototype and replace it with a cleaner (or carefully optimized, etc.) version, and be reasonably confident that your code will still work. It gives you more freedom to replace parts of a project, rather than getting stuck maintaining horrible legacy code. Of course, there are a few big ifs there. Writing very thorough test coverage is not a trivial undertaking, and may not be the best use of your time. Also, this says nothing whatsover about when you write the tests. This matches my experience (legacy code, ugh), and I watched a Google tech talk in which the main SQLite author, Dr. D. Richard Hipp, spoke at length about how SQLite's extensive automated tests (http://sqlite.org/testing.html http://sqlite.org/testing.html) have given them room to change the implementation of major internal systems several times.
cabalamat 17y ago
Writing tests before the code can have advantages. Specifically, it forces you to look at what the code's external interface is, rather than how it will be implemented. This is important, because the simpler the interface is, the less cognitive load it puts on someone using it, even if that person is yourself. But I agree that being dogmatic about requiring all code to have tests written beforehand is silly. Being dogmatic about most things is silly.

michael_dorfman 17y ago

I'm not dismissing the practice, nor am I opposed to TDD. But your experience and mine are very different if you haven't encountered TDD zealots. In fact, as much as I hate to say it, I think zealotry is the hallmark of the TDD community, such as it is.

abalashov 17y ago
zealotry is the hallmark of the TDD community Yes.

pvg 17y ago

To me, the fundamental benefit of TDD is that it proves that the code is in the state I think it's in before I make a change, and it proves that the change I made did what I expect. I think this is the sort of statement that makes some of us foam at the mouth and gesticulate wildly at well-intentioned TDD proponents. TDD doesn't prove anything of the sort. It proves that the code passed your tests before you made the change and passed them after - which is a very, very different thing from proving the change you made did what you expect. I'm not aware of any other technique that does that. Because there isn't one, in the non-trivial case but there are many that try to help - from static typing to design-by-contract and of course a zillion other testing methodologies. In fact, chances are very good that what you do is not strictly unit testing. And that's fine and good. TDD didn't invent testing. http://blogs.msdn.com/cashto/archive/2009/03/31/it-s-ok-not-to-write-unit-tests.aspx http://blogs.msdn.com/cashto/archive/2009/03/31/it-s-ok-not-... Another rather irritating thing is the insistence that TDD is a design methodology. If you haven't yet, read the informative case of an XP adherent trying to write a sudoku solver with TDD. Here's a blog post that links to all the relevant articles - the XP'ers attempt, Peter Norvig's solution and a reference to Coders at Work where Norvig talks about it a bit. http://ravimohan.blogspot.com/2007/04/learning-from-sudoku-solvers.html http://ravimohan.blogspot.com/2007/04/learning-from-sudoku-s... Long story short, if you don't know how to solve a problem and reason about a problem TDD won't help you. Neither will Design Patterns, UML or any particular fashionable technique, of course.

BrianHV 17y ago
I think perhaps in my quest for concision I failed to communicate. I certainly didn't mean to imply that TDD could provide certainty for the entire system. But I have to differ with you in one key point. TDD specifically proves that the code does not pass your test before you made the change. The red bar is, in my mind, the most important part of the practice. Given that you have a failing test, you make a change, and then you have a passing test, what more do you feel is required for proof that the change did what you expect? It's possible that "proof" is too strong a word, but I find that process gives me sufficient confidence. (Side note: I'm honestly trying to figure out the best way to write code. I don't think TDD is the silver bullet, but I think it's a useful tool in the mechanical part of development.)

Retric 17y ago

it proves that the code is in the state I think it's in before I make a change, and it proves that the change I made did what I expect This advantage is less useful the better you know the system and the better you are as a programmer. One experienced programmer I know (30+years programming) said "When I know what to test I don't need the tests." When I started programming it seemed like I spent most of my time getting the code to do what I want, but that’s morphing so I spend more time trying to understand what I need the code to do. IMO, TDD is like using a good debugger, they can be awesome tools when you have a vague idea of what's going on, but once you really understand the code base you they slowly become less useful. PS: The guy I am talking about was one of those "old masters" you occasionally hear about. One of my favorite stories about him was: "He once spent a week cleaning up someone’s code and sent it off to testing with some misgivings, which it passed and then failed in production. Apparently the testers had learned to ignore his code as it was normally a waste of their time to look at it, and after a few years this ended up biting them is the ass." So clearly testing is useful, but you need to balance cost and benefit.

jasonwatkinspdx 17y ago

"I'm not aware of any other technique that does that." Well, as pointed out by others, TDD doesn't actually prove that. There are however methods that do, which fall under the general heading of formal methods or program proof. Read "A Discipline of Programming" by Dijkstra for a reasonable introduction. The techniques work, but are laborious. Modern theorem provers can help, but it still involves considerable effort. Which implies that such methods likely won't provide a return on investment unless the cost of errors in the program is extremely high (avionics, medical control). I prefer to think of TDD as similar to working a problem backwards and forwards in elementary algebra. You may still have gotten the answer wrong, but you'll have to have made a mistake twice, so it helps your odds quite a bit.

vikram 17y ago

Though I don't have any opposition to TDD as a technique, I have an issue when some developers believe that just because the code has tests it means that it works and if the code doesn't have tests it doesn't work.