10 ms·

Unit testing in Coders at Work

109 points by mqt 17y ago

jrockway 17y ago
I sit somewhere in the middle. Unit testing GUI code is going to slow you down. Unit testing your core algorithms is essential. That is the stuff that must never break, and where breakages would not be immediately detectable. The unit tests ensure that you are always sure that that code works. (Real world example; testing somewhat hairy parsing code: http://github.com/jrockway/moosex-runnable/blob/master/t/arg-parser.t http://github.com/jrockway/moosex-runnable/blob/master/t/arg... I test every case I could come up with, so I could be sure that it works. Better to spend the time typing this file than to be hit with a weird misparse on the command-line, where you are probably not in the mood to fix the parser.) On top of the unit tests, I do integration tests, to make sure sections of my app mostly work together. This is somewhere below "Request -> Response" and somewhere above the individual routines/classes. In the web app case, this is mostly "instantiate an object graph; call the methods that provide data to the rendering functions; make sure the results make sense". (I assume my template renderer and object database work; those have their own test suite, after all.) I also test stuff like typical sequences of actions that share some state (usually "the session"). I neglected that once and it caused me lots of problems. But anyway, GUI testing == waste of time. As long as the buttons you click generate the right events, the GUI works. This is the sort of app Zawinski was working on, and this colors his thoughts on testing accordingly. Your mileage may vary.
- brown9-2 17y ago
  Why exactly is unit testing of GUI components a waste of time?
  - anonjon 17y ago
    They aren't safety critical and you can test them by using the GUI.
  - jrockway 17y ago
    Because it takes a lot of time and effort but does not provide much benefit.
    - jamesbritt 17y ago
      OK, but why? Is it not useful to know that various operations and navigations through the GUI still work as expected? And if it's a matter of a poor benefit/effort ratio, why are GUIs so hard to test? Tools like Swinger are pretty easy to get rolling with for testing Swing UIs. Where do these things break down? Are these things worth fixing?
      - akeefer 17y ago
        They break down because UI testing tends to rely on Strings: either labels for controls or embedded ids of them. It's really, really easy to have those change on you, and when they do the tests become difficult to debug: if the button with the id "Search" isn't found, is it because something broke to cause the button not to show up, because the id has changed to "SearchUsers," or what? And if I need to remove the "Search" button and replace it with somethine else, how do I figure out which tests to change? The high-level nature of the tests inherently makes them much harder to debug, because the test could have broken for any of a hundred reasons. In other words, with unit tests the linkage between the code being tested and the test tends to be pretty tight; with UI tests it tends to be very, very loose, and that makes the tests correspondingly more fragile and much harder to debug. We essentially do typesafe metaprogramming on our web UI that generates compile-time-checked constants for all the labels, buttons, etc. so that our tests don't compile if the UI changes, which has gone a long way to keeping the tests stable; it's the best solution we've come up with, but it's a huge investment, and our attempts to do testing of our Swing client have met with less success so far.
      - jrockway 17y ago
        In my experience, I have sunk a lot of time making sure that foo div has bar css class when the quux link is clicked... but it has never saved me much time. I have to click through the site regularly (content changes, "does this work in IE", etc.) anyway, and errors are usually noticeable immediately. Other problems I've noticed are that they are either so specific that they fail for no reason (it was supposed to be red, not green!) or too general that they don't catch issues that would annoy users. If I could have these tests for free, I'd take 'em. But since they're expensive and don't get me much, I don't bother. (BTW, if you have complicated algorithms in your JavaScript; refactor so you can test them with Rhino on the command-line. Don't do this stuff in the browser!) Anyway, I would be interested in hearing your UI testing success stories.
  - chrisconley 17y ago
    Maybe a better way to put it is that testing GUI components is not a complete waste of time, but it gives you a lot less bang for the buck.
silentbicycle 17y ago
The part about TDD in the interview with Peter Norvig really jumped out at me, too. A passing test doesn't always mean anything important has changed - like most other programming tools, there are circumstances where unit tests just create the illusion of progress. I also remember Norvig saying* that sometimes the scope of pass/fail tests may be too narrow, because sometimes having 18 of the first 20 results be reasonable is good enough, but typical testing packages aren't really structured to accommodate probabilistic thinking. Certainly more relevant for some problems than others, but interesting nonetheless. I've seen major benefits from automated testing, and I'm convinced they're a net win (especially for maintenance), but I also think that being dogmatic about any particular testing methodology (TDD, etc.) is going to be counterproductive sometimes. * I don't remember if it was from the same book, PAIP, AIMA, or something else - I've been bouncing around in several AI and Prolog-related texts lately.
- wglb 17y ago
  The Norvig points were particularly thought-provoking. How do you tell if google is returning the correct result. The other is that starting a unit test without knowing what you are aiming for (e.g., the puzzle solver) you may not get there, and TDD won't help you.
keyist 17y ago
If you enjoyed seeing the contrast in approaches between Norvig and Jeffries, you may also find the following interesting -- calculating bowling scores in 1. OCaml, no tests: http://alaska-kamtchatka.blogspot.com/2009/07/disfunctional-bowling.html http://alaska-kamtchatka.blogspot.com/2009/07/disfunctional-... 2. Clojure with unit tests: http://blog.objectmentor.com/articles/2009/07/19/uncle-bob-jsps-learning-clojure http://blog.objectmentor.com/articles/2009/07/19/uncle-bob-j... </amazon>
- tptacek 17y ago
  Unclear on how anybody can cite the author of the latter with any kind of reverence, but I see that happen all the time.
  - mhartl 17y ago
    This seems unfair. (a) OCaml != Clojure; maybe the former is better for this problem. (b) Uncle Bob writes I’m trying to learn Clojure, i.e., he admits he's brand new to the language and doesn't (yet) know what he's doing. So, maybe cut the guy some slack?
    - tptacek 17y ago
      I reread the Norvig vs. Jeffries posts, and you're right. That example was breathtaking, and it probably colored my judgement about this Uncle Bob example.
  - mahmud 17y ago
    Enlighten us folks who aren't methodology scenesters. What is wrong this Uncle Bob guy?
    - tptacek 17y ago
      I don't know him from Adam, but his "TDD-based" bowling attempt in Clojure seems like a microcosm of that other guy's Sudoku solver in Ruby.
plinkplonk 17y ago
fwiw, I wrote the original blog post (that Norvig was speaking about in his interview) which compared the Sudoku efforts of Norvig and Jeffries) - http://ravimohan.blogspot.com/2007/04/learning-from-sudoku-solvers.html http://ravimohan.blogspot.com/2007/04/learning-from-sudoku-s....
anonjon 17y ago
When I started programming, I think someone (most likely my father), told me that most of my time writing code should be spent thinking about how to solve the problem (not writing code). What I am concerned about is that methodologies end up trying stand in for this old fashioned technique of Thinking Really Hard and Being Smart. (TRHBS). You'll notice that Norvig follows the TRHBS methodology. (He is a serial TRHBSer). Anyway, I guess the difference is that I see TDD as an implementation strategy for when you already know what the answer to the problem is, and are having trouble writing the code to implement it. (Although sometimes that is a clue that you are doing it wrong... Whenever I see old code with a lot of debug printf statements, I know it is going to be a much more complicated answer than is probably actually required, I can't imagine it is much different with unit tests). -j
projectileboy 17y ago
That was a fabulous analysis. For my part, I can't quite figure out why people seem to get emotional about TDD one way or the other. If it helps you, great. If not, ditch it. Do people work in environments where they'd like to avoid TDD but are forced to work that way? Or vice versa?
- stcredzero 17y ago
  Do people work in environments where they'd like to avoid TDD but are forced to work that way? Or vice versa? Yes. Unfortunately, it's one of those things that only works well if you have total buy-in from everyone on the team. Maybe we should avoid methodology that requires too much of that. Reminds me of how communal living only works if everyone pitches in. How about a methodology that works entirely off of inherent human laziness and self interest? XP works off of some laziness. But it's more like Air Cavalry in Vietnam. Land the soldiers behind enemy lines, and they have to do something productive just to save their own butts. Likewise, write some failing tests, and you have to code something to get the tests to turn green. (pass) But the problem is getting the soldiers on the choppers to begin with, or getting developers to write the tests. In the Army, there's the threat of the stockade and the firing squad. In XP, it's just getting fired. Another problem that startups are supposed to be able to avoid.
  - nostrademons 17y ago
    Can't the teams just decide this by consensus? There're a bunch of these practices that require total buy-in from a team. Things like "Are we going to use bugtracking software to coordinate who's doing what?" "Are we going to have daily stand-ups?" "What'll the conventions be for X?" etc. On the projects I've been on lately, the answer has usually been "Fine by me. Let's give it a try and see if it works out." And if it's not working out, we don't do it anymore. But oftentimes they really do help, and we end up introducing the same practices to our next project.
    - bad_user 17y ago
      > * And if it's not working out, we don't do it anymore* It's not as simple as that. Once you start using a bug-tracker the bug database grows and grows, and that data is not easy to migrate to something else. And then you're stuck with a substandard solution that makes your life a hell of a lot harder than using plain old emails with naming conventions. Daily meetings are good for some people but bad for others. Scrum meetings are advertised as a way to "get in the flow" and to let others know of your problems so they can help you. But in practice a scrum meeting turns out to be just your average status report you give your manager, and it's not going to stop him from interrupting you later anyway. This means that whenever you weren't on your best behavior, you're going to game the system and lie, making the whole meeting pointless and a time-waist. "Conventions for X" in a team with lots of opinions quickly turn into decision meetings. I had to argue really hard to choose a sane naming-convention for our database, because our tech-lead was a Java programmer "with years of experience" and the project manager was incapable of making such decisions and was delegating these choices to the tech-lead. Two years later and the whole project is a mess, because some decisions where made as if we could change them later, and others where pulled out of somebody's ass as a conflict resolution. For an epic example of such a train-wreck, go read "Dreaming in Code". Teams aren't going to decide anything right by consensus. You need one or two programmers on your team that have strong-leadership skills and generally just kick ass, and let them decide.
johnrob 17y ago
I generally take a middle road - I write code that is unit-testable, but I rarely take the time to write exhaustive tests. When bugs arise, I start writing test cases in various components until I find them. Thus, the debugging effort is what grows the test coverage. Of course in a vacuum it's better to have the cases earlier rather than later, but I like this approach as a speed/quality compromise. The key to making this work is designing in a test friendly way - that is the true art.
- wastedbrains 17y ago
  I do that a lot to, I try to call it Test Focused Development opposed to TDD
- earl 17y ago
  I'm personally of the opinion that this comprises a large portion (>60%) of the value of unit tests. Unit testable code tends to not have many of the side effects / global variables / nasty state that make nasty bugs, so this alone tends to drive down the number and severity of bugs. The testing itself is icing on the cake.
- wenbert 17y ago
  I strongly agree with this. The sad reality is that some of us have deadlines to meet. I think this is an option that makes sense. Writing testable code to get the job done and meet the deadline and the unit tests come in later when needed. EG: new features, bugs, etc. Not a good practice and ideal but it is practical. IMHO
  - eru 17y ago
    It's not too bad. At least you avoid recessing bugs.
floodfx 17y ago
Unit testing is the only way to scale lines of code without scaling your team.
Aleran 17y ago
In all fairness, how can anyone compare Peter Norvig and Ron Jeffries on the same level? One is an AI genius and the other is a XP coach. They are on very different levels intellectually.
- mquander 17y ago
  Solving Sudoku isn't exactly rocket science. I can understand if you don't have the cleanest, most elegant solution in the world, but if you can't even make a tiny bit of progress toward writing a Sudoku solver in a few hours, then I don't think you have any business holding a programming job, much less telling other programmers how to do their jobs. I mean, those Ron Jeffries blog posts read like someone who has never solved a non-trivial programming problem in his life. He literally makes no headway on any difficult part of the problem, and he spends what appears to be the better part of several hours working hard to get nowhere on code that does very little. If someone is listening to him about how to approach programming projects, I've got a bridge to sell that man.
  - litewulf 17y ago
    In fact, solving Sudoku puzzles was a programming assignment given in my first year of undergraduate study. We were given input/output specifications, and told to write in C++. That was it. Seeing as how most of the class passes, I assume almost any joker can write a half-decent sudoku solver if sufficiently motivated.
- abecedarius 17y ago
  It's not just that Norvig is smart; he's specifically skilled at writing really good code, apparently because that's something he cared about and worked on. At JPL I worked with a bunch of smart researchers with PhDs but some wrote better code than others, and it didn't mean they were on a different level intellectually. Norvig's book _Paradigms of AI Programming_ has 900+ pages presenting code as instructive as that Sudoku solver; I've never seen a better collection. http://norvig.com/paip.html http://norvig.com/paip.html
- earl 17y ago
  Dude, a sudoku solver is really directly solved via brute force search. If a coder can't come up with the brute force algorithm for a 9x9 sudoku board, there's something wrong with that person.
mrshoe 17y ago
The most important lesson to learn here is that all code, including tests, is a means to an end. A lot of people geek out about new programming languages, object oriented design, automated testing, TDD, XP, and various other methodologies du jour. If those things help you achieve your goal, that's great; but it's important to remember that they're (usually) not your goal in and of themselves. I don't think jwz would say that unit testing is a bad idea. What he was trying to say when he dismissed them was that they were focused on making a browser, not on making a nice piece of code. Similarly, Norvig was focused on solving sudokus, not on exploring methodologies that might be used to write the code which solves sudokus.
- plinkplonk 17y ago
  "TDD, XP, and various other methodologies du jour. If those things help you achieve your goal, that's great; but it's important to remember that they're (usually) not your goal in and of themselves." When you are an agile "coach", Scrum Trainer etc, TDD, XP , Scrum etc are your goals!
mhartl 17y ago
Nothing can excuse testing a non-solution, so I won't defend it. But correct code, code that does solve your problem, isn't always pretty: sometimes it's straw instead of gold. And testing is orthogonal: you can write untested gold, untested straw, tested straw, or tested gold. Sometimes you're lucky and untested gold bursts fully-formed like Athena from the head of Zeus. Otherwise, be glad for testing, which lets you turn untested straw into tested straw, and thence, via a refactoring Rumpelstiltskin, to tested gold.
Tichy 17y ago
"One thing I noticed, reading through Jeffries’s blog posts, was that he got fixated on the problem of how to represent a Sudoku board." Reminds me of pg's writings on exploratory programming (I think that is what he called it). For me, maybe I am a bad programmer, but I always change my mind on how to represent things once I sit down to code. I can read specs or think about how to do stuff over and over, it all only happens when I finally start to code. So it sounds as if too much test first would make me stuck. Plus, as the article points out, it would be boring to always just work on unit tests and never actually produce code that does stuff. Not that I dislike unit tests, but I tend to write many of them after coding the main stuff (in the testing phase).
bad_user 17y ago
I simply loved this article and the comparison made between the two Sudoku solvers. Peter Norvig is one of my favorite programmers ... his solutions are always simple and elegant. Have a look at his spell-checker here (a solution I used in production): http://norvig.com/spell-correct.html http://norvig.com/spell-correct.html He just kicks ass, and it reinforces that people (at least in software development) are more important than processes. I don't do TDD because our consultancy gigs only last for 2 months tops, during which we have to do what other firms are doing in 1 or 2 years. We are also most of the time in uncharted waters, and doing proper unit-testing of such modules requires even more thought than the actual functionality. And after two months tops, the project is over, and we won't get any money for extra work. We are doing unit-testing for really critical sections though. But I do wish I had the time and the skills to do TDD.
whichdokta 17y ago
When testing helps, test. When testing hinders, stop testing. This business of "must" and "always" anything ? Not so much. Unless your business is a lucrative hourly rate telling those who believe in "must" and "always" what they should be doing instead of thinking for themselves.
DanielBMarkham 17y ago
Great article. Lots of detail and the author doesn't get into editorializing too much. I'm in the "want to believe" category with TDD, solely for the reason that I think there are a lot of smart people advocating it. Having said that, I haven't seen a lot of teams that use it and continue using it for a competitive advantage -- and that sets off alarm bells. The hype may very well be ahead of reality on this one. I honestly don't know.
10ren 17y ago
One day I hope Joel eventually realizes this. Programmers who say they don’t have time to write tests are living in the stone age. This kind of attitude really pisses me off. It's so confident in its condescension and name-calling, I have a little wonder that they might be right. In fact, it's just dogmatic abuse, unencumbered by an objective factual appraisal of the issue at hand. Probably, I should just remember that dogmatism tends to be inversely related to wisdom. For one thing, the purpose of the code makes a tremendous difference: e.g. unit-testing is great for maintainability, but terrible for evolving an API (as in prototyping or exploratory code). And thank goodness for Knuth.
- ZeroGravitas 17y ago
  I was amused by his comment on Knuth: "So Knuth too disagrees with the notion that unit testing always makes you go faster. Maybe he too is living in the stone age." This follows him describing writing a program, in pencil, in 1977.
  - briancooley 17y ago
    I think the list of programmers throughout history who could reliably write something as complex as TeX using nothing but paper and pencil is incredibly short.
    - dkarl 17y ago
      It's a lot easier than you think. When I was a kid I wrote longer programs in Basic using paper and pencil, because it was a lot easier for me than typing. I would type in the program after I was pretty sure it was right. The errors were almost always local. It seemed so easy to check the high-level structure of something when I could spread it out on the ground in front of me. Later, in college, I always printed out drafts of my papers and marked them up completely before I started editing.
- nickelplate 17y ago
  What I find most irritating is that the TDD or other methodology zealots usually cannot point to a success of exactly the sort people like jwz or Joel have been influential in shipping. There also seems to be a tendency among these people to confuse rejection of TDD with rejection of unit testing.
ZeroGravitas 17y ago
So the general conclusion, from various programming legends, seems to be: • (semi-)automated testing for any complex code: essential • unit tests for any long lived code: great • TDD (writing the tests before the code): good in theory but alien to most folks • tests speed/slow you up/down: open question (though it would seem logical that there will be certain projects that benefit more than others, e.g. the longer lived the code the longer you can earn "interest" on your initial "investment") Yet there still seems to be a general reaction here of "I'm too smart and/or busy to write tests" and the corollary "Anyone who writes tests, or advocates testing, isn't a good programmer". Seems a strange disconnect.
- plinkplonk 17y ago
  "(semi-)automated testing for any complex code: essential" Where did you see this idea in Coders At Work? Most of them say testing is important . Most of them don't say "semi automated" testing is "essential". So where is your perceived "consensus" of programming legends coming from? or this "TDD (writing the tests before the code): good in theory but alien to most folks" for that matter? Sibel mentions 5 developers in his blog post - Zawinski, Knuth, Norvig, Bloch, Armstrong. I didn't see anyone saying "good in theory but alien to most folks". Only Armstrong says he regularly does (soemthing like) "test driven", but if you read his interview he also says he spends a lot of time doodling and getting his abstractions right before coding, hardly the "YAGNI" style of coding and theory of "emergent design" propounded by most agile consultants. Who in CaW do you think concludes either of the two statements you put forward as "general conclusions from programming legends". Or are you saying Robert Martin and co are programming legends? Bit hard to justify I'd think ;-) You say, "There still seems to be a general reaction here" .. "I'm too smart and/or busy to write tests" and the corollary "Anyone who writes tests, or advocates testing, isn't a good programmer"." The former " I am too busy to write (TDD style) tests" is a justifiable argument in certain circumstances (Zawinski says as much) and the latter "Anyone who advocates testing is a poor programmer" is something I don't see anyone here saying. Who said this? Links please? And if someone did, why do you think it is a "general reaction"? fwiw I see the evolving consensus here ) as "writing unit tests is valuable in certain circumstances (and not so in others) as long as you aren't fanatical about them" and "what is important is not the methodology but the end result". You seem to be putting forward your beliefs about the value/validity of TDD/testing etc as the "general conclusion, from various programming legends". I am curious as to how you came up with this apparent consensus. Don't get me wrong, you certainly have a right to your conclusions. I am just saying the CaW interviews and the posts here don't seem to converge to a conclusion along the lines of you say they did.
ErrantX 17y ago
I agree with Spolsky on this one; and I cant help feeling TDD is like agile. One of those things cool companies and corporates use to sound "cutting edge". We used TDD for a few projects (and agile too) and found it does slow you down no end. And it also ultimately doesn't catch many useful bugs and problems. We still had to go through the usual end-point testing cycles. We write unit tests for any of our API's and also for some of our core code; but only after original versions are in place. They are supposed to catch any mistakes or errors in futures we enhance and evolve the programming so we don't break backwards compatibility. I think it really does come down to what works best for your teams though. Im sure plenty of people find TDD a great addition to their arsenal. But at the end of the day it's just a buzz word like everything else - and we've found that by avoiding those kinds of things we produce good, solid, working code at a fast (not necessarily faster pace) - but with less headaches :)
- gaius 17y ago
  The thing with unit testing is, a lot of the time it's used to try and make a weakly-typed language with uncontrollable side-effects behave like a strongly typed language in which pure functions can be written. A team lead who doesn't understand the above sentiment is at best only going through the motions with unit testing.
  - ErrantX 17y ago
    Im not sure I agree. Unit testing is about code being to a specific standard (i.e. that API works like X or like Y). Surely your referring to Fuzz testing?
gcv 17y ago
As far as testing goes, I have a simple approach. If I just wrote some code and had to poke at it in a REPL with some different inputs to check if it works, then I should take whatever I did and turn it into a test. With a good testing framework, like the one built into Clojure (formerly clojure.contrib.test-is and now clojure.test), it's really easy. I just copy from my REPL and paste it into the appropriate test file, along with the expected results. If I find a bug, I fix it, write a test for it, and also copy whatever I did to check my work from the REPL into a test file. That's it. I routinely change code around with all the confidence TDD advocates claim. I don't test trivialities. I don't test if my database connection was correctly established --- but I will test database connection loss error handling if I put any non-trivial logic in there. I'll often write a little snippet of sample code for an API I'm working on, but I never write a full formal test in advance. It makes letting the actual API evolve as it grows too cumbersome.
- eru 17y ago
  Sensible. Though writing a test case for an evolving API might be of benefit from time to time, because it forces you to use the API (and thus see if it is sensible).
maryrosecook 17y ago
You know what has improved the quality of my code? Having every error on my live servers emailed to me. I feel compelled to fix it because I know that an actual user has had a problem. I tried to do unit testing for a project. I'd write a method, then write some tests for it. Some observations: * I thought that writing the tests would give me refactoring ideas for the actual methods. It did not. * Running my test suite prior to each deploy saved me from shipping a bug once. Once. * Writing tests is fucking boring. This was all code that I knew the purpose of in advance. I have no idea how one writes tests when doing exploratory coding.
- geebee 17y ago
  "I have no idea how one writes tests when doing exploratory coding." You're probably already writing tests, especially when you're doing exploratory coding. How much code do you really write before you check to see if something works? You're almost certainly doing something. Maybe it's just a small main program where you validate that some methods are giving you the output you expected, or maybe it's a simple web form that you populate with mock data to verify it's persisting to a database. Unless you wait until the entire functionality is ready before a trial run (and if you're doing exploratory coding, there's no such thing as "ready" anyway), you are writing something to test during iterations. Instead of throwing these tests away, keep them somewhere and run them periodically. If they break, figure out why - and either fix the code, or change the test. When you're done, you'll have a bunch of unit tests. (just a quick note - the process I described above works extremely well for some situations, but poorly for others. Some frameworks make it a real hassle to write unit tests. The process I described also is not TDD, it's more of a write a little, test a little approach, which feels more natural to me. I also don't really like to use TDD, especially for exploratory coding).
  - maryrosecook 17y ago
    You're right: I write a little, do a test run of the code to see what comes back, fix it if it doesn't work, write a bit more and so forth. And I see how saving those little tests could be useful.
toadpipe 17y ago
So now that I've voted up every plinkplonk comment, my comment is that most people obviously haven't read the book. The post is pretty good, but the book is better. You should read it. If you do read the book, you will see that the discussion begins with Norvig saying that he thinks one of the most important things is being able to keep everything in your head at once. Extra tools come in when the problem gets too big to do that, but here's the key: he saw right from the very beginning that the Sudoku problem could be solved with two tools from the AI toolbox. In other words, he saw the entire solution immediately, and it was never too large to fit easily in his head. Seibel's analysis fine as far as it goes, but he misses the really important thing here, which is that this is not just an example of someone recognizing a problem that they already knew how to solve. It's a case of someone with the mental tools that allow them to dramatically reduce the (apparent) complexity and size of a very large class of problems that happens to include Sudoku. Seibel takes a bottom up look at the data structures Norvig used, but this can be misleading because they weren't designed bottom up, they were designed all at once. You simply cannot do that unless you have the necessary training in abstract thinking (read: mathematics/formal logic/language development/ai techniques/etc). No amount of code centric techniques or tools will ever make up for not having these tools. There will always be (relatively trivial) problems that you will never be able solve without them, because you will not be able to fit everything in your head and your ability to reason about the problem will be crippled by that. Debates about TDD are not even wrong, because they are at the wrong level of abstraction. Spending any significant amount of time discussing it is premature optimization.

109 points by mqt 17y ago

I sit somewhere in the middle. Unit testing GUI code is going to slow you down. Unit testing your core algorithms is essential. That is the stuff that must never break, and where breakages would not be immediately detectable. The unit tests ensure that you are always sure that that code works. (Real world example; testing somewhat hairy parsing code: http://github.com/jrockway/moosex-runnable/blob/master/t/arg-parser.t http://github.com/jrockway/moosex-runnable/blob/master/t/arg... I test every case I could come up with, so I could be sure that it works. Better to spend the time typing this file than to be hit with a weird misparse on the command-line, where you are probably not in the mood to fix the parser.) On top of the unit tests, I do integration tests, to make sure sections of my app mostly work together. This is somewhere below "Request -> Response" and somewhere above the individual routines/classes. In the web app case, this is mostly "instantiate an object graph; call the methods that provide data to the rendering functions; make sure the results make sense". (I assume my template renderer and object database work; those have their own test suite, after all.) I also test stuff like typical sequences of actions that share some state (usually "the session"). I neglected that once and it caused me lots of problems. But anyway, GUI testing == waste of time. As long as the buttons you click generate the right events, the GUI works. This is the sort of app Zawinski was working on, and this colors his thoughts on testing accordingly. Your mileage may vary.

brown9-2 17y ago
Why exactly is unit testing of GUI components a waste of time?
- anonjon 17y ago
  They aren't safety critical and you can test them by using the GUI.
- jrockway 17y ago
  Because it takes a lot of time and effort but does not provide much benefit.
  - jamesbritt 17y ago
    OK, but why? Is it not useful to know that various operations and navigations through the GUI still work as expected? And if it's a matter of a poor benefit/effort ratio, why are GUIs so hard to test? Tools like Swinger are pretty easy to get rolling with for testing Swing UIs. Where do these things break down? Are these things worth fixing?
    - akeefer 17y ago
      They break down because UI testing tends to rely on Strings: either labels for controls or embedded ids of them. It's really, really easy to have those change on you, and when they do the tests become difficult to debug: if the button with the id "Search" isn't found, is it because something broke to cause the button not to show up, because the id has changed to "SearchUsers," or what? And if I need to remove the "Search" button and replace it with somethine else, how do I figure out which tests to change? The high-level nature of the tests inherently makes them much harder to debug, because the test could have broken for any of a hundred reasons. In other words, with unit tests the linkage between the code being tested and the test tends to be pretty tight; with UI tests it tends to be very, very loose, and that makes the tests correspondingly more fragile and much harder to debug. We essentially do typesafe metaprogramming on our web UI that generates compile-time-checked constants for all the labels, buttons, etc. so that our tests don't compile if the UI changes, which has gone a long way to keeping the tests stable; it's the best solution we've come up with, but it's a huge investment, and our attempts to do testing of our Swing client have met with less success so far.
    - jrockway 17y ago
      In my experience, I have sunk a lot of time making sure that foo div has bar css class when the quux link is clicked... but it has never saved me much time. I have to click through the site regularly (content changes, "does this work in IE", etc.) anyway, and errors are usually noticeable immediately. Other problems I've noticed are that they are either so specific that they fail for no reason (it was supposed to be red, not green!) or too general that they don't catch issues that would annoy users. If I could have these tests for free, I'd take 'em. But since they're expensive and don't get me much, I don't bother. (BTW, if you have complicated algorithms in your JavaScript; refactor so you can test them with Rhino on the command-line. Don't do this stuff in the browser!) Anyway, I would be interested in hearing your UI testing success stories.
- chrisconley 17y ago
  Maybe a better way to put it is that testing GUI components is not a complete waste of time, but it gives you a lot less bang for the buck.

silentbicycle 17y ago

The part about TDD in the interview with Peter Norvig really jumped out at me, too. A passing test doesn't always mean anything important has changed - like most other programming tools, there are circumstances where unit tests just create the illusion of progress. I also remember Norvig saying* that sometimes the scope of pass/fail tests may be too narrow, because sometimes having 18 of the first 20 results be reasonable is good enough, but typical testing packages aren't really structured to accommodate probabilistic thinking. Certainly more relevant for some problems than others, but interesting nonetheless. I've seen major benefits from automated testing, and I'm convinced they're a net win (especially for maintenance), but I also think that being dogmatic about any particular testing methodology (TDD, etc.) is going to be counterproductive sometimes. * I don't remember if it was from the same book, PAIP, AIMA, or something else - I've been bouncing around in several AI and Prolog-related texts lately.

wglb 17y ago
The Norvig points were particularly thought-provoking. How do you tell if google is returning the correct result. The other is that starting a unit test without knowing what you are aiming for (e.g., the puzzle solver) you may not get there, and TDD won't help you.

keyist 17y ago

If you enjoyed seeing the contrast in approaches between Norvig and Jeffries, you may also find the following interesting -- calculating bowling scores in 1. OCaml, no tests: http://alaska-kamtchatka.blogspot.com/2009/07/disfunctional-bowling.html http://alaska-kamtchatka.blogspot.com/2009/07/disfunctional-... 2. Clojure with unit tests: http://blog.objectmentor.com/articles/2009/07/19/uncle-bob-jsps-learning-clojure http://blog.objectmentor.com/articles/2009/07/19/uncle-bob-j... </amazon>

tptacek 17y ago
Unclear on how anybody can cite the author of the latter with any kind of reverence, but I see that happen all the time.
- mhartl 17y ago
  This seems unfair. (a) OCaml != Clojure; maybe the former is better for this problem. (b) Uncle Bob writes I’m trying to learn Clojure, i.e., he admits he's brand new to the language and doesn't (yet) know what he's doing. So, maybe cut the guy some slack?
  - tptacek 17y ago
    I reread the Norvig vs. Jeffries posts, and you're right. That example was breathtaking, and it probably colored my judgement about this Uncle Bob example.
- mahmud 17y ago
  Enlighten us folks who aren't methodology scenesters. What is wrong this Uncle Bob guy?
  - tptacek 17y ago
    I don't know him from Adam, but his "TDD-based" bowling attempt in Clojure seems like a microcosm of that other guy's Sudoku solver in Ruby.

plinkplonk 17y ago

fwiw, I wrote the original blog post (that Norvig was speaking about in his interview) which compared the Sudoku efforts of Norvig and Jeffries) - http://ravimohan.blogspot.com/2007/04/learning-from-sudoku-solvers.html http://ravimohan.blogspot.com/2007/04/learning-from-sudoku-s....

anonjon 17y ago

When I started programming, I think someone (most likely my father), told me that most of my time writing code should be spent thinking about how to solve the problem (not writing code). What I am concerned about is that methodologies end up trying stand in for this old fashioned technique of Thinking Really Hard and Being Smart. (TRHBS). You'll notice that Norvig follows the TRHBS methodology. (He is a serial TRHBSer). Anyway, I guess the difference is that I see TDD as an implementation strategy for when you already know what the answer to the problem is, and are having trouble writing the code to implement it. (Although sometimes that is a clue that you are doing it wrong... Whenever I see old code with a lot of debug printf statements, I know it is going to be a much more complicated answer than is probably actually required, I can't imagine it is much different with unit tests). -j

projectileboy 17y ago

That was a fabulous analysis. For my part, I can't quite figure out why people seem to get emotional about TDD one way or the other. If it helps you, great. If not, ditch it. Do people work in environments where they'd like to avoid TDD but are forced to work that way? Or vice versa?

stcredzero 17y ago
Do people work in environments where they'd like to avoid TDD but are forced to work that way? Or vice versa? Yes. Unfortunately, it's one of those things that only works well if you have total buy-in from everyone on the team. Maybe we should avoid methodology that requires too much of that. Reminds me of how communal living only works if everyone pitches in. How about a methodology that works entirely off of inherent human laziness and self interest? XP works off of some laziness. But it's more like Air Cavalry in Vietnam. Land the soldiers behind enemy lines, and they have to do something productive just to save their own butts. Likewise, write some failing tests, and you have to code something to get the tests to turn green. (pass) But the problem is getting the soldiers on the choppers to begin with, or getting developers to write the tests. In the Army, there's the threat of the stockade and the firing squad. In XP, it's just getting fired. Another problem that startups are supposed to be able to avoid.
- nostrademons 17y ago
  Can't the teams just decide this by consensus? There're a bunch of these practices that require total buy-in from a team. Things like "Are we going to use bugtracking software to coordinate who's doing what?" "Are we going to have daily stand-ups?" "What'll the conventions be for X?" etc. On the projects I've been on lately, the answer has usually been "Fine by me. Let's give it a try and see if it works out." And if it's not working out, we don't do it anymore. But oftentimes they really do help, and we end up introducing the same practices to our next project.
  - bad_user 17y ago
    > * And if it's not working out, we don't do it anymore* It's not as simple as that. Once you start using a bug-tracker the bug database grows and grows, and that data is not easy to migrate to something else. And then you're stuck with a substandard solution that makes your life a hell of a lot harder than using plain old emails with naming conventions. Daily meetings are good for some people but bad for others. Scrum meetings are advertised as a way to "get in the flow" and to let others know of your problems so they can help you. But in practice a scrum meeting turns out to be just your average status report you give your manager, and it's not going to stop him from interrupting you later anyway. This means that whenever you weren't on your best behavior, you're going to game the system and lie, making the whole meeting pointless and a time-waist. "Conventions for X" in a team with lots of opinions quickly turn into decision meetings. I had to argue really hard to choose a sane naming-convention for our database, because our tech-lead was a Java programmer "with years of experience" and the project manager was incapable of making such decisions and was delegating these choices to the tech-lead. Two years later and the whole project is a mess, because some decisions where made as if we could change them later, and others where pulled out of somebody's ass as a conflict resolution. For an epic example of such a train-wreck, go read "Dreaming in Code". Teams aren't going to decide anything right by consensus. You need one or two programmers on your team that have strong-leadership skills and generally just kick ass, and let them decide.

johnrob 17y ago

I generally take a middle road - I write code that is unit-testable, but I rarely take the time to write exhaustive tests. When bugs arise, I start writing test cases in various components until I find them. Thus, the debugging effort is what grows the test coverage. Of course in a vacuum it's better to have the cases earlier rather than later, but I like this approach as a speed/quality compromise. The key to making this work is designing in a test friendly way - that is the true art.

wastedbrains 17y ago
I do that a lot to, I try to call it Test Focused Development opposed to TDD
earl 17y ago
I'm personally of the opinion that this comprises a large portion (>60%) of the value of unit tests. Unit testable code tends to not have many of the side effects / global variables / nasty state that make nasty bugs, so this alone tends to drive down the number and severity of bugs. The testing itself is icing on the cake.
wenbert 17y ago
I strongly agree with this. The sad reality is that some of us have deadlines to meet. I think this is an option that makes sense. Writing testable code to get the job done and meet the deadline and the unit tests come in later when needed. EG: new features, bugs, etc. Not a good practice and ideal but it is practical. IMHO
- eru 17y ago
  It's not too bad. At least you avoid recessing bugs.

floodfx 17y ago

Unit testing is the only way to scale lines of code without scaling your team.

Aleran 17y ago

In all fairness, how can anyone compare Peter Norvig and Ron Jeffries on the same level? One is an AI genius and the other is a XP coach. They are on very different levels intellectually.

mquander 17y ago
Solving Sudoku isn't exactly rocket science. I can understand if you don't have the cleanest, most elegant solution in the world, but if you can't even make a tiny bit of progress toward writing a Sudoku solver in a few hours, then I don't think you have any business holding a programming job, much less telling other programmers how to do their jobs. I mean, those Ron Jeffries blog posts read like someone who has never solved a non-trivial programming problem in his life. He literally makes no headway on any difficult part of the problem, and he spends what appears to be the better part of several hours working hard to get nowhere on code that does very little. If someone is listening to him about how to approach programming projects, I've got a bridge to sell that man.
- litewulf 17y ago
  In fact, solving Sudoku puzzles was a programming assignment given in my first year of undergraduate study. We were given input/output specifications, and told to write in C++. That was it. Seeing as how most of the class passes, I assume almost any joker can write a half-decent sudoku solver if sufficiently motivated.
abecedarius 17y ago
It's not just that Norvig is smart; he's specifically skilled at writing really good code, apparently because that's something he cared about and worked on. At JPL I worked with a bunch of smart researchers with PhDs but some wrote better code than others, and it didn't mean they were on a different level intellectually. Norvig's book _Paradigms of AI Programming_ has 900+ pages presenting code as instructive as that Sudoku solver; I've never seen a better collection. http://norvig.com/paip.html http://norvig.com/paip.html
earl 17y ago
Dude, a sudoku solver is really directly solved via brute force search. If a coder can't come up with the brute force algorithm for a 9x9 sudoku board, there's something wrong with that person.

mrshoe 17y ago

The most important lesson to learn here is that all code, including tests, is a means to an end. A lot of people geek out about new programming languages, object oriented design, automated testing, TDD, XP, and various other methodologies du jour. If those things help you achieve your goal, that's great; but it's important to remember that they're (usually) not your goal in and of themselves. I don't think jwz would say that unit testing is a bad idea. What he was trying to say when he dismissed them was that they were focused on making a browser, not on making a nice piece of code. Similarly, Norvig was focused on solving sudokus, not on exploring methodologies that might be used to write the code which solves sudokus.

plinkplonk 17y ago
"TDD, XP, and various other methodologies du jour. If those things help you achieve your goal, that's great; but it's important to remember that they're (usually) not your goal in and of themselves." When you are an agile "coach", Scrum Trainer etc, TDD, XP , Scrum etc are your goals!

mhartl 17y ago

Nothing can excuse testing a non-solution, so I won't defend it. But correct code, code that does solve your problem, isn't always pretty: sometimes it's straw instead of gold. And testing is orthogonal: you can write untested gold, untested straw, tested straw, or tested gold. Sometimes you're lucky and untested gold bursts fully-formed like Athena from the head of Zeus. Otherwise, be glad for testing, which lets you turn untested straw into tested straw, and thence, via a refactoring Rumpelstiltskin, to tested gold.

Tichy 17y ago

"One thing I noticed, reading through Jeffries’s blog posts, was that he got fixated on the problem of how to represent a Sudoku board." Reminds me of pg's writings on exploratory programming (I think that is what he called it). For me, maybe I am a bad programmer, but I always change my mind on how to represent things once I sit down to code. I can read specs or think about how to do stuff over and over, it all only happens when I finally start to code. So it sounds as if too much test first would make me stuck. Plus, as the article points out, it would be boring to always just work on unit tests and never actually produce code that does stuff. Not that I dislike unit tests, but I tend to write many of them after coding the main stuff (in the testing phase).

bad_user 17y ago

I simply loved this article and the comparison made between the two Sudoku solvers. Peter Norvig is one of my favorite programmers ... his solutions are always simple and elegant. Have a look at his spell-checker here (a solution I used in production): http://norvig.com/spell-correct.html http://norvig.com/spell-correct.html He just kicks ass, and it reinforces that people (at least in software development) are more important than processes. I don't do TDD because our consultancy gigs only last for 2 months tops, during which we have to do what other firms are doing in 1 or 2 years. We are also most of the time in uncharted waters, and doing proper unit-testing of such modules requires even more thought than the actual functionality. And after two months tops, the project is over, and we won't get any money for extra work. We are doing unit-testing for really critical sections though. But I do wish I had the time and the skills to do TDD.

whichdokta 17y ago

When testing helps, test. When testing hinders, stop testing. This business of "must" and "always" anything ? Not so much. Unless your business is a lucrative hourly rate telling those who believe in "must" and "always" what they should be doing instead of thinking for themselves.

DanielBMarkham 17y ago

Great article. Lots of detail and the author doesn't get into editorializing too much. I'm in the "want to believe" category with TDD, solely for the reason that I think there are a lot of smart people advocating it. Having said that, I haven't seen a lot of teams that use it and continue using it for a competitive advantage -- and that sets off alarm bells. The hype may very well be ahead of reality on this one. I honestly don't know.

10ren 17y ago

One day I hope Joel eventually realizes this. Programmers who say they don’t have time to write tests are living in the stone age. This kind of attitude really pisses me off. It's so confident in its condescension and name-calling, I have a little wonder that they might be right. In fact, it's just dogmatic abuse, unencumbered by an objective factual appraisal of the issue at hand. Probably, I should just remember that dogmatism tends to be inversely related to wisdom. For one thing, the purpose of the code makes a tremendous difference: e.g. unit-testing is great for maintainability, but terrible for evolving an API (as in prototyping or exploratory code). And thank goodness for Knuth.

ZeroGravitas 17y ago
I was amused by his comment on Knuth: "So Knuth too disagrees with the notion that unit testing always makes you go faster. Maybe he too is living in the stone age." This follows him describing writing a program, in pencil, in 1977.
- briancooley 17y ago
  I think the list of programmers throughout history who could reliably write something as complex as TeX using nothing but paper and pencil is incredibly short.
  - dkarl 17y ago
    It's a lot easier than you think. When I was a kid I wrote longer programs in Basic using paper and pencil, because it was a lot easier for me than typing. I would type in the program after I was pretty sure it was right. The errors were almost always local. It seemed so easy to check the high-level structure of something when I could spread it out on the ground in front of me. Later, in college, I always printed out drafts of my papers and marked them up completely before I started editing.
nickelplate 17y ago
What I find most irritating is that the TDD or other methodology zealots usually cannot point to a success of exactly the sort people like jwz or Joel have been influential in shipping. There also seems to be a tendency among these people to confuse rejection of TDD with rejection of unit testing.

ZeroGravitas 17y ago

So the general conclusion, from various programming legends, seems to be: • (semi-)automated testing for any complex code: essential • unit tests for any long lived code: great • TDD (writing the tests before the code): good in theory but alien to most folks • tests speed/slow you up/down: open question (though it would seem logical that there will be certain projects that benefit more than others, e.g. the longer lived the code the longer you can earn "interest" on your initial "investment") Yet there still seems to be a general reaction here of "I'm too smart and/or busy to write tests" and the corollary "Anyone who writes tests, or advocates testing, isn't a good programmer". Seems a strange disconnect.

plinkplonk 17y ago
"(semi-)automated testing for any complex code: essential" Where did you see this idea in Coders At Work? Most of them say testing is important . Most of them don't say "semi automated" testing is "essential". So where is your perceived "consensus" of programming legends coming from? or this "TDD (writing the tests before the code): good in theory but alien to most folks" for that matter? Sibel mentions 5 developers in his blog post - Zawinski, Knuth, Norvig, Bloch, Armstrong. I didn't see anyone saying "good in theory but alien to most folks". Only Armstrong says he regularly does (soemthing like) "test driven", but if you read his interview he also says he spends a lot of time doodling and getting his abstractions right before coding, hardly the "YAGNI" style of coding and theory of "emergent design" propounded by most agile consultants. Who in CaW do you think concludes either of the two statements you put forward as "general conclusions from programming legends". Or are you saying Robert Martin and co are programming legends? Bit hard to justify I'd think ;-) You say, "There still seems to be a general reaction here" .. "I'm too smart and/or busy to write tests" and the corollary "Anyone who writes tests, or advocates testing, isn't a good programmer"." The former " I am too busy to write (TDD style) tests" is a justifiable argument in certain circumstances (Zawinski says as much) and the latter "Anyone who advocates testing is a poor programmer" is something I don't see anyone here saying. Who said this? Links please? And if someone did, why do you think it is a "general reaction"? fwiw I see the evolving consensus here ) as "writing unit tests is valuable in certain circumstances (and not so in others) as long as you aren't fanatical about them" and "what is important is not the methodology but the end result". You seem to be putting forward your beliefs about the value/validity of TDD/testing etc as the "general conclusion, from various programming legends". I am curious as to how you came up with this apparent consensus. Don't get me wrong, you certainly have a right to your conclusions. I am just saying the CaW interviews and the posts here don't seem to converge to a conclusion along the lines of you say they did.

ErrantX 17y ago

I agree with Spolsky on this one; and I cant help feeling TDD is like agile. One of those things cool companies and corporates use to sound "cutting edge". We used TDD for a few projects (and agile too) and found it does slow you down no end. And it also ultimately doesn't catch many useful bugs and problems. We still had to go through the usual end-point testing cycles. We write unit tests for any of our API's and also for some of our core code; but only after original versions are in place. They are supposed to catch any mistakes or errors in futures we enhance and evolve the programming so we don't break backwards compatibility. I think it really does come down to what works best for your teams though. Im sure plenty of people find TDD a great addition to their arsenal. But at the end of the day it's just a buzz word like everything else - and we've found that by avoiding those kinds of things we produce good, solid, working code at a fast (not necessarily faster pace) - but with less headaches :)

gaius 17y ago
The thing with unit testing is, a lot of the time it's used to try and make a weakly-typed language with uncontrollable side-effects behave like a strongly typed language in which pure functions can be written. A team lead who doesn't understand the above sentiment is at best only going through the motions with unit testing.
- ErrantX 17y ago
  Im not sure I agree. Unit testing is about code being to a specific standard (i.e. that API works like X or like Y). Surely your referring to Fuzz testing?

gcv 17y ago

As far as testing goes, I have a simple approach. If I just wrote some code and had to poke at it in a REPL with some different inputs to check if it works, then I should take whatever I did and turn it into a test. With a good testing framework, like the one built into Clojure (formerly clojure.contrib.test-is and now clojure.test), it's really easy. I just copy from my REPL and paste it into the appropriate test file, along with the expected results. If I find a bug, I fix it, write a test for it, and also copy whatever I did to check my work from the REPL into a test file. That's it. I routinely change code around with all the confidence TDD advocates claim. I don't test trivialities. I don't test if my database connection was correctly established --- but I will test database connection loss error handling if I put any non-trivial logic in there. I'll often write a little snippet of sample code for an API I'm working on, but I never write a full formal test in advance. It makes letting the actual API evolve as it grows too cumbersome.

eru 17y ago
Sensible. Though writing a test case for an evolving API might be of benefit from time to time, because it forces you to use the API (and thus see if it is sensible).

maryrosecook 17y ago

You know what has improved the quality of my code? Having every error on my live servers emailed to me. I feel compelled to fix it because I know that an actual user has had a problem. I tried to do unit testing for a project. I'd write a method, then write some tests for it. Some observations: * I thought that writing the tests would give me refactoring ideas for the actual methods. It did not. * Running my test suite prior to each deploy saved me from shipping a bug once. Once. * Writing tests is fucking boring. This was all code that I knew the purpose of in advance. I have no idea how one writes tests when doing exploratory coding.

geebee 17y ago
"I have no idea how one writes tests when doing exploratory coding." You're probably already writing tests, especially when you're doing exploratory coding. How much code do you really write before you check to see if something works? You're almost certainly doing something. Maybe it's just a small main program where you validate that some methods are giving you the output you expected, or maybe it's a simple web form that you populate with mock data to verify it's persisting to a database. Unless you wait until the entire functionality is ready before a trial run (and if you're doing exploratory coding, there's no such thing as "ready" anyway), you are writing something to test during iterations. Instead of throwing these tests away, keep them somewhere and run them periodically. If they break, figure out why - and either fix the code, or change the test. When you're done, you'll have a bunch of unit tests. (just a quick note - the process I described above works extremely well for some situations, but poorly for others. Some frameworks make it a real hassle to write unit tests. The process I described also is not TDD, it's more of a write a little, test a little approach, which feels more natural to me. I also don't really like to use TDD, especially for exploratory coding).
- maryrosecook 17y ago
  You're right: I write a little, do a test run of the code to see what comes back, fix it if it doesn't work, write a bit more and so forth. And I see how saving those little tests could be useful.

toadpipe 17y ago

So now that I've voted up every plinkplonk comment, my comment is that most people obviously haven't read the book. The post is pretty good, but the book is better. You should read it. If you do read the book, you will see that the discussion begins with Norvig saying that he thinks one of the most important things is being able to keep everything in your head at once. Extra tools come in when the problem gets too big to do that, but here's the key: he saw right from the very beginning that the Sudoku problem could be solved with two tools from the AI toolbox. In other words, he saw the entire solution immediately, and it was never too large to fit easily in his head. Seibel's analysis fine as far as it goes, but he misses the really important thing here, which is that this is not just an example of someone recognizing a problem that they already knew how to solve. It's a case of someone with the mental tools that allow them to dramatically reduce the (apparent) complexity and size of a very large class of problems that happens to include Sudoku. Seibel takes a bottom up look at the data structures Norvig used, but this can be misleading because they weren't designed bottom up, they were designed all at once. You simply cannot do that unless you have the necessary training in abstract thinking (read: mathematics/formal logic/language development/ai techniques/etc). No amount of code centric techniques or tools will ever make up for not having these tools. There will always be (relatively trivial) problems that you will never be able solve without them, because you will not be able to fit everything in your head and your ability to reason about the problem will be crippled by that. Debates about TDD are not even wrong, because they are at the wrong level of abstraction. Spending any significant amount of time discussing it is premature optimization.