8 ms·

TDD, Straw Men, and Rhetoric

177 points by gary_bernhardt 12y ago

eclark 12y ago
All of these arguments seem like they are overly prescriptive. Individuals should use what works best for their needs. TDD may work wonders for me today on one project, and then it could be horrible tomorrow on the next project. My co-worker might have the inverse. Stop arguing over how other people create software; Ship Code instead.
- JeremyMorgan 12y ago
  You said it better than me. No reason to get religious over it. When I'm the architect on a project, I do it in some cases, and other times I don't. I'm not over concerned what everyone else is doing in this regard.
- GuiA 12y ago
  > Individuals should use what works best for their needs Maybe rather: teams should use what works best for their needs ? EDIT: whee, downvotes. Apparently HN'ers work on teams that ship software and where a developer uses kanban, another TDD, another waterfall, another scrum, and another only commits code when the 8 planets are aligned.
  - eclark 12y ago
    TDD vs Testing after is a personal choice apart from how the team set goals. That is to say you can be on a waterfall schedule and still practice TDD. As long as code shows up to code review with good tests, I don't personally care if the tests were written before or after.
  - jasonlotito 12y ago
    > whee, downvotes. I'm going to take a stab at the down votes, as I could easily understand why someone might. First, consider your original comment was pedantic and offered little in the way of original insight. It would be like me replying to you and saying: "Maybe rather: teams should use hwat works best for their needs per project?" What does that really add? Basically, it added little to the conversation. Heck, your edit complaining about the down votes is longer than your comment.
  - MartinCron 12y ago
    Maybe you are being down voted by fans of the former-planet Pluto? Sensitive bunch, those guys.
- ericHosick 12y ago
  > Stop arguing over how other people create software; Ship Code instead. Electrical engineering, mechanical engineering, architectural design and the medical profession, to name a few, have bodies of knowledge they are required to use. Is it really a good idea for software developers to say "stop arguing over how other people create software; Ship Code instead" considering we don't have any industry standard bodies of knowledge?
  - lugg 12y ago
    Considering we don't have any industry standard bodies of knowledge, yes it probably is a good idea, because we don't have anyone who can lay out a definitive answer on many of our preference based arguments. i.e. these arguments will never end if there is no definitive answer, or body to pick one.
    - gary_bernhardt 12y ago
      Industry standard bodies of knowledge arise from people doing, then talking about what they did, then doing some more, then talking some more. At no point did a God of Electrical Engineering hand down tablets.
      - lugg 12y ago
        Very good point. I stand corrected.
    - Silhouette 12y ago
      There is at least one serious effort underway to collect (or perhaps, catalogue) such a body of knowledge: http://www.computer.org/portal/web/swebok http://www.computer.org/portal/web/swebok I think SWEBOK is interesting both as a demonstration of how far we have to go before we get anywhere near the standards of established engineering fields, and as a demonstration of how much we have already collectively figured out, even if most of us aren't familiar with more than a small fraction of what is out there.
- ebiester 12y ago
  I believe that it's a useful conversation to have -- if we are all just trying to ship code, when are we going to have the discussion of what makes code readable? What makes code reliable? What makes code quicker to deliver? Further, as a automated test advocate (whether or not it's TDD or unit or mocks or stubs or whatever flavor you like) I want to walk into projects where I don't have to deal with changing code without having a good and fast test base to minimize the potential defects. If TDD gets us to that point, I'm all for it. I don't care how you create software unless I have to use it or inherit it.
ssmoot 12y ago
> "Second, that sentence is false. Isolation from the database, or anything else, is generally done with mocks, but mocks didn't even exist when TDD was rediscovered by Kent Beck in 1994-1995." See, that's not how I remember it. Stubs implementing the Interfaces you wrote (because Composition over Inheritance) was the common solution. AR's idea of a Unit Test has always been wrong to my mind. They aren't Unit tests. They're Integration tests. This only really became an issue with the rise of Rails and the fact that you couldn't really write Unit tests for an ActiveRecord. At least that's how I remember it.
- steveklabnik 12y ago
  Considering Gary provides citations for his timeline, I'm inclined to believe his over your memory. That said, I think you're confusing _unit_ tests and _TDD_. Unit tests imply that you test a single isolation, TDD implies that you write your tests before you write your implementation. You can TDD unit tests, but you don't have to, and you can do all-acceptance TDD if you want.
  - yeukhon 12y ago
    Exactly. I think most of the TDD probably come to some tutorials which test addition a+b of that sort and demonstrate how to test "ah there's logical error! how incredible TDD helps us to identify error early on!". I do some TDD from time to time, but outside of my textbook algorithm, writing pure unit test which involves using mocks is really tough and time-consuming (unless someone has built a test driver for you to use throughout the whole development). In any, if you ever do curl http://localhost:port/mynewapp/page1 http://localhost:port/mynewapp/page1 that's already testing and is doing functional testing. If you write that down before you start writing a piece of code, you are doing TDD to test whether your app will return 200 or not.
  - ssmoot 12y ago
    The quote Gary references is: > "The classical definition of a unit test in TDD lore is one that doesn't touch the database." So nope. Not confused. ;-) That said the conference he referred to happened in 2000 (http://martinfowler.com/articles/xp2000.html http://martinfowler.com/articles/xp2000.html) and his claim is that Mocks didn't take hold until long after. There's a big swath of time between the rise of the xUnits and the modern Mock fancy he doesn't cover (or provide citations for). Considering the early TDD mantra was red-green-refactor, saying you could do it with all Acceptance tests is a bit of a stretch. I mean tomato tomatoe I guess, but that's certainly not a definition I've ever come across (and it would certainly run afoul of Uncle Bob's 3 rules of TDD). edit: BTW, c2 has a wealth of information on the subject: http://c2.com/cgi/wiki?TestDrivenDevelopment http://c2.com/cgi/wiki?TestDrivenDevelopment Try looking for how many times "design" is mentioned on the page, and in what context. That's how TDD was sold to me. Is your component difficult to test in isolation? Then the design is flawed. One of the big advantages of TDD early on (and it seems to hold true today) is that it helps you write testable code (in Units). BTW, I didn't suggest XP and TDD were the same. XP2000 was a conference. You can believe me or not I guess, but back in the day all you had was your xUnit. Selenium was a twinkle in someone's eye and local databases were not the norm. It was definitely all about the Units (and I think if you read the material of the time the emphasis will be pretty obvious). You might also consider that http://en.wikipedia.org/wiki/Test-driven_development#xUnit_frameworks http://en.wikipedia.org/wiki/Test-driven_development#xUnit_f... pretty strongly associates TDD with Unit Testing.
JeremyMorgan 12y ago
This banter is really getting old and hogging a lot of space. If you don't like TDD, don't do it. If your employer or organization forces you to do it, leave. If you do like it and think it matters, do it. It's really that simple.
- sergiotapia 12y ago
  I'm flagging the TDD he-said-she-said banter submissions. I'm not sure if this is kosher or not, but they feel a little trite no?
  - steveklabnik 12y ago
    You can flag whatever you'd like, but > If you flag something, please don't also comment that you did. http://ycombinator.com/newsguidelines.html http://ycombinator.com/newsguidelines.html
    - gbaygon 12y ago
      > You can flag whatever you'd like, but No, you can't. If you flag articles just because you are tired to see some topic in the front page your flagging habilities will be removed (happened to me).
      - steveklabnik 12y ago
        I just said you could, I didn't say it was a good idea.
        gbaygon 12y ago
        fair enough
      - sergiotapia 12y ago
        I didn't know that, I just unflagged the submission.
  - 12y ago
andyl 12y ago
Fast tests are awesome, but hard to achieve - at least for me. TDD advocates: prove DHH wrong with easy to adopt frameworks and working software, not blog posts or books.
- _pius 12y ago
  TDD advocates: prove DHH wrong with easy to adopt frameworks and working software, not blog posts or books. Gary Bernhardt's about as credible on this as one can be, as he's literally recorded hours of himself building working software with TDD. From the article: These tests are fast enough that I can hit enter (my test-running keystroke) and have a response before I have time to think. It means that the flow of my thoughts never breaks. If you've watched Destroy All Software screencasts, you know that I'll sometimes run tests ten times per minute. All of my screencasts are recorded live after doing many takes to smooth the presentation out, so you're seeing my actual speed, not the result of editing. I understand disagreeing, but responding to this post with "show, not tell" just makes it look like you haven't read the article.
  - gary_bernhardt 12y ago
    Humorously, I already did create a software tool that does isolation automatically. It's called Dingus and it's five years old: https://github.com/garybernhardt/dingus https://github.com/garybernhardt/dingus. It was a terrible idea. It only worked well if you applied the same discipline that you would've had to apply when doing it manually, but it lured you into a false sense of security when using it sloppily. (Dingus is fine when used as a standard mock/stub/spy library, but the automatic isolation features are dangerous and are documented as such in the README.)
  - why-el 12y ago
    Ok, Gary's casts are great, but there aren't near as substantial to conclude anything from them in terms of software architecture.
  - andyl 12y ago
    I subscribed to DAS and watched all the videos. I admire Gary and other TDD advocates. I believe in the power of fast tests. But still, after watching the videos, I can't do what Gary does. I'm not that good, and nobody I have ever worked with is that good either. For the common developer to achieve Gary-like performance, the tools/frameworks/conventions have got to improve. IMHO. In time I expect this will happen, and that DHH will be proven wrong! :-)
blatherard 12y ago
This comports quite well with my personal experience. In late 1999 I happened to pick up XP Explained and set about to try this testing thing. At the time, there wasn't any "Unit Tests == Fast Tests" thing. What we did on our project was have two different types of tests: "Fast" tests and "SlowAndExpensive" tests, that we ran separately. But there wasn't much some fundamental distinction between them. That came a lot later.
avaku 12y ago
Is it me, or there are other people who feel stomach ache when reading this sort of stuff? Why don't just learn algorithms and other computer science "direct" approach by analysing problem and solving it in most officiant way? 100 lines of tests for 50 lines of code for a _catalog_? Really? I'm sorry just reading this brings bad feelings, like those which traders describe when they feel something's wrong with the market...
- matthewmacleod 12y ago
  Why don't just learn algorithms and other computer science "direct" approach by analysing problem and solving it in most officiant way? This doesn't really say anything; your statement amounts to "Why not do it properly, instead of using tests?" - it should be obvious why that's invalid. In particular, unit testing protects against "minor changes with unexpected consequences." This happens all of the time. "analysing problem and solving it" will not make a bit of difference. Do you expect that people who use unit tests are instead "ignoring the problem and not solving it?" A test ratio of 2:1 isn't a big deal. It serves as a record: "Here's a specification of what my code does. You can automatically verify that it does what it says." Think of it as part spec, part test.
- 6d0debc071 12y ago
  > Why don't just learn algorithms and other computer science "direct" approach by analysing problem and solving it in most efficient way? --- The problem often isn't clearly defined. To a certain extent programming is an exploration of the problem until you're in a position to go 'And so...' Add to that, that knowledge of the context and lower level abstraction layer that your algorithm corresponds to is imperfect. (Imagine doing maths where a malicious demon sometimes changes the contents of variables on you according to a set of rules that you don't know.) As such, any algorithm that someone wrote while trying to express the problem and provide a solution may or may not do what they expect it to when given in any particular language on any particular machine. There are a couple of ways to try to address some of the difficulties with that. One is to write code at such a low level that you can be sure of the blocks you're using. Another is to try to climb the abstraction layers using a language with certain safeties built in to try to limit the context you have to be aware of. But there are drawbacks with both of those methods, and testing is a way to deal with the inevitable oversights involved. (It's also good in getting people to more tightly define the problem in the first place for you. Acceptance tests are a wonderful thing sometimes.)
dasil003 12y ago
The elephant in the room here is the bugs that crop up in the interfaces between units. Depending on your problem domain, it may be easier or harder to create interfaces between all these TDD'd, isolated modules. There's certainly value in a spec as isolated component documentation, but I'd argue there's more value in specs for regressions, particularly when refactoring. This is especially the case in dynamic languages like Ruby which require tests to avoid the most basic runtime errors. The answer to this conundrum is often: that's the job of the integration tests. Okay, but integration tests are slow, so you'll never test all the permutations of the interactions of the components. DHH said during the keynote, "it's easy to make your tests fast when they don't test anything". Of course that's hyperbole, but there's a kernel of truth to be investigated there. When working with something as complex as an ORM like ActiveRecord, isolating the business logic and using the ORM strictly for persistence may allow for fast tests, but you still run the risk of bugs creeping in on the interface because of some assumption about or change in the way ActiveRecord works between versions or whatever. That's why, as ugly and slow as Rails unit tests are, they are a simplifying compromise that strikes a balance between the theoretical ideal of a unit test and an integration test. ActiveRecord itself is this way too, in that often times the business logic just isn't complex enough to warrant the complexity of separating persistence from domain logic. As much as DHH may be talking out his ass without really having ever grokked TDD, I don't think his complaints are completely without merit.
- gary_bernhardt 12y ago
  DHH's complaints certainly aren't without merit, but they are naively formed. I did a conference talk called "Boundaries" on this topic, and how a particular type of disciplined functional style can mitigate it to a large extent (https://www.destroyallsoftware.com/talks/boundaries https://www.destroyallsoftware.com/talks/boundaries). I think that we can probably have (most of) our cake and eat it to, just as we have so many times before: microprocessors reliably perform computations; TCP reliably delivers over unreliable networks; etc. But we're not going to get there by throwing up our hands.
  - ritchiea 12y ago
    When you say naively formed, what does that mean exactly? A lot of your post was dedicated to correcting DHH's TDD history. I appreciate the hard work and attention to detail but I don't actually think when the idea of ultra-fast tests originated has much to do with DHH's complaints. You also spend time in your post talking about how much you value the tight feedback loop between your tests and your code and how your tests being ultra fast helps that. That's great! That isn't really a response to DHH though, that's explaining why your method works for you. Which is a valuable contribution to the discussion, but not really a critique. You also bring up DHH's lack of theoretical computer science knowledge. I don't have the slightest idea what that has to do with the value of TDD, can you explain that? I guess what I'm trying to get at here is that you didn't really propose a counter argument to DHH here, you said his historical knowledge is wrong, he doesn't know computer science and you get a lot of value from fast tests. The core of DHH's argument is that the metrics that are being touted to measure test suites are not metrics that actually help us write good code or code that is reliably well tested. It would be great to hear your direct responses to those arguments.
sync 12y ago
> "I want my feedback to be so fast that I can't think before it shows up. If I can think, then I'll sometimes lose attention, and I don't want to lose attention." Don't you want to think while programming? I feel like that's practically all I do -- I spend most of my time thinking about a problem and very little actually writing code.
- maaaats 12y ago
  Ironic that your reply to a piece about strawmen is nothing but a strawman. Of course he is not meaning what you're saying, he's talking about context switching. If I have to wait 15 seconds for my tests to run, I start doing something else and my thought process around the problem is gone.
  - sync 12y ago
    I don't understand. Are you not continuing to think about the problem (or the next problem) while tests are running, regardless of how long they take?
    - thetrb 12y ago
      If you move on to the next problem, then your test finishes and forces you to move back to the original problem then you just had to do 2 context switches. If you never had to wait for the test result then you could have eliminated those.
- codereflection 12y ago
  If you'd ever seen Gary code, you'd know that he is a very quick thinker (almost nonhuman like) that requires very fast feedback loop to keep up the pace.
  - Retric 12y ago
    Your not watching him think, it's a few takes before he get's the presentation the way he wants. Edit: it's a little misleading, but "All of my screencasts are recorded live after doing many takes to smooth the presentation out"
desireco42 12y ago
I have great respect for Gary Bernhardt. I think he is missing DHH point here, which as I understand it, that: tests can be an end in itself I think this is reasonable argument on DHH part that we want to spend more time writing actual code, instead of going through movements of TDD. That doesn't mean TDD is not valuable.
tieTYT 12y ago
> Classical TDD does not involve mocking or other forms of synthetic isolation by definition. We even use the term "classical TDD" to mean "TDD without isolation". I wish there was a source on this paragraph. According to this talk that often refers to Kent Beck's book ( the talk: http://vimeo.com/68375232 http://vimeo.com/68375232 / a good summary of the talk, in text: https://groups.google.com/forum/#!topic/growing-object-oriented-software/Hxp8cVfE4gI https://groups.google.com/forum/#!topic/growing-object-orien... ), you are supposed to isolate from the database.
- gary_bernhardt 12y ago
  Well, like I say in the post, mocks didn't exist back then, so they couldn't have been mocking in the sense that we are now. I wasn't there, but I believe it's true that in some cases they were doing what we'd now consider "fakes", which are simplified replacements for production dependencies that have minimal working implementations. (An in-memory database is an example.) Fakes get you around the question of database integration, but not the question of integration in general. You'd have to create a fake of every class in the system for that, in which case you probably just re-invented mocks and/or stubs. I probably could've more explicitly called out the fact that DHH is obsessing over database integration when it's just a special case of what isolated testers are actually worried about, which is integration in general. It was already a long post, though.
  - ssmoot 12y ago
    At least in .NET land stubs were very common (several libraries actually generated them for you based on your Interfaces). Nice post BTW.
  - tieTYT 12y ago
    Thanks for the reply and for the article. I took your quote out of context. I'm not very interested in DHH's mis/understanding of TDD. I'm more interested in my understanding of TDD. I'm trying to figure out if I'm doing TDD correctly or incorrectly when I isolate the database. Whether it's done through a mock or a fake or whatever is not that important to me. The thing I don't like about TDD is I never know if I'm doing it correctly. There's always that No True Scotsman fallacy^1 situation. "Oh, you tried TDD and it didn't work out for you? If you were doing real TDD it would have worked out." And I say this as a person who has watched three of Uncle Bob's videos on TDD that you have to pay for: http://cleancoders.com/episode/clean-code-episode-6-p1/show http://cleancoders.com/episode/clean-code-episode-6-p1/show http://cleancoders.com/episode/clean-code-episode-6-p2/show http://cleancoders.com/episode/clean-code-episode-6-p2/show http://cleancoders.com/episode/clean-code-episode-19-p1/show http://cleancoders.com/episode/clean-code-episode-19-p1/show Sorry for hijacking the topic. I've been trying TDD for years, and I never know if I'm doing it correctly. I saw this line and it made me question it yet again. ^1: http://rationalwiki.org/wiki/No_True_Scotsman http://rationalwiki.org/wiki/No_True_Scotsman EDIT: Sorry, I commented as I read. Looks like you're going to explain it in the article.
jonahx 12y ago
Gary, I've been hoping that you would write an intelligent response to DHH's latest articles and talk ever since I saw them, thank you for doing it. I suspect that his claims of TDD isolation damaging the integrity of designs are mostly red-herrings too. I know they are for me personally -- in fact, the opposite is true. I'd love to see you address this issue as well. In particular, your thoughts as they relate the technique of moving logic completely out of controllers (as you do in raptor). This technique seems to infuriate DHH in the context of rails, while to me it seems a far superior design.
- matthewmacleod 12y ago
  I suspect that his claims of TDD isolation damaging the integrity of designs are mostly red-herrings too. I know they are for me personally -- in fact, the opposite is true. I'd be interested to see more examples of that. In my experience, this sort of warning genuinely isn't a red-herring - I've worked on several projects that have been seriously creaking under the weight of test suites, and in which the desire to isolate units in particular has led to both poor code architecture, and ironically poor tests. In particular, your thoughts as they relate the technique of moving logic completely out of controllers (as you do in raptor). This technique seems to infuriate DHH in the context of rails, while to me it seems a far superior design. Raptor's design is interesting, but I dispute that the entire "C" in "MVC" is worthless. I agree that such is often misused, but something like Raptor essentially pushes the logic that it's OK to have in a Rails controller into the router, and I'm not sure that would actually be scalable in practice.
  - gary_bernhardt 12y ago
    I've experienced (and created) exactly those weighty, oppressive test suites. I think that they're probably more a symptom of us collectively learning to test than anything else. Even today, there are very few people who are experts at test isolation; five years ago there were almost none. It's tempting to give up in situations like that, but I like to think back to "GOTO Considered Harmful". It was contentious at the time! There were people who literally thought that you couldn't build complex software systems without GOTO. It's a reminder that we always underestimate how good we can get at something, and how much freedom we'll have after adopting a constraint.
jgon 12y ago
On the one hand this article does at least provide citations for some of the timelines involved. On the other hand it makes me throw my hands up when I see some of the examples provided. 100 lines of code to test 50 lines of code, which implement a catalog is pretty par for the course it seems in TDD advocacy. This could honestly be me venting my current frustrations, but I get tired of this type of advocacy, also often seen in the functional world, of proving how great something is by using the most trivial possible example, in a domain squarely in your technique's wheelhouse. 100 lines of code runs in 0.24s? What is this, I don't even. Your functional algorithm is incredibly elegant at implementing the fibonacci sequence? Awesome. But are these things honestly the bulk of what people do? Is testing that you can put objects in a catalog, access them, and remove them most of what people do and need assurances on? Am I the only person who has spent most of their career working on software with codebases measured in the millions of lines, with significant user interfaces as well as significant technical domain knowledge embedded in them? I know that TDD says if our objects have many collaborators we are "doing it wrong" but honestly how far do you break down something without it blowing up into a million classes and a ton of code? How do you get around the fact that a button press can set off a numerical simulation (actually these are super testable and I support that), several trips to the database, multiple changes to the user interface, all in the face of dozens of constraints at all levels of the program to ultimately end up with the graphical representation of chemical pressure that the user wants? Especially when every moving piece in that calculation is supposed to be tested apparently in independent units? I would estimate that the bulk of code that I have experienced in my career relied on multiple other objects to be effective. Do I just mock these out and pay the price of keeping those mocks in sync with the classes they are imitating? Do I then have my test be tightly coupled to the internals of the function being tested, making sure this mock is called in this way, this many times? Do I rewrite things such that every function takes its collaborators as arguments and returns the results I am looking for? What happens when each of those collaborators is itself a giant series of other stateful objects or at least provides access to a highly complex piece of state which is a pain in the ass to setup? I ask these questions seriously, because on the one hand TDD advocates keep telling me I am not professional if I don't follow their practices, but on the other the details of how to do so in the face of real-world constraints (not 50 line data-containers) are always somehow left out of the discussion. I want to believe, I do, it just gets hard to sometimes.
raverbashing 12y ago
Of course creating mocks or circumventing the database may create more bugs (or hide more) than just wait the couple of minutes for the real thing Adding another failure point is exactly what the name says: another possibility of error or bug Sure, we can talk about the "one true way" of doing tests, how to make hundreds of tests run in a short time, etc (the fact that TDD insists in creating one test for each tiny thing goes against it, btw) but yeah, I prefer spending more time solving the problem, and not testing around it.
- gary_bernhardt 12y ago
  TDD is a way of writing tests, not a prescription about when to write them. I explicitly say in the post that I only do TDD 75% of the time for web apps and more like 50% for other code.
matthewmacleod 12y ago
I think there is a good point here, in that a very tight TDD loop requires fast tests. Nothing's untrue about that, and if you've worked with a set of speedy tests I'm sure you know it can be quite efficient, and indeed it can be quite a different way of working. Ultimately I don't really think that this is particularly in conflict with what DHH was talking about. A tight TDD loop might work for some developers, but if you get to the stage where you are making substantial architectural changes to enable this, then it looks like a problem. And I've seen a lot of this first hand. FWIW, I take your point that things like Spring aren't ideal solutions. But with a bit of care they can help offer the best of both worlds; I've got Guard and Spring working together on the Rails project I currently have open. Looking at an equivalent model (100 LOC, 200 test LOC), it runs without database isolation in less than 400ms, which is less time than it takes me to switch to my terminal. It's really great, and I don't have to worry about isolation. TLDR; there are compromise solutions possible. Maybe not suited for everyone, but in some cases better than going too far with "design for testability."
deleted 12y ago
[deleted]
sandal 12y ago
The critical flaw in this post is that Gary is not refuting what DHH said. Gary makes this claim: > You finally get to see what's really going on. David's tests run in a few minutes, and he's fine with that. > I'm not fine with that. A lot of other people are not fine with that. But what DHH actually said is this: > You might think, well, that's pretty fast for a whole suite, but I still wouldn't want to wait 80 seconds every time I make a single change to my model, and want to test that. Of course not! Why on earth would you run your entire test harness for every single line change in a particular model? If you have so little confidence in the locality of your changes, the tests are indeed telling you that the system has overly high coupling. and this: > These days I can run the entire test suite for our Person model — 52 cases, 111 assertions — in just under 4 seconds from start to finish. Plenty fast enough for a great feedback cycle! Using a workflow like Gary's, there's an argument to be made that 4 seconds is not acceptable, and this is why we want single files that can run in a few milliseconds. However, that's not the only possible way of running tests, and the difference between 4 seconds and 300ms for the feedback you're actually interested is massively different than 300ms vs "a few minutes". For a post that calls DHH out on a strawman, this is in itself a great example of one.
- gary_bernhardt 12y ago
  Yes, I focus on my per-file runtime in the post, and I mention David's suite runtime in one sentence at the beginning. They are not meant to be compared. David's file runtime is four seconds. This is unacceptable to me. This is unacceptable to other people who replied to your tweets. This would double the length of my high-speed TDD loop, which would make those portions of my TDD process take twice as long. Yes, it would've been clearer for me to specifically address both suite runtimes and both unit runtimes. You know what else would've been clearer? All of the 2,000 words or so that I deleted from that post while I was editing it down into its final form. This is just how writing works. I don't think that it's misleading as written. Of course, I've already told you, on Twitter, exactly my reasons for rejecting both four-minute suites and four-second test files. They're not in the post, but you know the reasons. You know that I wasn't selectively attacking a subset of his argument, because you know that I do have an answer for test file runtime. And yet, for some reason, here we are! (For anyone reading this later, the tweets in question are gone. Lately I've been deleting all replies, as well as trivial non-replies, for Reasons.)
nshepperd 12y ago
> That file is going to be a plain old Ruby class: not a Rails model, controller, etc. Isolating those would lead to pain, as anyone who's tried to do it knows. I keep models very simple and allow them to integrate with the database; I keep controllers very simple and generally don't unit test them at all, although I do integration test them. This is what I was thinking of when I read DHH's post. Mock objects and indirection and all that certainly add complexity, but if your "business logic" is all in pure functions, or at least self-contained source files that don't do IO (and don't use "frameworks"), you don't need mocks to test it. Better to keep your glue code simple and do integration tests, while unit testing your actual logic, than to shoehorn extra indirection for mocking into complicated glue code. I gather that DHH complained because he had been doing the latter, and found that it sucked.
jongraehl 12y ago
I'd believe the same endorsements from someone who makes their living primarily off working code, rather than selling edu. material promoting test-heavy coding. That said, rapid "all's still well" feedback is awesome when you can get it.
GFK_of_xmaspast 12y ago
The best testing is the kind you use and monitor.

177 points by gary_bernhardt 12y ago

All of these arguments seem like they are overly prescriptive. Individuals should use what works best for their needs. TDD may work wonders for me today on one project, and then it could be horrible tomorrow on the next project. My co-worker might have the inverse. Stop arguing over how other people create software; Ship Code instead.

JeremyMorgan 12y ago
You said it better than me. No reason to get religious over it. When I'm the architect on a project, I do it in some cases, and other times I don't. I'm not over concerned what everyone else is doing in this regard.
GuiA 12y ago
> Individuals should use what works best for their needs Maybe rather: teams should use what works best for their needs ? EDIT: whee, downvotes. Apparently HN'ers work on teams that ship software and where a developer uses kanban, another TDD, another waterfall, another scrum, and another only commits code when the 8 planets are aligned.
- eclark 12y ago
  TDD vs Testing after is a personal choice apart from how the team set goals. That is to say you can be on a waterfall schedule and still practice TDD. As long as code shows up to code review with good tests, I don't personally care if the tests were written before or after.
- jasonlotito 12y ago
  > whee, downvotes. I'm going to take a stab at the down votes, as I could easily understand why someone might. First, consider your original comment was pedantic and offered little in the way of original insight. It would be like me replying to you and saying: "Maybe rather: teams should use hwat works best for their needs per project?" What does that really add? Basically, it added little to the conversation. Heck, your edit complaining about the down votes is longer than your comment.
- MartinCron 12y ago
  Maybe you are being down voted by fans of the former-planet Pluto? Sensitive bunch, those guys.
ericHosick 12y ago
> Stop arguing over how other people create software; Ship Code instead. Electrical engineering, mechanical engineering, architectural design and the medical profession, to name a few, have bodies of knowledge they are required to use. Is it really a good idea for software developers to say "stop arguing over how other people create software; Ship Code instead" considering we don't have any industry standard bodies of knowledge?
- lugg 12y ago
  Considering we don't have any industry standard bodies of knowledge, yes it probably is a good idea, because we don't have anyone who can lay out a definitive answer on many of our preference based arguments. i.e. these arguments will never end if there is no definitive answer, or body to pick one.
  - gary_bernhardt 12y ago
    Industry standard bodies of knowledge arise from people doing, then talking about what they did, then doing some more, then talking some more. At no point did a God of Electrical Engineering hand down tablets.
    - lugg 12y ago
      Very good point. I stand corrected.
  - Silhouette 12y ago
    There is at least one serious effort underway to collect (or perhaps, catalogue) such a body of knowledge: http://www.computer.org/portal/web/swebok http://www.computer.org/portal/web/swebok I think SWEBOK is interesting both as a demonstration of how far we have to go before we get anywhere near the standards of established engineering fields, and as a demonstration of how much we have already collectively figured out, even if most of us aren't familiar with more than a small fraction of what is out there.
ebiester 12y ago
I believe that it's a useful conversation to have -- if we are all just trying to ship code, when are we going to have the discussion of what makes code readable? What makes code reliable? What makes code quicker to deliver? Further, as a automated test advocate (whether or not it's TDD or unit or mocks or stubs or whatever flavor you like) I want to walk into projects where I don't have to deal with changing code without having a good and fast test base to minimize the potential defects. If TDD gets us to that point, I'm all for it. I don't care how you create software unless I have to use it or inherit it.

ssmoot 12y ago

> "Second, that sentence is false. Isolation from the database, or anything else, is generally done with mocks, but mocks didn't even exist when TDD was rediscovered by Kent Beck in 1994-1995." See, that's not how I remember it. Stubs implementing the Interfaces you wrote (because Composition over Inheritance) was the common solution. AR's idea of a Unit Test has always been wrong to my mind. They aren't Unit tests. They're Integration tests. This only really became an issue with the rise of Rails and the fact that you couldn't really write Unit tests for an ActiveRecord. At least that's how I remember it.

steveklabnik 12y ago
Considering Gary provides citations for his timeline, I'm inclined to believe his over your memory. That said, I think you're confusing _unit_ tests and _TDD_. Unit tests imply that you test a single isolation, TDD implies that you write your tests before you write your implementation. You can TDD unit tests, but you don't have to, and you can do all-acceptance TDD if you want.
- yeukhon 12y ago
  Exactly. I think most of the TDD probably come to some tutorials which test addition a+b of that sort and demonstrate how to test "ah there's logical error! how incredible TDD helps us to identify error early on!". I do some TDD from time to time, but outside of my textbook algorithm, writing pure unit test which involves using mocks is really tough and time-consuming (unless someone has built a test driver for you to use throughout the whole development). In any, if you ever do curl http://localhost:port/mynewapp/page1 http://localhost:port/mynewapp/page1 that's already testing and is doing functional testing. If you write that down before you start writing a piece of code, you are doing TDD to test whether your app will return 200 or not.
- ssmoot 12y ago
  The quote Gary references is: > "The classical definition of a unit test in TDD lore is one that doesn't touch the database." So nope. Not confused. ;-) That said the conference he referred to happened in 2000 (http://martinfowler.com/articles/xp2000.html http://martinfowler.com/articles/xp2000.html) and his claim is that Mocks didn't take hold until long after. There's a big swath of time between the rise of the xUnits and the modern Mock fancy he doesn't cover (or provide citations for). Considering the early TDD mantra was red-green-refactor, saying you could do it with all Acceptance tests is a bit of a stretch. I mean tomato tomatoe I guess, but that's certainly not a definition I've ever come across (and it would certainly run afoul of Uncle Bob's 3 rules of TDD). edit: BTW, c2 has a wealth of information on the subject: http://c2.com/cgi/wiki?TestDrivenDevelopment http://c2.com/cgi/wiki?TestDrivenDevelopment Try looking for how many times "design" is mentioned on the page, and in what context. That's how TDD was sold to me. Is your component difficult to test in isolation? Then the design is flawed. One of the big advantages of TDD early on (and it seems to hold true today) is that it helps you write testable code (in Units). BTW, I didn't suggest XP and TDD were the same. XP2000 was a conference. You can believe me or not I guess, but back in the day all you had was your xUnit. Selenium was a twinkle in someone's eye and local databases were not the norm. It was definitely all about the Units (and I think if you read the material of the time the emphasis will be pretty obvious). You might also consider that http://en.wikipedia.org/wiki/Test-driven_development#xUnit_frameworks http://en.wikipedia.org/wiki/Test-driven_development#xUnit_f... pretty strongly associates TDD with Unit Testing.

JeremyMorgan 12y ago

This banter is really getting old and hogging a lot of space. If you don't like TDD, don't do it. If your employer or organization forces you to do it, leave. If you do like it and think it matters, do it. It's really that simple.

sergiotapia 12y ago
I'm flagging the TDD he-said-she-said banter submissions. I'm not sure if this is kosher or not, but they feel a little trite no?
- steveklabnik 12y ago
  You can flag whatever you'd like, but > If you flag something, please don't also comment that you did. http://ycombinator.com/newsguidelines.html http://ycombinator.com/newsguidelines.html
  - gbaygon 12y ago
    > You can flag whatever you'd like, but No, you can't. If you flag articles just because you are tired to see some topic in the front page your flagging habilities will be removed (happened to me).
    - steveklabnik 12y ago
      I just said you could, I didn't say it was a good idea.
      - gbaygon 12y ago
        fair enough
    - sergiotapia 12y ago
      I didn't know that, I just unflagged the submission.
- 12y ago

andyl 12y ago

Fast tests are awesome, but hard to achieve - at least for me. TDD advocates: prove DHH wrong with easy to adopt frameworks and working software, not blog posts or books.

_pius 12y ago
TDD advocates: prove DHH wrong with easy to adopt frameworks and working software, not blog posts or books. Gary Bernhardt's about as credible on this as one can be, as he's literally recorded hours of himself building working software with TDD. From the article: These tests are fast enough that I can hit enter (my test-running keystroke) and have a response before I have time to think. It means that the flow of my thoughts never breaks. If you've watched Destroy All Software screencasts, you know that I'll sometimes run tests ten times per minute. All of my screencasts are recorded live after doing many takes to smooth the presentation out, so you're seeing my actual speed, not the result of editing. I understand disagreeing, but responding to this post with "show, not tell" just makes it look like you haven't read the article.
- gary_bernhardt 12y ago
  Humorously, I already did create a software tool that does isolation automatically. It's called Dingus and it's five years old: https://github.com/garybernhardt/dingus https://github.com/garybernhardt/dingus. It was a terrible idea. It only worked well if you applied the same discipline that you would've had to apply when doing it manually, but it lured you into a false sense of security when using it sloppily. (Dingus is fine when used as a standard mock/stub/spy library, but the automatic isolation features are dangerous and are documented as such in the README.)
- why-el 12y ago
  Ok, Gary's casts are great, but there aren't near as substantial to conclude anything from them in terms of software architecture.
- andyl 12y ago
  I subscribed to DAS and watched all the videos. I admire Gary and other TDD advocates. I believe in the power of fast tests. But still, after watching the videos, I can't do what Gary does. I'm not that good, and nobody I have ever worked with is that good either. For the common developer to achieve Gary-like performance, the tools/frameworks/conventions have got to improve. IMHO. In time I expect this will happen, and that DHH will be proven wrong! :-)

blatherard 12y ago

This comports quite well with my personal experience. In late 1999 I happened to pick up XP Explained and set about to try this testing thing. At the time, there wasn't any "Unit Tests == Fast Tests" thing. What we did on our project was have two different types of tests: "Fast" tests and "SlowAndExpensive" tests, that we ran separately. But there wasn't much some fundamental distinction between them. That came a lot later.

avaku 12y ago

Is it me, or there are other people who feel stomach ache when reading this sort of stuff? Why don't just learn algorithms and other computer science "direct" approach by analysing problem and solving it in most officiant way? 100 lines of tests for 50 lines of code for a _catalog_? Really? I'm sorry just reading this brings bad feelings, like those which traders describe when they feel something's wrong with the market...

matthewmacleod 12y ago
Why don't just learn algorithms and other computer science "direct" approach by analysing problem and solving it in most officiant way? This doesn't really say anything; your statement amounts to "Why not do it properly, instead of using tests?" - it should be obvious why that's invalid. In particular, unit testing protects against "minor changes with unexpected consequences." This happens all of the time. "analysing problem and solving it" will not make a bit of difference. Do you expect that people who use unit tests are instead "ignoring the problem and not solving it?" A test ratio of 2:1 isn't a big deal. It serves as a record: "Here's a specification of what my code does. You can automatically verify that it does what it says." Think of it as part spec, part test.
6d0debc071 12y ago
> Why don't just learn algorithms and other computer science "direct" approach by analysing problem and solving it in most efficient way? --- The problem often isn't clearly defined. To a certain extent programming is an exploration of the problem until you're in a position to go 'And so...' Add to that, that knowledge of the context and lower level abstraction layer that your algorithm corresponds to is imperfect. (Imagine doing maths where a malicious demon sometimes changes the contents of variables on you according to a set of rules that you don't know.) As such, any algorithm that someone wrote while trying to express the problem and provide a solution may or may not do what they expect it to when given in any particular language on any particular machine. There are a couple of ways to try to address some of the difficulties with that. One is to write code at such a low level that you can be sure of the blocks you're using. Another is to try to climb the abstraction layers using a language with certain safeties built in to try to limit the context you have to be aware of. But there are drawbacks with both of those methods, and testing is a way to deal with the inevitable oversights involved. (It's also good in getting people to more tightly define the problem in the first place for you. Acceptance tests are a wonderful thing sometimes.)

dasil003 12y ago

The elephant in the room here is the bugs that crop up in the interfaces between units. Depending on your problem domain, it may be easier or harder to create interfaces between all these TDD'd, isolated modules. There's certainly value in a spec as isolated component documentation, but I'd argue there's more value in specs for regressions, particularly when refactoring. This is especially the case in dynamic languages like Ruby which require tests to avoid the most basic runtime errors. The answer to this conundrum is often: that's the job of the integration tests. Okay, but integration tests are slow, so you'll never test all the permutations of the interactions of the components. DHH said during the keynote, "it's easy to make your tests fast when they don't test anything". Of course that's hyperbole, but there's a kernel of truth to be investigated there. When working with something as complex as an ORM like ActiveRecord, isolating the business logic and using the ORM strictly for persistence may allow for fast tests, but you still run the risk of bugs creeping in on the interface because of some assumption about or change in the way ActiveRecord works between versions or whatever. That's why, as ugly and slow as Rails unit tests are, they are a simplifying compromise that strikes a balance between the theoretical ideal of a unit test and an integration test. ActiveRecord itself is this way too, in that often times the business logic just isn't complex enough to warrant the complexity of separating persistence from domain logic. As much as DHH may be talking out his ass without really having ever grokked TDD, I don't think his complaints are completely without merit.

gary_bernhardt 12y ago
DHH's complaints certainly aren't without merit, but they are naively formed. I did a conference talk called "Boundaries" on this topic, and how a particular type of disciplined functional style can mitigate it to a large extent (https://www.destroyallsoftware.com/talks/boundaries https://www.destroyallsoftware.com/talks/boundaries). I think that we can probably have (most of) our cake and eat it to, just as we have so many times before: microprocessors reliably perform computations; TCP reliably delivers over unreliable networks; etc. But we're not going to get there by throwing up our hands.
- ritchiea 12y ago
  When you say naively formed, what does that mean exactly? A lot of your post was dedicated to correcting DHH's TDD history. I appreciate the hard work and attention to detail but I don't actually think when the idea of ultra-fast tests originated has much to do with DHH's complaints. You also spend time in your post talking about how much you value the tight feedback loop between your tests and your code and how your tests being ultra fast helps that. That's great! That isn't really a response to DHH though, that's explaining why your method works for you. Which is a valuable contribution to the discussion, but not really a critique. You also bring up DHH's lack of theoretical computer science knowledge. I don't have the slightest idea what that has to do with the value of TDD, can you explain that? I guess what I'm trying to get at here is that you didn't really propose a counter argument to DHH here, you said his historical knowledge is wrong, he doesn't know computer science and you get a lot of value from fast tests. The core of DHH's argument is that the metrics that are being touted to measure test suites are not metrics that actually help us write good code or code that is reliably well tested. It would be great to hear your direct responses to those arguments.

sync 12y ago

> "I want my feedback to be so fast that I can't think before it shows up. If I can think, then I'll sometimes lose attention, and I don't want to lose attention." Don't you want to think while programming? I feel like that's practically all I do -- I spend most of my time thinking about a problem and very little actually writing code.

maaaats 12y ago
Ironic that your reply to a piece about strawmen is nothing but a strawman. Of course he is not meaning what you're saying, he's talking about context switching. If I have to wait 15 seconds for my tests to run, I start doing something else and my thought process around the problem is gone.
- sync 12y ago
  I don't understand. Are you not continuing to think about the problem (or the next problem) while tests are running, regardless of how long they take?
  - thetrb 12y ago
    If you move on to the next problem, then your test finishes and forces you to move back to the original problem then you just had to do 2 context switches. If you never had to wait for the test result then you could have eliminated those.
codereflection 12y ago
If you'd ever seen Gary code, you'd know that he is a very quick thinker (almost nonhuman like) that requires very fast feedback loop to keep up the pace.
- Retric 12y ago
  Your not watching him think, it's a few takes before he get's the presentation the way he wants. Edit: it's a little misleading, but "All of my screencasts are recorded live after doing many takes to smooth the presentation out"

desireco42 12y ago

I have great respect for Gary Bernhardt. I think he is missing DHH point here, which as I understand it, that: tests can be an end in itself I think this is reasonable argument on DHH part that we want to spend more time writing actual code, instead of going through movements of TDD. That doesn't mean TDD is not valuable.

tieTYT 12y ago

> Classical TDD does not involve mocking or other forms of synthetic isolation by definition. We even use the term "classical TDD" to mean "TDD without isolation". I wish there was a source on this paragraph. According to this talk that often refers to Kent Beck's book ( the talk: http://vimeo.com/68375232 http://vimeo.com/68375232 / a good summary of the talk, in text: https://groups.google.com/forum/#!topic/growing-object-oriented-software/Hxp8cVfE4gI https://groups.google.com/forum/#!topic/growing-object-orien... ), you are supposed to isolate from the database.

gary_bernhardt 12y ago
Well, like I say in the post, mocks didn't exist back then, so they couldn't have been mocking in the sense that we are now. I wasn't there, but I believe it's true that in some cases they were doing what we'd now consider "fakes", which are simplified replacements for production dependencies that have minimal working implementations. (An in-memory database is an example.) Fakes get you around the question of database integration, but not the question of integration in general. You'd have to create a fake of every class in the system for that, in which case you probably just re-invented mocks and/or stubs. I probably could've more explicitly called out the fact that DHH is obsessing over database integration when it's just a special case of what isolated testers are actually worried about, which is integration in general. It was already a long post, though.
- ssmoot 12y ago
  At least in .NET land stubs were very common (several libraries actually generated them for you based on your Interfaces). Nice post BTW.
- tieTYT 12y ago
  Thanks for the reply and for the article. I took your quote out of context. I'm not very interested in DHH's mis/understanding of TDD. I'm more interested in my understanding of TDD. I'm trying to figure out if I'm doing TDD correctly or incorrectly when I isolate the database. Whether it's done through a mock or a fake or whatever is not that important to me. The thing I don't like about TDD is I never know if I'm doing it correctly. There's always that No True Scotsman fallacy^1 situation. "Oh, you tried TDD and it didn't work out for you? If you were doing real TDD it would have worked out." And I say this as a person who has watched three of Uncle Bob's videos on TDD that you have to pay for: http://cleancoders.com/episode/clean-code-episode-6-p1/show http://cleancoders.com/episode/clean-code-episode-6-p1/show http://cleancoders.com/episode/clean-code-episode-6-p2/show http://cleancoders.com/episode/clean-code-episode-6-p2/show http://cleancoders.com/episode/clean-code-episode-19-p1/show http://cleancoders.com/episode/clean-code-episode-19-p1/show Sorry for hijacking the topic. I've been trying TDD for years, and I never know if I'm doing it correctly. I saw this line and it made me question it yet again. ^1: http://rationalwiki.org/wiki/No_True_Scotsman http://rationalwiki.org/wiki/No_True_Scotsman EDIT: Sorry, I commented as I read. Looks like you're going to explain it in the article.

jonahx 12y ago

Gary, I've been hoping that you would write an intelligent response to DHH's latest articles and talk ever since I saw them, thank you for doing it. I suspect that his claims of TDD isolation damaging the integrity of designs are mostly red-herrings too. I know they are for me personally -- in fact, the opposite is true. I'd love to see you address this issue as well. In particular, your thoughts as they relate the technique of moving logic completely out of controllers (as you do in raptor). This technique seems to infuriate DHH in the context of rails, while to me it seems a far superior design.

matthewmacleod 12y ago
I suspect that his claims of TDD isolation damaging the integrity of designs are mostly red-herrings too. I know they are for me personally -- in fact, the opposite is true. I'd be interested to see more examples of that. In my experience, this sort of warning genuinely isn't a red-herring - I've worked on several projects that have been seriously creaking under the weight of test suites, and in which the desire to isolate units in particular has led to both poor code architecture, and ironically poor tests. In particular, your thoughts as they relate the technique of moving logic completely out of controllers (as you do in raptor). This technique seems to infuriate DHH in the context of rails, while to me it seems a far superior design. Raptor's design is interesting, but I dispute that the entire "C" in "MVC" is worthless. I agree that such is often misused, but something like Raptor essentially pushes the logic that it's OK to have in a Rails controller into the router, and I'm not sure that would actually be scalable in practice.
- gary_bernhardt 12y ago
  I've experienced (and created) exactly those weighty, oppressive test suites. I think that they're probably more a symptom of us collectively learning to test than anything else. Even today, there are very few people who are experts at test isolation; five years ago there were almost none. It's tempting to give up in situations like that, but I like to think back to "GOTO Considered Harmful". It was contentious at the time! There were people who literally thought that you couldn't build complex software systems without GOTO. It's a reminder that we always underestimate how good we can get at something, and how much freedom we'll have after adopting a constraint.

jgon 12y ago

On the one hand this article does at least provide citations for some of the timelines involved. On the other hand it makes me throw my hands up when I see some of the examples provided. 100 lines of code to test 50 lines of code, which implement a catalog is pretty par for the course it seems in TDD advocacy. This could honestly be me venting my current frustrations, but I get tired of this type of advocacy, also often seen in the functional world, of proving how great something is by using the most trivial possible example, in a domain squarely in your technique's wheelhouse. 100 lines of code runs in 0.24s? What is this, I don't even. Your functional algorithm is incredibly elegant at implementing the fibonacci sequence? Awesome. But are these things honestly the bulk of what people do? Is testing that you can put objects in a catalog, access them, and remove them most of what people do and need assurances on? Am I the only person who has spent most of their career working on software with codebases measured in the millions of lines, with significant user interfaces as well as significant technical domain knowledge embedded in them? I know that TDD says if our objects have many collaborators we are "doing it wrong" but honestly how far do you break down something without it blowing up into a million classes and a ton of code? How do you get around the fact that a button press can set off a numerical simulation (actually these are super testable and I support that), several trips to the database, multiple changes to the user interface, all in the face of dozens of constraints at all levels of the program to ultimately end up with the graphical representation of chemical pressure that the user wants? Especially when every moving piece in that calculation is supposed to be tested apparently in independent units? I would estimate that the bulk of code that I have experienced in my career relied on multiple other objects to be effective. Do I just mock these out and pay the price of keeping those mocks in sync with the classes they are imitating? Do I then have my test be tightly coupled to the internals of the function being tested, making sure this mock is called in this way, this many times? Do I rewrite things such that every function takes its collaborators as arguments and returns the results I am looking for? What happens when each of those collaborators is itself a giant series of other stateful objects or at least provides access to a highly complex piece of state which is a pain in the ass to setup? I ask these questions seriously, because on the one hand TDD advocates keep telling me I am not professional if I don't follow their practices, but on the other the details of how to do so in the face of real-world constraints (not 50 line data-containers) are always somehow left out of the discussion. I want to believe, I do, it just gets hard to sometimes.

raverbashing 12y ago

Of course creating mocks or circumventing the database may create more bugs (or hide more) than just wait the couple of minutes for the real thing Adding another failure point is exactly what the name says: another possibility of error or bug Sure, we can talk about the "one true way" of doing tests, how to make hundreds of tests run in a short time, etc (the fact that TDD insists in creating one test for each tiny thing goes against it, btw) but yeah, I prefer spending more time solving the problem, and not testing around it.

gary_bernhardt 12y ago
TDD is a way of writing tests, not a prescription about when to write them. I explicitly say in the post that I only do TDD 75% of the time for web apps and more like 50% for other code.

matthewmacleod 12y ago

I think there is a good point here, in that a very tight TDD loop requires fast tests. Nothing's untrue about that, and if you've worked with a set of speedy tests I'm sure you know it can be quite efficient, and indeed it can be quite a different way of working. Ultimately I don't really think that this is particularly in conflict with what DHH was talking about. A tight TDD loop might work for some developers, but if you get to the stage where you are making substantial architectural changes to enable this, then it looks like a problem. And I've seen a lot of this first hand. FWIW, I take your point that things like Spring aren't ideal solutions. But with a bit of care they can help offer the best of both worlds; I've got Guard and Spring working together on the Rails project I currently have open. Looking at an equivalent model (100 LOC, 200 test LOC), it runs without database isolation in less than 400ms, which is less time than it takes me to switch to my terminal. It's really great, and I don't have to worry about isolation. TLDR; there are compromise solutions possible. Maybe not suited for everyone, but in some cases better than going too far with "design for testability."

deleted 12y ago

[deleted]

sandal 12y ago

The critical flaw in this post is that Gary is not refuting what DHH said. Gary makes this claim: > You finally get to see what's really going on. David's tests run in a few minutes, and he's fine with that. > I'm not fine with that. A lot of other people are not fine with that. But what DHH actually said is this: > You might think, well, that's pretty fast for a whole suite, but I still wouldn't want to wait 80 seconds every time I make a single change to my model, and want to test that. Of course not! Why on earth would you run your entire test harness for every single line change in a particular model? If you have so little confidence in the locality of your changes, the tests are indeed telling you that the system has overly high coupling. and this: > These days I can run the entire test suite for our Person model — 52 cases, 111 assertions — in just under 4 seconds from start to finish. Plenty fast enough for a great feedback cycle! Using a workflow like Gary's, there's an argument to be made that 4 seconds is not acceptable, and this is why we want single files that can run in a few milliseconds. However, that's not the only possible way of running tests, and the difference between 4 seconds and 300ms for the feedback you're actually interested is massively different than 300ms vs "a few minutes". For a post that calls DHH out on a strawman, this is in itself a great example of one.

gary_bernhardt 12y ago
Yes, I focus on my per-file runtime in the post, and I mention David's suite runtime in one sentence at the beginning. They are not meant to be compared. David's file runtime is four seconds. This is unacceptable to me. This is unacceptable to other people who replied to your tweets. This would double the length of my high-speed TDD loop, which would make those portions of my TDD process take twice as long. Yes, it would've been clearer for me to specifically address both suite runtimes and both unit runtimes. You know what else would've been clearer? All of the 2,000 words or so that I deleted from that post while I was editing it down into its final form. This is just how writing works. I don't think that it's misleading as written. Of course, I've already told you, on Twitter, exactly my reasons for rejecting both four-minute suites and four-second test files. They're not in the post, but you know the reasons. You know that I wasn't selectively attacking a subset of his argument, because you know that I do have an answer for test file runtime. And yet, for some reason, here we are! (For anyone reading this later, the tweets in question are gone. Lately I've been deleting all replies, as well as trivial non-replies, for Reasons.)

nshepperd 12y ago

> That file is going to be a plain old Ruby class: not a Rails model, controller, etc. Isolating those would lead to pain, as anyone who's tried to do it knows. I keep models very simple and allow them to integrate with the database; I keep controllers very simple and generally don't unit test them at all, although I do integration test them. This is what I was thinking of when I read DHH's post. Mock objects and indirection and all that certainly add complexity, but if your "business logic" is all in pure functions, or at least self-contained source files that don't do IO (and don't use "frameworks"), you don't need mocks to test it. Better to keep your glue code simple and do integration tests, while unit testing your actual logic, than to shoehorn extra indirection for mocking into complicated glue code. I gather that DHH complained because he had been doing the latter, and found that it sucked.

jongraehl 12y ago

I'd believe the same endorsements from someone who makes their living primarily off working code, rather than selling edu. material promoting test-heavy coding. That said, rapid "all's still well" feedback is awesome when you can get it.

GFK_of_xmaspast 12y ago

The best testing is the kind you use and monitor.