10 ms·
When I do TDD and when I don’t
- jbverschoor 5y agoI think people confuse TDD with just writing tests
- orobinson 5y agoI find TDD is great when you have an interface to some code defined and are changing the behaviour of what happens when interacting with that interface. You can then add some tests to the existing test suite for the interface, watch them fail, then get to work implementing and be content that you’re done when all the tests are passing. However, if you’re still developing the interface definition itself, TDD just wastes time when it transpires the interface you envisaged and wrote tests for doesn’t quite match up with the implementation as you write it.
- Jenk 5y agoThat's missed the beat a bit.. you are supposed to let the tests drive that design out, not big-design upfront and then fill out tests.
- tarkin2 5y agoChanging and newly-discovered requirements, rather than tests, drive your code.
- Jenk 5y agoDisagree. You are developing to a set of requirements at anyone time. What emerges from (A)TDD is the implementation of those requirements and the design therein.
- tarkin2 5y agoWhen those requirements change, and they can very, very quickly, you need to rewrite your tests--fine if you don't mind spending that time.
- AnimalMuppet 5y agoHopefully you only need to change some of your tests. When your requirements change, and you change your code, you have two questions. First, did my code change do what is needed for the requirements change? You modify tests (or write new ones) to answer that. Second, did I break anything in the process? The remaining tests answer that.
- monocasa 5y agoI've found the flow true for some environments with extremely well defined structures for doing any work (classic backend API servers being one example), but it's definitely not true for all envs. A lot of envs have you spiking so much on just how to even approach the feature that writing tests up front is wasteful overhead.
- cgrealy 5y agoIf you are spiking, then you should be throwing away that code. Don't productionise prototypes.
- monocasa 5y agoDon't just throw untested prototypes in production, but you don't gain anything in a lot of envs by throwing away a prototype completely. You're likely to introduce additional issues by rewriting it for no reason.
- AnimalMuppet 5y agoWell, if a spike is going to become production, when it does, then write the tests (along with all the other cleanup that needs to happen if your going to turn prototype code into production).
- AnimalMuppet 5y agoBut, given the changing and newly-discovered requirements, letting tests drive the design of the implementing code can still be surprisingly effective. Why? Because if I find it hard to write the test, it's telling me that I'm likely to find it hard to write non-test code that uses the class/module/subsystem/whatever. It forces you to use the interface to your code. If it's hard to use, that's telling you to consider changing the design.
- sidlls 5y agoThis assumes the test isn’t making it hard to use unnecessarily.
- squeaky-clean 5y agoRequirements drive your tests, which drive your code.
- sidlls 5y agoOne doesn’t design around tests, unless the test code is the design spec. In which case something has gone horribly wrong in the requirements and analysis for the project.
- cgrealy 5y ago> One doesn’t design around tests No design gets to the coding stage until someone has answered "how will this be tested?" So, yeah, you should absolutely be designing around tests.
- sidlls 5y ago"how will this be tested?" can be answered without writing a single line of test code or defining what the test suite will look like. The design spec documents determine what the implementation should look like, and the tests should verify the implementation works as desired, not drive its structure. I don't want the implementation to be chopped up and scattered everywhere in the name of "testability" because someone decided that a tool should dictate the architecture of the code. That leads to unreadable, hard-to-maintain, inefficient code that, with a tight coupling to the test suite that makes both them of fragile in the presence of changes in the other. For very little gain.
- cgrealy 5y ago> That leads to unreadable, hard-to-maintain, inefficient code that, with a tight coupling to the test suite that makes both them of fragile in the presence of changes in the other. For very little gain. I completely disagree with everything above. Literally, take the sentence and make almost every word the opposite.
- drran 5y ago
- sockpuppet69 5y ago
- eric-hu 5y agoMy experience agrees with the GP, orobinson. I used to work at a company that wanted us to TDD everything. Everyone working there knew this from the time of interview, so it’s not like any of us were unwilling to give it a shot. When it came to adding a feature to a Rails app, the interface is pretty well defined, it’s going to serve up HTML and js, it’s got to fit on the existing structure of the page or conform to a design. Adjacent parts of our company were working on a Heroku clone PaaS, and I got some time on a team building an auto scaler service to spin up and down other services. We didn’t know how our service would kill and start other services. Tests were changing as much as production code while we worked on that. A lot of our work was spiking to see how our service fit into the ecosystem of other services.
- Jenk 5y ago> I used to work at a company that wanted us to TDD everything. Dogma is the bain of everyone :) I will never advocate "100% TDD." It is a tool like any other that has its place and, more importantly, doesn't fit everywhere. Though I will freely admit I do encourage people to use it more than what I have seen to be the typical amount. > When it came to adding a feature to a Rails app, the interface is pretty well defined, it’s going to serve up HTML and js, it’s got to fit on the existing structure of the page or conform to a design. It's very unlikely I'd use TDD on that. Boilerplate/plumbing is something I rarely use/advocate TDD for - it's the "clever" stuff you want to cover. The stuff that is following rules outside of the code itself. > Tests were changing as much as production code while we worked on that. Absolutely nothing wrong with that, imo.
- thibauts 5y agoExactly this. Encoding business requirements or expected output of a complex algorithm by writing tests before code can make some sense. Writing tests for something that has no defined shape yet doesn’t.
- agonz253 5y agoJohn Ousterhout has a different take on this in "A Philosophy of Software Design": The problem with test-driven development is that it focuses attention on getting specific features working, rather than finding the best design. This is tactical [as opposed to strategic] programming pure and simple, with all of its disadvantages. Test-driven development is too incremental: at any point in time, it’s tempting to just hack in the next feature to make the next test pass. There’s no obvious time to design, so it’s easy to end up with a mess. One place where it makes sense to write the tests first is when fixing bugs. Before fixing a bug, write a unit test that fails because of the bug. Then fix the bug and make sure that the unit test now passes. This is the best way to make sure you really have fixed the bug. If you fix the bug before writing the test, it’s possible that the new unit test doesn’t actually trigger the bug, in which case it won’t tell you whether you really fixed the problem.
- BurningFrog 5y ago> * There’s no obvious time to design* Design time is the refactoring after your tests pass. That's when you can move your code around with confidence, since you can lean on your tests.
- pydry 5y agoThis works provided you didnt encode your design mistakes in the tests themselves. Then youve just doubled the cost of fixing that mistake. It's hard to overstate how common this is.
- Jtsummers 5y ago> This works provided you didnt encode your design mistakes in the tests themselves. Then youve just doubled the cost of fixing that mistake. If your design mistakes are in your code and your tests, they'll also likely be in your mental model and your documentation (if it exists). At which point you what? Throw it all away and start over? In extreme cases, but in others you do partial rewrites and refactors. Address the issues. But not having tests because they may encode your design mistakes is as foolish as not having documentation or comments because they may also encode your design mistakes. In a few years, you'll have a blob of code that, in the best case, is perfectly readable and comprehensible. But, in reality, is likely to fail at communicating the overall design intent and requirements. And so what if it doubles the cost of fixing the mistake? You made a mistake and it had to be fixed anyways. In the end, I've never found tests to cost more than they saved. A lack of tests has always led to higher development costs (primarily as measured in time to delivery). Regressions are ludicrously common without tests, which eats away at your time (and therefore adds to your costs) very quickly.
- efsavage 5y ago> However, if you’re still developing the interface definition itself, TDD just wastes time when it transpires the interface you envisaged and wrote tests for doesn’t quite match up with the implementation as you write it. This sounds like TDD is WAI in this case. If the tests break because of the implementation of the interface, either the tests were wrong (likely because the interface was poorly or incompletely described) or the requirements changed (and broken tests means you had decent coverage). Catching this with a simple test run during initial development is far cheaper than debugging against someone else's expectations in the future.
- DerArzt 5y agoI will recommend folks take the time to go and read the O.G. book on TDD[1] to really get a feel for what it was intended to be originally and form their opinion from there. Most of the time when I see people talk about TDD they are either cargo cultists or uninformed people. [1] https://www.goodreads.com/book/show/387190.Test_Driven_Development https://www.goodreads.com/book/show/387190.Test_Driven_Devel...
- MonaroVXR 5y agoI will check the link, it's difficult to believe anyone.
- DerArzt 5y agoThe book is worth reading, even if you don't end up adding TDD to your normal workflow. One of the things that stood out to me about it is that Kent mentions that you should throw away your lower level tests that you used while you were developing and exploring the problem. I have yet to see anyone ever mention that little tidbit.
- cloogshicer 5y agoHey, thanks for your comment, just curious, do you think the author of the article is uninformed? If so, could you briefly explain how so? Genuinely curious, since I generally agree with what they said. However, I've been recommended that book before with the same sentiment and am quite curious about opposing opinions, just haven't found the time to read it yet.
- Jenk 5y agoI find TDD helps me mind-map out my implementations. It's a bit hand-wavy, but generally I'll use TDD when things are complex and I need to clear my head of the bigger picture in favour of focussing on the smaller picture (I.e., lean on the the "next interesting test" mantra) to let the tests lead me through the path. Will I use TDD for basic things? No. I'll very probably write tests for them though just to provide regression coverage.
- thatswrong0 5y agoAgree with the reasoning here, so as a result I almost never use TDD: I pretty much always have to spike + iterate in order to come to a reasonable and clean solution for whatever I'm working on.. no matter how much I've tried to spec things out exactly how they should work, it always ends up being different. I'm also just not great at abstract vs. hands on thinking. On top of this, I've also come to find unit tests to generally require more work and be less useful for identifying regressions than integration tests for the work I do. Maybe this is just a result of the kind of work I do though.
- bvirb 5y agoSame here. So much so we've basically just dropped "unit" testing in favor of integration testing. I guess we still call it TDD though. I was actually surprised what the author was describing was somehow _not_ TDD. I thought it was pretty normal to try some things out, make some decisions, and then return to codifying those decisions in tests. We do web app development, and 90% of our tests are browser-based integration tests that test whether the given inputs (usually forms) lead to the expected outputs (usually something printed on a web page). I wonder if this is more natural when you're mostly writing integration tests?
- deleted 5y ago[deleted]
- jasonswett 5y agoI personally don't see any reason why the type of test matters when doing TDD. As I see it you can do TDD with integration tests just as much as you can do TDD with unit tests. The only difference to me is that the work involved might take on a bit of a different character. But that doesn't make it not TDD to me.
- pydry 5y agoI've found TDD can work pretty well with integration tests. Often better. I dont really get why the practise is so intrinsically linked to the practise of writing unit tests. Increasingly I'm starting to believe unit tests are a scam, but TDD is going to stick around in one form of another forever.
- HNSucksAss 5y ago
- johnklos 5y agoYou know, not everyone thinks TDD has to do with development: https://en.wikipedia.org/wiki/Telecommunications_device_for_the_deaf https://en.wikipedia.org/wiki/Telecommunications_device_for_...
- lbhdc 5y agoI feel like the average HN reader would probably assume its about testing.
- exdsq 5y ago“When I do Telecommunications device for the deaf and when I don’t” doesn’t make sense though which helps
- johnklos 5y agoUmmm... There are times when using TDD makes sense over trying to talk with automated menus. TDD can save lots of time with certain menu systems. Use your imagination some time.
- pictur 5y agoTDD is something I have never tried before. but since I like to prepare a certain checklist first and develop on it, I guess I can.
- amriksohata 5y agoAny surveys done worldwide as to how many people do TDD, how many just wrap in tests afterwards, and how many don't unit test at all ? At different N tier levels?
- kevinskii 5y agoNo matter how carefully and thoroughly we design a system up front, we almost discover more elegant solutions once we start writing the actual code. I suspect this is true for nearly all non-trivial systems. If we were to fully implement every unit test up front we would usually end up wasting a lot of time.
- oweiler 5y agoWhat you describe is not TDD.
- convolvatron 5y agono, i took this as a good argument about why writing tests up front is really poor idea if you're exploring the space to find out what a good api might look like.
- webmobdev 5y agoEverything can be improved. At what point do you stop? Scope creep and gold-plating ...
- isbadawi 5y agoI think the person you're replying to is saying that "fully implement every unit test up front" does not describe TDD. With TDD you continuously alternate between writing tests and writing production code. It's not like you write a whole test suite up front.
- oweiler 5y agoYes this was my point.
- deleted 5y ago[deleted]
- rileymat2 5y agoOne thing I do not see mentioned is that tests let me try out the interfaces I am building as a consumer of them earlier than I would otherwise. So even in exploration it can be useful, sometimes.
- sdoering 5y agoI am not a programmer. And I am by far not good enough in any language to spontaneously think architectural enough to create unit tests and do TDD. I always feel like I should be doing more tests. That this is the correct way to do things. Being a data/web analyst I have to say that I once created extensive tests for a specific part of custom javascript logic setup by a former agency of my client in the tag management solution they used. I even had to mock basic functionality from the tag manager (Adobe Launch) to make it work locally. When I took over it was a mess of code that mapped url parameters to their specific marketing channel logic (you could have done this purely in Adobe Analytics, though - but I was never able to find a way to explain the unnecessary complexity of the implemented solution). In the end I had created around 60 test cases to ensure this fragile bit was in a way that enabled refactoring. It worked. Especially when around one year later, a colleague of mine who had taken over the client needed to change stuff in there, had it break and asked me. I thought his solution should work, but the tests told a different story. Within half an hour we had it nailed down, fixed and running flawlessly. If I ever have such a complex piece of logic anywhere I will surely learn how to write tests in the respective language of choice. Until then I will happily fatfingered stuff on my own amateur projects, though.
- Anon6747463 5y agoTDD is climbing with ropes; it's useful in situations where you don't know how to get all the way to the top but want to try stuff from where you've gotten to without falling all the way to the bottom.
- pc86 5y agoIsn't the argument TFA is making the opposite, though? That TDD only really makes sense when you're working a very "crisply defined" specification?
- eesmith 5y agoYes, that's my interpretation. This reminded me of when Ron Jeffries tried to use TDD to write a Sudoku solver, but didn't manage to do it. The take-home lesson was summarized at https://www.infoq.com/news/2007/05/tdd-sudoku/ https://www.infoq.com/news/2007/05/tdd-sudoku/ as "while TDD may not be the best tool for inventing new algorithms, it may very well be the best tool for applying those algorithms to the problem at hand." In this context, I think of "crisply defined" as being when know the algorithms, and input/output format, and want to get things working together. I still don't find TDD useful. Even in that case, I much more a spike-then-stabilize developer, with most of my tests added at the end, followed by coverage analysis to identify missing tests. (And I don't believe for a minute the claims of TDD supporters that TDD naturally leads to 100% coverage.)
- pc86 5y ago> And I don't believe for a minute the claims of TDD supporters that TDD naturally leads to 100% coverage. I agree with you, but at the risk of No True Scotsman'ing this, one of the TDD tenets I'm familiar with is that you write the least amount of code necessary to pass a test. So if you do it The Right Way, then you should be at 100% or something very very close to that.
- eesmith 5y agoYes, you might start that way. Then over time you lose it, unless you are actively doing coverage testing. Here's the clearest example. For a while I had a code base which supported both Python 2 and Python 3. I dropped Python 2 support and removed the compatibility layer. I'm still finding places where I have Python 2 code paths. (Clearly I'm not using enough coverage testing.) Here's another example. Suppose I have a single public entry point, which internally calls a number of private units. I've carefully tested only at that public entry point so my test coupling isn't an issue. I then realize that I can special case (say) n=0 early in the public entry point. Now, a number of the private units no longer need to handle the n=0 case. Manual inspection during refactoring might catch all of those code paths which are no longer used. I know I'm not diligent enough to find all those cases. I've even had times where I find an entire function is no longer being called at all, because a refactoring removed the need to have it. Once you've had a few refactorings of a non-trivial code base, even during its greenfield development phase, you're almost certainly going to have dead code. Even when using TDD. No, I don't know what the coverage rate would be on a TDD project which doesn't use code coverage. But my experience tells me not to believe Kent Beck's statement "TDD followed religiously should result in 100% statement coverage".
- cgrealy 5y agoThe main value in TDD is not when you write the initial implementation. Sure, it can help, but most devs are competent enough to implement a function/class/whatever and have it do what the programmer thinks it will do. The value from TDD is documenting those assumptions. "The db will always be connected before this class is used", "This function assumes the user is authenticated" and so on. Then in 6 months, a year or even 5, when you or someone else comes back to this code, you can be reasonably certain changes to it will break tests, because the tests tell you how the REST of the codebase is using this. Of course, that assumes a decent shelf life for the code. If you can be reasonably certain you're going to handover this and never touch it again, don't bother with the tests. And delete the repo while you're at it.... (not comfortable with that, are we? :D )
- bobobob420 5y ago
- bvirb 5y agoIs TDD really about never figuring out unknowns using untested code? What about using diagrams? Or white boarding? Test first design makes a lot of sense, but when you’re unsure about the design using code to figure it out seems just as reasonable as any other method. If not what’s left? Do everything waterfall?
- ipaddr 5y agoI would like to see waterfall make a comeback. No one agile's alone but many waterfall.
- Jtsummers 5y agoWaterfall does not need to make a comeback, it never died. It's still a gross and stupid process for anything but trivial projects or well-understood (by the developers) domains. If you're working on a large scale project, it's one of the worst ways to work. The next worst is to just type on a keyboard and hope you manage to write code and somehow trigger the build and deploy commands.
- duped 5y agoWaterfall is alive and well, most places just refer to it as something else like "agile" or "scrum."
- thibauts 5y agoWhen tests amount to specs waterfall tends be be referred to as TDD.
- Jtsummers 5y agoNo, because TDD (at least as defined) does not have you spend months to years up front just writing tests. You write them at the same time as (well, in strict TDD just prior to) writing the code and (possibly) other artifacts. Waterfall says, "Spend a stupidly long amount of time coming up with detailed requirements and specs and a hyper-detailed, and likely wrong, development and test plan. Then don't receive any feedback until you test, which is verification and conducted after the development is done, and deliver, which is validation." TDD at least has you do the verification part throughout the development process.
- bobobob420 5y agoJust write the code and then write the test. Or just write the test and then write the code. Or think about the test when writing the code. Or think about the code when writing the test. It doesn't really matter right. The thought process is going to occur anyways. Focusing on resuable patterns, consistency, and creating reliable and easy to read code is 10x more important than the TDD drivel that is out there. Honestly, honestly, The only reason we discuss this crap is because there are people who make off running around the country giving talks and writing books on all things developers (nothing wrong with that).
- deleted 5y ago[deleted]
- strife25 5y agoI've had the most luck w/ TDD for designing APIs. When I write the tests first, it gives me a feel for how consumers of the API will use it. It's a great method to understand the UX of the re-usable code.
- strife25 5y agoWhen it comes to how much testing should be done, I think it's contextual on the project. I expand on this specific point in a blog post: https://www.buildthestage.com/just-enough-automated-tests/ https://www.buildthestage.com/just-enough-automated-tests/
- DerArzt 5y agoI'm re-reading PP[1] right now and their Tip #67 feels relevant here: > A test is the first user of your code [1] The Pragmatic Programmer 20th Anniversary Edition (Thomas, Hunt)
- cjvirtucio 5y agoI find tests more helpful when the implementation is stable. It's a little tedious to re-write test cases as unknowns come up.
- LouisSayers 5y agoWorking on my own project as a solo dev, I hardly ever write tests. What I've learnt over the years is that tests lock your code in making it hard to adapt (you could say making you less agile). When I'm smashing out new code and I haven't established patterns then it doesn't make sense to lock that new code in. Once I do have a pattern going then I might add tests, but it's not for the sake of test coverage or to gloat that I have tests, it's purely because I actually care about making sure those bits of the code work as intended and that the edge cases are covered. For these tests I do indeed do TDD. Otherwise it's a trade-off - would I rather be closer to a feature people can use or would I rather be slogging away writing and maintaining tests. This being said, I will say that when I come to a point of adding more people to the team tests will become much more important. The way I see it though, not having tests and having people complaining about it means that I've done something right.
- somewhereoutth 5y agoEither testing finds the bugs, or the user finds the bugs. If the user is a machine, use a machine to test the code. If the user is a human, use a human. The user does not know or care about unit tests. The state space of any non-trival algorithm is far greater than can be sensibly enumerated. Software is design - designing something inside out makes no sense.
- erdo 5y agoI do TDD in one specific case, and it's always a unit test. Very occasionally I will need to write a function or a small class do some complicated logic that I'm too lazy or stupid to work out how to do. I do know exactly what the results should look like though, including all the possible edge cases. ( I think the last time I did this was for a point of sale terminal I was working on, which needed a solution similar to the change making problem https://en.m.wikipedia.org/wiki/Change-making_problem https://en.m.wikipedia.org/wiki/Change-making_problem ) Anyway, for those situations, I write a large number of tests cases, covering every reasonable scenario, plus a bunch of unreasonable scenarios. Then I write a half-assed implementation that fails on several tests, and I keep hacking about until more of the tests pass. Once they all pass, I stop. Even if at that point I have no idea why that particular version of the code completely works. It's nasty I know, but sometimes it's the quickest way to a robust implementation
- markmaglana 5y agoI tend to go by how much anxiety a piece of code might give me once it goes to production. If I expect the level of anxiety to breach a certain threshold (usually when there’s too many logic branching than I can keep in my head), then I write the tests first. The above assumes though that I’ve done enough previous discovery to have a relatively robust model of the problem domain. If not, I prototype first and during this stage I won’t be using TDD. Prototypes produced are throwaway, of course.
- GuB-42 5y agoTDD always felt wrong to me. And in fact, a lots of my worst code was written with a TDD-like approach. The basic idea is: read the specs, write the tests, then write the code that passes the tests. The big problem here is that you add a level of indirection, and something may be lost in translation. That is, if the tests are wrong, then your code will be wrong. Since you already have the tests when you start coding, there is a tendency to forget the specs and code the tests, since your goal now is to have "all green". And because if you did it well, it is very easy to test your code, it is tempting to just tweak it until it passes, without you understanding what you are doing. I know because that's the process that lead me to the first mentioned terrible code. For a simple analogy, TDD would be like giving students the test subject before they start studying. Sure, they will get good grades, but don't count on them studying what is not on the test, and some will simply memorize the answers. That's why at school, you almost never have the test in advance. Obviously, I am not against testing, especially automated unit testing, but the core idea of TDD, that is writing the test first, is, I think, a bad idea. The often cited argument is that because you start with a failing test (because you haven't implemented the feature yet), you are guaranteed to actually test something, but I find it to be of limited value, except maybe on very large projets where not implementing a feature at all is something that can happen accidentally. An approach that I think works better is to write the code, write the tests, pass the tests, intentionally break your code, check that the tests fail, revert your changes. Code and tests can be written at the same time, ideally by different people, but if you do thing in order, I would go code first. In fact, I think the only real "advantage" of TDD is that it simply forces you to write tests. It is an anti-corner-cutting technique.