9 ms·

Sadly, in this day and age of development and the need to constantly ship things, TDD has been dead for a long time. In my 12 year career, I have heard lots of

Sadly, in this day and age of development and the need to constantly ship things, TDD has been dead for a long time. In my 12 year career, I have heard lots of people talk about test-driven development, but I've never seen it in a workplace (at least none I've worked at). On my own personal projects I have dabbled with TDD and I've seen the benefits it can provide, but it does make even simple programming tasks take a lot longer. Sadly, companies these days (especially during the pandemic) can no longer afford the luxury of development taking longer, even if it does mean the end result will most likely be cleaner and have less bugs. The company I work for see shipping potentially buggy code and fixing it as bugs are reported as an acceptable development practice. With the advent of automated builds and deployment processes, it is way too easy to quickly ship code and roll back bad releases or push out emergency patches. Things don't have to be perfect the first or second time around. The optics to non-technical executives seeing code go out and features released are a lot better than seeing things take longer to develop.

bigmanwalter 6y ago
The key to TDD is to write tests that evaluate on the order of 10ms. A whole test suite should take less than 1s. Only this way does TDD not interfere with development velocity.
- rco8786 6y ago
  What a dream that would be. But unfortunately not super realistic on a project of any real size.
  - bigmanwalter 6y ago
    There are ways to structure your code so that this can be done. Of course, it would require a complete refactor to achieve full test coverage on an existing project, but if you build like this from the beginning, it's feasible.
    - rco8786 6y ago
      Is there any literature on this? A test suite that runs in 1s is...kinda crazy sounding on any non-trivial codebase.
      - bigmanwalter 6y ago
        Gary Bernhardt has a pretty good talk: https://www.youtube.com/watch?v=RAxiiRPHS9k https://www.youtube.com/watch?v=RAxiiRPHS9k
  - ratww 6y ago
    It's pretty easy to get those numbers if your business code isn't intertwined with your UI/database/MVC code, or if your core code can be tested in a pure manner (like the "core" of an image editing software, or parsers, or serializers/protocols, etc). Of course, if all you have is a basic CRUD app, then there's not much testable "core" code that you can separate from your framework, so TDD and unit tests are probably not the best idea. IMO end-to-end and integration tests are the way to go anyway.
- bdcravens 6y ago
  Pretty sure the parent comment was talking about the time to write the tests, not the time to execute it.
  - bigmanwalter 6y ago
    The issue with slow execution is that you don't run the test suite often enough. I like running my tests almost as often as I compile my code. If they run fast enough, it creates the most addictive feedback loop for TDD. Besides, we should be measuring the time to write tests against the time it takes to manually test. If you don't execute tests every time you make a change, you're not really sure if your code works. And manual testing just means we have no idea whether or not our code works.
rzwitserloot 6y ago
That should itself be rather telling. TDD is extremely well known, in my experience. I bet if I stuck a microphone under the nose of random passersby at any development convention [1], 90%+ will be able to tell me what 'TDD' is short for, and even if they don't, they'll have heard of the concept at least. Hell, I bet most would tell me they 'aspire to do it'. And yet, more or less nobody does. So either it is next to impossible to begin doing it (seems like a bizarre conclusion), or, perhaps more likely, nobody wants to do it, and the few dev teams that do manage to do this have not managed to turn that into a competitive advantage. Which makes the value of TDD rather questionable based on simple evidence. To explain this observed behaviour that TDD is clearly not a competitive advantage[2], I can name a million pet theories. But without going into any of those, the sheer fact that it's __this__ rare in practice says a lot, no? [1] I mostly go to java related ones, maybe it's less well known amongst other communities. [2] What other explanation is there? Clearly not 'ah, but, TDD is brand new and you have to give it some time for teams to get familiar with it, and for the concept to percolate through, maybe wait for tooling support to catch up' - TDD's quite an old concept!
- 3pt14159 6y ago
  Well I do TDD around 85% of the time and I'm doing pretty well financially as a dev. Sometimes it isn't possible to do TDD, like when I'm iterating on a data science algorithm or exploring data and relationships before settling on a final approach. But you're right, most devs I know don't do it, though I've noticed the ones that do generally make more.
  - guscost 6y ago
    > I’ve noticed the ones that do generally make more. The market may be rewarding the TDD skill, it makes sense. But consider that causality makes sense the other way around too. Anyone with the patience, eye for detail, and organizational skills demanded by TDD could be just an above-average programmer. Personally I like a “don’t forget the tests” approach for larger units of business logic, but I’m against exhaustive micro-test suites that just turn into bespoke, buggy, and useless-at-compile-time type systems.
ceedan 6y ago
I treat it like a tool. I use TDD when I encounter a problem that is complex enough that I cannot consider or remember all of the possible inputs/outputs during development. Getting all of those dumped into a test is the best way to reduce cognitive load.
rco8786 6y ago
I don’t think it’s “this day and age”. I think that the reality is that TDD has never been the norm (even the article were lamenting over is 6 years old), for reasons you gave but also because it’s just not that good of a pattern. Is it useful? Probably. Is it useful relative to the trade off in shipping time? Less of an obvious answer.
- passthefist 6y ago
  To me, the value in TDD is the same as testing in general. It's not so much for initial feature development but rather later in a project or application maintenance you see the value. I view it as amortizing the complexity over the life of the project. Sure, it might take a bit longer to ship a simple feature, but bugs also take away from feature development, and in a complex enough application small changes might result large bugs. I guess that's true about software testing in general, but I think most developers would say there's value in having tests. Often, the tradeoff in shipping quickly is technical debt that impacts your ability to ship quickly in the long run. I think TDD can help manage that, especially if you're planning to write tests for a feature anyway. It does depend on what you're working on, of course, but I think most companies over-estimate their need to rapidly ship features. In the grand scheme of things adding a week to a 6 week project isn't that big a deal and might even save time in the long run.
  - rco8786 6y ago
    > In the grand scheme of things adding a week to a 6 week project isn't that big a deal mmm maybe once. But that's a compounding delay. If every project is delayed by ~15% for TDD then over the course of years you can fall well behind your competitors. > complex enough application small changes might result large bugs Totally agree - but also on really tightly TDD-tests code bases small changes can result in huge test refactorings...many times to the point of just not doing something because the time to update the tests is prohibitively expensive. There's a balance to all of this, and (empirically) TDD seems to be on the extreme end of the balance.
- sidlls 6y ago
  TDD is one of the many things this industry does to have the facade of rigor without actually developing the methodology sufficiently to move beyond a facade. It's driven largely by inertia and arguments from popular figures in the industry.
justaguyhere 6y ago
I recently tried TDD. It was fun for a while, but it took too much time writing tests for mundane things. I am not convinced about 100% coverage etc. For me the biggest takeaway was the way I was organizing my code. I work on a 15 year old web application with lots of legacy stuff, so it isn't easy to make changes fast. New code that I write depends on old code, so I am constrained a bit. Even still, I found marked improvement in my code structure after trying TDD. I do not know whether I'll stick with TDD, but I definitely will remember the code organizing lessons it taught me
- StevePerkins 6y ago
  It's usually counterproductive to strive for 100% test coverage. You wind up with nonsense tests for default getters/setters, and occasionally you face things that simply CAN'T be 100% tested due to how a 3rd-party library works. The most aggressive shop I've ever seen set their bar at 95% coverage.
- circlefavshape 6y ago
  In my last job we had 100% coverage, but if something was trivial we'd annotate so the code coverage tool would ignore it. It meant every decision not to cover something was documented, and visible to the code reviewer
ratww 6y ago
IMO, TDD really depends on which kind of code you're writing, which language, which tooling... I just finished a quite large collection of parsers and did it entirely using TDD, got 100% coverage and it was much easier than when I did it without it in the past. I kinda HAD to get 100% coverage because how else would I test my code? I only wrote the part that read from files at the end of the project. Also, when I was writing code that communicated with banks or telcos in arcane COBOL-era protocols, I didn't really have a way to "test" my code other than in production, so I relied on TDD for my day-to-day coding. It worked fine. For GUI stuff, or web development? I used TDD in the past and didn't gain anything for it. This should be obvious for everyone, but TDD only works when it works... it's not a silver bullet.
- UglyToad 6y ago
  I think your post gets to the core point. My view is that 90% of the code is fine without tests (assuming a crud app and some way of catching type errors). It's just plumbing to get data from a to b. There's around 10% that it's important to test, the core logic. I wrote a pdf parsing library and it's a joy to test and the tests give you a lot of certainty the code works because the unit is the library itself, the input is the file and the output is well defined. But I really am beginning to dislike testing most code in a web app. If you have a pure calculation or some business logic, test away. For everything else some higher level tests give far more assurance, when you're moving fast those might even be primarily manual. A good rule of thumb I think is that a test using mocks is a bit of an anti pattern, they make you feel like you're testing code when generally you're testing your test. The typical test pyramid is upside-down.
  - fendy3002 6y ago
    What people usually don't realize is automatic test is a "contract", and it works best for something that have specification. Libraries usually have specification. Web gui? Not so much.
  - ratww 6y ago
    > But I really am beginning to dislike testing most code in a web app. If you have a pure calculation or some business logic, test away. For everything else some higher level tests give far more assurance, when you're moving fast those might even be primarily manual. Yes, I completely agree! It took me a while to get used to write code like this, but having the business logic completely unbraided from the interface/database code is probably the biggest productivity boon I ever had, because my tests run blazingly fast. People don't think it matters, but instant feedback makes a lot of difference. This is also how things like DDD, Hexagonal Architecture, Functional-Core-Imperative-Shell are structured, so we're not alone. It doesn't have to be as complex as some of those: it just has to be easily testable... - > A good rule of thumb I think is that a test using mocks is a bit of an anti pattern, they make you feel like you're testing code when generally you're testing your test. Oh, I couldn't agree more. My personal pet peeve is having to write unit tests for thin-controllers or Spring-style services. I take an hour to carefully mock 10 deep dependencies, and in the end all I'm testing is if the language is able to call methods in other classes. Instead, if I just write the silliest integration test, I'll probably get better coverage and it will be able to uncover more bugs.
AgloeDreams 6y ago
All of this is spot on, especially the second point. As someone who lives with ADHD, TDD is hell incarnate for my productivity. 'Lets write a test for what will be an Observable and account for for failing, passing, bad data, empty data sets...oh wait, what was the original task?' It's so unbelievably monotonous that it's REALLY easy to lose track of the original task and worse, sometimes I'll be doing something and think up an additional or better way to accomplish something, it creates real-time tech debt faster. My SO has OCD and to see the ways we think in action being so different makes it super clear to me that things like TDD probably are heaven for some people.
- jacques_chester 6y ago
  I have ADHD also. I find TDD to be very helpful, as it encourages me to break the problem into the smallest possible next step. That said, I've found the most boost from the full XP menu -- pair programming, TDD etc. I can get a lot done with halved daily ritalin intake vs working alone.
loa_in_ 6y ago
Test driven development is most useful when there are multiple contributors. The added work time doesn't calculate when I am sole architect/implementor. If I was to be an architect only for a system though, I'd likely appreciate TDD to ensure implementors do what I think I spec'd
yebyen 6y ago
> The company I work for see shipping potentially buggy code and fixing it as bugs are reported as an acceptable development practice. That's what I've heard called "trading external quality" for time to market. That is exactly how you're supposed to do it, at least according to the dev coach that I've been listening to (@GeePawHill) Why you write tests and refactor is to keep your code's internal quality up. Those are the things that customers can't see, the things that make it harder for you to change your code when you come back to do more work later. The Correlation Principal states that ISQ and developer productivity are directly correlated and that you cannot trade internal quality for development time to market without this trade-off quickly coming back to bite you, in the form of lost productivity. A feature that is only partially implemented, or an edge case that isn't tested properly and causes something to fail at runtime, are both examples of external quality. You can indeed come back and fix that later, and you can make this trade cautiously to get your work to production faster. But if you write a long method for example that "just works" and ship it without further introspection, refactoring, or complete test coverage, you may never recover from that. It might require devs to spend an extra hour or more each time that method should need to change in the future. In some future state it has become so long and complicated that nobody can fully understand it, the time to stop digging is whenever you see yourself inside of this hole. And according to Sandi Metz, those big long methods are almost always guaranteed to be the ones that will need to change again, they are long and complicated because those are the business objects that you care about. They should be well factored as early as possible to facilitate future changes, (unless you have a crystal ball and can say for sure that won't ever need to happen!)
- meheleventyone 6y ago
  Factoring a long and complex method doesn't magically make the process it's doing less lengthy (unless there is a lot of repetition) or less complex. In fact it's likely to make it longer and more spread out so harder to understand from scratch. When I encounter such code it's far easier to change if you keep it together and document it well so future people can find out what it's doing and see the whole in context. For future changes, factoring also requires a crystal ball since you have no idea how requirements will change or if your factoring will actually be useful for them. Better to keep things simple and contained until you actually need to make them more complex.

bigmanwalter 6y ago

The key to TDD is to write tests that evaluate on the order of 10ms. A whole test suite should take less than 1s. Only this way does TDD not interfere with development velocity.

rco8786 6y ago
What a dream that would be. But unfortunately not super realistic on a project of any real size.
- bigmanwalter 6y ago
  There are ways to structure your code so that this can be done. Of course, it would require a complete refactor to achieve full test coverage on an existing project, but if you build like this from the beginning, it's feasible.
  - rco8786 6y ago
    Is there any literature on this? A test suite that runs in 1s is...kinda crazy sounding on any non-trivial codebase.
    - bigmanwalter 6y ago
      Gary Bernhardt has a pretty good talk: https://www.youtube.com/watch?v=RAxiiRPHS9k https://www.youtube.com/watch?v=RAxiiRPHS9k
- ratww 6y ago
  It's pretty easy to get those numbers if your business code isn't intertwined with your UI/database/MVC code, or if your core code can be tested in a pure manner (like the "core" of an image editing software, or parsers, or serializers/protocols, etc). Of course, if all you have is a basic CRUD app, then there's not much testable "core" code that you can separate from your framework, so TDD and unit tests are probably not the best idea. IMO end-to-end and integration tests are the way to go anyway.
bdcravens 6y ago
Pretty sure the parent comment was talking about the time to write the tests, not the time to execute it.
- bigmanwalter 6y ago
  The issue with slow execution is that you don't run the test suite often enough. I like running my tests almost as often as I compile my code. If they run fast enough, it creates the most addictive feedback loop for TDD. Besides, we should be measuring the time to write tests against the time it takes to manually test. If you don't execute tests every time you make a change, you're not really sure if your code works. And manual testing just means we have no idea whether or not our code works.

rzwitserloot 6y ago

That should itself be rather telling. TDD is extremely well known, in my experience. I bet if I stuck a microphone under the nose of random passersby at any development convention [1], 90%+ will be able to tell me what 'TDD' is short for, and even if they don't, they'll have heard of the concept at least. Hell, I bet most would tell me they 'aspire to do it'. And yet, more or less nobody does. So either it is next to impossible to begin doing it (seems like a bizarre conclusion), or, perhaps more likely, nobody wants to do it, and the few dev teams that do manage to do this have not managed to turn that into a competitive advantage. Which makes the value of TDD rather questionable based on simple evidence. To explain this observed behaviour that TDD is clearly not a competitive advantage[2], I can name a million pet theories. But without going into any of those, the sheer fact that it's __this__ rare in practice says a lot, no? [1] I mostly go to java related ones, maybe it's less well known amongst other communities. [2] What other explanation is there? Clearly not 'ah, but, TDD is brand new and you have to give it some time for teams to get familiar with it, and for the concept to percolate through, maybe wait for tooling support to catch up' - TDD's quite an old concept!

3pt14159 6y ago
Well I do TDD around 85% of the time and I'm doing pretty well financially as a dev. Sometimes it isn't possible to do TDD, like when I'm iterating on a data science algorithm or exploring data and relationships before settling on a final approach. But you're right, most devs I know don't do it, though I've noticed the ones that do generally make more.
- guscost 6y ago
  > I’ve noticed the ones that do generally make more. The market may be rewarding the TDD skill, it makes sense. But consider that causality makes sense the other way around too. Anyone with the patience, eye for detail, and organizational skills demanded by TDD could be just an above-average programmer. Personally I like a “don’t forget the tests” approach for larger units of business logic, but I’m against exhaustive micro-test suites that just turn into bespoke, buggy, and useless-at-compile-time type systems.

ceedan 6y ago

I treat it like a tool. I use TDD when I encounter a problem that is complex enough that I cannot consider or remember all of the possible inputs/outputs during development. Getting all of those dumped into a test is the best way to reduce cognitive load.

rco8786 6y ago

I don’t think it’s “this day and age”. I think that the reality is that TDD has never been the norm (even the article were lamenting over is 6 years old), for reasons you gave but also because it’s just not that good of a pattern. Is it useful? Probably. Is it useful relative to the trade off in shipping time? Less of an obvious answer.

passthefist 6y ago
To me, the value in TDD is the same as testing in general. It's not so much for initial feature development but rather later in a project or application maintenance you see the value. I view it as amortizing the complexity over the life of the project. Sure, it might take a bit longer to ship a simple feature, but bugs also take away from feature development, and in a complex enough application small changes might result large bugs. I guess that's true about software testing in general, but I think most developers would say there's value in having tests. Often, the tradeoff in shipping quickly is technical debt that impacts your ability to ship quickly in the long run. I think TDD can help manage that, especially if you're planning to write tests for a feature anyway. It does depend on what you're working on, of course, but I think most companies over-estimate their need to rapidly ship features. In the grand scheme of things adding a week to a 6 week project isn't that big a deal and might even save time in the long run.
- rco8786 6y ago
  > In the grand scheme of things adding a week to a 6 week project isn't that big a deal mmm maybe once. But that's a compounding delay. If every project is delayed by ~15% for TDD then over the course of years you can fall well behind your competitors. > complex enough application small changes might result large bugs Totally agree - but also on really tightly TDD-tests code bases small changes can result in huge test refactorings...many times to the point of just not doing something because the time to update the tests is prohibitively expensive. There's a balance to all of this, and (empirically) TDD seems to be on the extreme end of the balance.
sidlls 6y ago
TDD is one of the many things this industry does to have the facade of rigor without actually developing the methodology sufficiently to move beyond a facade. It's driven largely by inertia and arguments from popular figures in the industry.

justaguyhere 6y ago

I recently tried TDD. It was fun for a while, but it took too much time writing tests for mundane things. I am not convinced about 100% coverage etc. For me the biggest takeaway was the way I was organizing my code. I work on a 15 year old web application with lots of legacy stuff, so it isn't easy to make changes fast. New code that I write depends on old code, so I am constrained a bit. Even still, I found marked improvement in my code structure after trying TDD. I do not know whether I'll stick with TDD, but I definitely will remember the code organizing lessons it taught me

StevePerkins 6y ago
It's usually counterproductive to strive for 100% test coverage. You wind up with nonsense tests for default getters/setters, and occasionally you face things that simply CAN'T be 100% tested due to how a 3rd-party library works. The most aggressive shop I've ever seen set their bar at 95% coverage.
circlefavshape 6y ago
In my last job we had 100% coverage, but if something was trivial we'd annotate so the code coverage tool would ignore it. It meant every decision not to cover something was documented, and visible to the code reviewer

ratww 6y ago

IMO, TDD really depends on which kind of code you're writing, which language, which tooling... I just finished a quite large collection of parsers and did it entirely using TDD, got 100% coverage and it was much easier than when I did it without it in the past. I kinda HAD to get 100% coverage because how else would I test my code? I only wrote the part that read from files at the end of the project. Also, when I was writing code that communicated with banks or telcos in arcane COBOL-era protocols, I didn't really have a way to "test" my code other than in production, so I relied on TDD for my day-to-day coding. It worked fine. For GUI stuff, or web development? I used TDD in the past and didn't gain anything for it. This should be obvious for everyone, but TDD only works when it works... it's not a silver bullet.

UglyToad 6y ago
I think your post gets to the core point. My view is that 90% of the code is fine without tests (assuming a crud app and some way of catching type errors). It's just plumbing to get data from a to b. There's around 10% that it's important to test, the core logic. I wrote a pdf parsing library and it's a joy to test and the tests give you a lot of certainty the code works because the unit is the library itself, the input is the file and the output is well defined. But I really am beginning to dislike testing most code in a web app. If you have a pure calculation or some business logic, test away. For everything else some higher level tests give far more assurance, when you're moving fast those might even be primarily manual. A good rule of thumb I think is that a test using mocks is a bit of an anti pattern, they make you feel like you're testing code when generally you're testing your test. The typical test pyramid is upside-down.
- fendy3002 6y ago
  What people usually don't realize is automatic test is a "contract", and it works best for something that have specification. Libraries usually have specification. Web gui? Not so much.
- ratww 6y ago
  > But I really am beginning to dislike testing most code in a web app. If you have a pure calculation or some business logic, test away. For everything else some higher level tests give far more assurance, when you're moving fast those might even be primarily manual. Yes, I completely agree! It took me a while to get used to write code like this, but having the business logic completely unbraided from the interface/database code is probably the biggest productivity boon I ever had, because my tests run blazingly fast. People don't think it matters, but instant feedback makes a lot of difference. This is also how things like DDD, Hexagonal Architecture, Functional-Core-Imperative-Shell are structured, so we're not alone. It doesn't have to be as complex as some of those: it just has to be easily testable... - > A good rule of thumb I think is that a test using mocks is a bit of an anti pattern, they make you feel like you're testing code when generally you're testing your test. Oh, I couldn't agree more. My personal pet peeve is having to write unit tests for thin-controllers or Spring-style services. I take an hour to carefully mock 10 deep dependencies, and in the end all I'm testing is if the language is able to call methods in other classes. Instead, if I just write the silliest integration test, I'll probably get better coverage and it will be able to uncover more bugs.

AgloeDreams 6y ago

All of this is spot on, especially the second point. As someone who lives with ADHD, TDD is hell incarnate for my productivity. 'Lets write a test for what will be an Observable and account for for failing, passing, bad data, empty data sets...oh wait, what was the original task?' It's so unbelievably monotonous that it's REALLY easy to lose track of the original task and worse, sometimes I'll be doing something and think up an additional or better way to accomplish something, it creates real-time tech debt faster. My SO has OCD and to see the ways we think in action being so different makes it super clear to me that things like TDD probably are heaven for some people.

jacques_chester 6y ago
I have ADHD also. I find TDD to be very helpful, as it encourages me to break the problem into the smallest possible next step. That said, I've found the most boost from the full XP menu -- pair programming, TDD etc. I can get a lot done with halved daily ritalin intake vs working alone.

loa_in_ 6y ago

Test driven development is most useful when there are multiple contributors. The added work time doesn't calculate when I am sole architect/implementor. If I was to be an architect only for a system though, I'd likely appreciate TDD to ensure implementors do what I think I spec'd

yebyen 6y ago

> The company I work for see shipping potentially buggy code and fixing it as bugs are reported as an acceptable development practice. That's what I've heard called "trading external quality" for time to market. That is exactly how you're supposed to do it, at least according to the dev coach that I've been listening to (@GeePawHill) Why you write tests and refactor is to keep your code's internal quality up. Those are the things that customers can't see, the things that make it harder for you to change your code when you come back to do more work later. The Correlation Principal states that ISQ and developer productivity are directly correlated and that you cannot trade internal quality for development time to market without this trade-off quickly coming back to bite you, in the form of lost productivity. A feature that is only partially implemented, or an edge case that isn't tested properly and causes something to fail at runtime, are both examples of external quality. You can indeed come back and fix that later, and you can make this trade cautiously to get your work to production faster. But if you write a long method for example that "just works" and ship it without further introspection, refactoring, or complete test coverage, you may never recover from that. It might require devs to spend an extra hour or more each time that method should need to change in the future. In some future state it has become so long and complicated that nobody can fully understand it, the time to stop digging is whenever you see yourself inside of this hole. And according to Sandi Metz, those big long methods are almost always guaranteed to be the ones that will need to change again, they are long and complicated because those are the business objects that you care about. They should be well factored as early as possible to facilitate future changes, (unless you have a crystal ball and can say for sure that won't ever need to happen!)

meheleventyone 6y ago
Factoring a long and complex method doesn't magically make the process it's doing less lengthy (unless there is a lot of repetition) or less complex. In fact it's likely to make it longer and more spread out so harder to understand from scratch. When I encounter such code it's far easier to change if you keep it together and document it well so future people can find out what it's doing and see the whole in context. For future changes, factoring also requires a crystal ball since you have no idea how requirements will change or if your factoring will actually be useful for them. Better to keep things simple and contained until you actually need to make them more complex.