6 ms·

My 2c: tdd is great in order to learn to create testable designs. You can’t tdd nontestable code. Once you understand how to design testable code, tdd offers m

My 2c: tdd is great in order to learn to create testable designs. You can’t tdd nontestable code. Once you understand how to design testable code, tdd offers minimal benefits. The real value comes from thorough, automated testing suites, whatever their origin is.

ashtonkem 5y ago
Good observability standards and fast releases catch more bugs than meticulously maintained test suites, imho.
- drenei 5y ago
  I actually think if you squint testing is part of observability, except it gives you signal during the development phase. And observability as a concept is fundamental to a reliable system.
  - ashtonkem 5y ago
    I see where you’re going, but I think if we start defining tests as part of observability then we’ll water that term down to meaninglessness.
    - snovv_crash 5y ago
      If there's a bug and no test suite to try to reproduce it, how do you know you actually fixed it?
      - ashtonkem 5y ago
        I define observability as the tools I use to check on the behavior of my system in ad-how ways in production. This is quite different from tests, which are designed to exercise specific behavior in a fixed environment. Both are used towards the goal of software quality, but they’re different.
        snovv_crash 5y ago
        I dunno. Working on projects with no tests for me always feels like a complete crapshoot. Any change could have broken some implicit assumption in some module you didn't know depended on it. Even worse is if it's a dynamic language and you don't even have a compiler to do the basic checks. For things like reproducing bugs, in particular, functional tests plus a debugger have been indispensable in seeing how the code actually behaves. Sure, production monitoring is nice. But it really only tells you about high level problems, and there are entire classes of bugs, eg. silently corrupted data, which you'll never see there.
        ashtonkem 5y ago
        You’re arguing against a more extreme version of the point I’m making. I’m not arguing for no tests, I’m arguing that uncle bob over states their capabilities by a lot.
pydry 5y ago
Sacrificing at the altar of unit testability doesnt necessarily make better code. Unit tests' inability to handle state is way too often viewed as a problem with the code it can't properly test than the general crappiness of this form of test.
- pc86 5y ago
  Doesn't the state get handled in integration tests? The best environment I've worked in (from a testing perspective) had thousands of super fast unit tests. We had it run continuously (.NET / Visual Studio) and the entire suite probably ran in 3-4 seconds. When you submitted a PR, a suite of longer-running integration tests kicked off that took a couple minutes but a failure automatically kicked the PR back to the dev, and success notified everyone there was a new PR ready for review.
  - pydry 5y ago
    It does. For some reason "code testability" is almost exclusively used to refer to the overuse of dependency inversion to make it easier (although not necessarily more useful) to write unit tests, though. I find that code falls into three camps - integration/logical/mixed. If it's integration code only an integration test is really useful and a unit test is a pointless waste of time that will do little more than mirror the code you wrote and fail when it is changed for any reason at all. If it's logical code a unit test is most useful and integration tests are likely going to be too slow. If it's mixed, integration tests are most useful although drawing out and decoupling the logical from the integration code and testing them separately is better in the long run.
  - bcrosby95 5y ago
    Most of our tests are "integration" tests. With an in memory database which is blazingly fast. Our longer tests use a real database, which is just a config away. Our projects are so database centric, I've almost never seen a true unit test fail. The majority of time it's the tests that use the in memory RDBMS.
Chris_Newton 5y ago
The real value comes from thorough, automated testing suites, whatever their origin is. Automated testing is useful, for sure. However, TDD purists tend to focus on one very specific type of testing: automated unit testing, in the small, where you already know the expected output for given input and can easily specify that output using assertions in code. By its nature, TDD emphasises being testable in that specific sense above all else. I don’t think that is necessarily a good thing, partly because that type of universal unit testing may not be a good strategy for every software system, and partly because other useful properties of the code might be diminished because of the changes needed to make it “testable” in the TDD sense.
- alanfranz 5y ago
  > where you already know the expected output for given input and can easily specify that output using assertions in code Well, "knowing" (or manually generating) the output for your code is required to check whether your code is working. If you execute your code and copypaste its output blindly, you risk testing the wrong thing, don't you. And if you can't specify your code's output, how do you test? BTW, I agree with the fact that TDD is overly pedantic at times.
  - Chris_Newton 5y ago
    Well, "knowing" (or manually generating) the output for your code is required to check whether your code is working. Not all code is written knowing the answer in advance. Sometimes you’re implementing a mathematical model to predict next week’s weather, or the effects of a change in economic policy, or the performance of different hull shapes for a ship under different conditions at sea. My second standard challenge to TDD evangelists is to write a program to draw a classic multicoloured Mandelbrot image, showing an arbitrary rectangular region of the complex plane, using only the mathematical specification without any reference to existing code or output data. This can be done with a short program that is simple enough to verify by eye that it matches the required mathematical specification. However, the problem has an infinite input space, its fractal nature means predicting output values is impossible in the most interesting parts of that input space, and you have no idea where those interesting parts are anyway.

ashtonkem 5y ago

Good observability standards and fast releases catch more bugs than meticulously maintained test suites, imho.

drenei 5y ago
I actually think if you squint testing is part of observability, except it gives you signal during the development phase. And observability as a concept is fundamental to a reliable system.
- ashtonkem 5y ago
  I see where you’re going, but I think if we start defining tests as part of observability then we’ll water that term down to meaninglessness.
  - snovv_crash 5y ago
    If there's a bug and no test suite to try to reproduce it, how do you know you actually fixed it?
    - ashtonkem 5y ago
      I define observability as the tools I use to check on the behavior of my system in ad-how ways in production. This is quite different from tests, which are designed to exercise specific behavior in a fixed environment. Both are used towards the goal of software quality, but they’re different.
      - snovv_crash 5y ago
        I dunno. Working on projects with no tests for me always feels like a complete crapshoot. Any change could have broken some implicit assumption in some module you didn't know depended on it. Even worse is if it's a dynamic language and you don't even have a compiler to do the basic checks. For things like reproducing bugs, in particular, functional tests plus a debugger have been indispensable in seeing how the code actually behaves. Sure, production monitoring is nice. But it really only tells you about high level problems, and there are entire classes of bugs, eg. silently corrupted data, which you'll never see there.
        ashtonkem 5y ago
        You’re arguing against a more extreme version of the point I’m making. I’m not arguing for no tests, I’m arguing that uncle bob over states their capabilities by a lot.

pydry 5y ago

Sacrificing at the altar of unit testability doesnt necessarily make better code. Unit tests' inability to handle state is way too often viewed as a problem with the code it can't properly test than the general crappiness of this form of test.

pc86 5y ago
Doesn't the state get handled in integration tests? The best environment I've worked in (from a testing perspective) had thousands of super fast unit tests. We had it run continuously (.NET / Visual Studio) and the entire suite probably ran in 3-4 seconds. When you submitted a PR, a suite of longer-running integration tests kicked off that took a couple minutes but a failure automatically kicked the PR back to the dev, and success notified everyone there was a new PR ready for review.
- pydry 5y ago
  It does. For some reason "code testability" is almost exclusively used to refer to the overuse of dependency inversion to make it easier (although not necessarily more useful) to write unit tests, though. I find that code falls into three camps - integration/logical/mixed. If it's integration code only an integration test is really useful and a unit test is a pointless waste of time that will do little more than mirror the code you wrote and fail when it is changed for any reason at all. If it's logical code a unit test is most useful and integration tests are likely going to be too slow. If it's mixed, integration tests are most useful although drawing out and decoupling the logical from the integration code and testing them separately is better in the long run.
- bcrosby95 5y ago
  Most of our tests are "integration" tests. With an in memory database which is blazingly fast. Our longer tests use a real database, which is just a config away. Our projects are so database centric, I've almost never seen a true unit test fail. The majority of time it's the tests that use the in memory RDBMS.

Chris_Newton 5y ago

The real value comes from thorough, automated testing suites, whatever their origin is. Automated testing is useful, for sure. However, TDD purists tend to focus on one very specific type of testing: automated unit testing, in the small, where you already know the expected output for given input and can easily specify that output using assertions in code. By its nature, TDD emphasises being testable in that specific sense above all else. I don’t think that is necessarily a good thing, partly because that type of universal unit testing may not be a good strategy for every software system, and partly because other useful properties of the code might be diminished because of the changes needed to make it “testable” in the TDD sense.

alanfranz 5y ago
> where you already know the expected output for given input and can easily specify that output using assertions in code Well, "knowing" (or manually generating) the output for your code is required to check whether your code is working. If you execute your code and copypaste its output blindly, you risk testing the wrong thing, don't you. And if you can't specify your code's output, how do you test? BTW, I agree with the fact that TDD is overly pedantic at times.
- Chris_Newton 5y ago
  Well, "knowing" (or manually generating) the output for your code is required to check whether your code is working. Not all code is written knowing the answer in advance. Sometimes you’re implementing a mathematical model to predict next week’s weather, or the effects of a change in economic policy, or the performance of different hull shapes for a ship under different conditions at sea. My second standard challenge to TDD evangelists is to write a program to draw a classic multicoloured Mandelbrot image, showing an arbitrary rectangular region of the complex plane, using only the mathematical specification without any reference to existing code or output data. This can be done with a short program that is simple enough to verify by eye that it matches the required mathematical specification. However, the problem has an infinite input space, its fractal nature means predicting output values is impossible in the most interesting parts of that input space, and you have no idea where those interesting parts are anyway.