3 ms·

TDD often doesn’t work for greenfield development because you have no idea how the software will be designed and structured until after you’ve done substantial

TDD often doesn’t work for greenfield development because you have no idea how the software will be designed and structured until after you’ve done substantial implementation to prove out the viability of the software architecture and how implementation details interact. Major aspects of software architecture may need to be redesigned and reimplemented multiple times in order to better optimize the tradeoffs that manifest. By definition, greenfield development doesn’t lend itself to trivial analysis upfront, particularly for non-trivial systems. Tests don’t help you design software, they help verify properties of the implementation. For non-trivial greenfield systems, the full set of properties (which is a very high dimensionality tradeoff space) and their expression in the implementation isn’t really knowable ahead of time.

lll-o-lll 2y ago
Designing the tests means you have to design the software. This is a low-key way of enforcing some upfront design, which is good. Also, this is not specific to software. Defining how you will validate a system, before you go build the system, is simply good engineering.
- deleted 2y ago
  [deleted]
- jandrewrogers 2y ago
  The design of some software, even in substantial detail, is not sufficient to design tests of an implementation. Good tests can be completely dependent on the implementation details far below the level of design because those implementation details determine how the thing you are testing can even be measured. How it is measured isn’t actually important. You can get closer with extreme (over-)specification of the implementation, which almost no one does because it comes with an inordinate cost, but even that can fail because software properties are also a function of the hardware it runs on. To some extent, modeling the interaction of software on hardware for the purposes of building good tests is by actually measuring running code on real hardware. Hardcoding specific implementation detail assumptions into the design of complex non-trivial systems so that you can write a test is a brittle way to go about writing software. There is a reason that waterfall-style never seems to produce good systems in a reasonable amount of time or money. I write non-trivial greenfield systems and put inordinate amounts of design work into them (literally years in some cases). I have never found a case where that allowed me to write useful tests prior to implementation. At a minimum you need a complete architectural skeleton that runs and has had its non-functional properties verified analytically before you can write tests that you’ll likely use. I’m not against writing tests. I invest far more in testing than most devs due to atypically high reliability and robustness requirements. I’m against the complete waste of time that would be writing tests before there is sufficient implementation to design maximally efficient and appropriate tests. Testing of some software is extremely expensive in both time and money, so optimizing test effectiveness and efficiency becomes valuable.
- TeMPOraL 2y ago
  Designing tests at the unit level requires you to design low-level/"tactical" aspects of your software, which automatically forces you to commit to a certain general/"strategic" design of the overall program. This is often the opposite of what you want in greenfield work - you want to first rapidly explore the space of high-level designs, not precommit to the first one you thought of that makes tests easier. Starting with TDD is like being a general whose recommendation to every international politics issue is to commence training for a combined arms assault on the capital of the country's historical enemy, because that's something that can be iterated to provide good estimates and error-free execution. Except, that general misses other, possibly much more effective approaches, such as airstrikes, assassinations, supplying domestic terrorists in enemy territory, or... not shooting anything at all because we're not at war and the problem can be trivially solved by respective VPs over some drinks. Or the problem isn't even a problem. Keep the TDD General around for when you're actually fighting a land war - but for the love of $deity, don't let him drive international policy in peace time.
  - hitchstory 2y ago
    TDD != starting at unit level. It's about red-green-refactor. Or, it should be anyway. People talk about how it "drives" design but I dont think it should. I think tests should be as design-agnostic as possible, providing freedom to redesign. I always start with an end to end test. These dont force you to commit to anything.
    - voiceofunreason 2y ago
      There is definitely room for confusion as to whether the "design" that is "driven" by TDD is design-the-noun or design-the-verb.
motorest 2y ago
> TDD often doesn’t work for greenfield development because you have no idea how the software will be designed and structured until after you’ve done substantial implementation to prove out the viability of the software architecture and how implementation details interact. I don't agree. TDD is only relevant regarding the interfaces used by units of behavior. Software architecture isn't something that you design or verify or even assess with unit tests. Software architecture is purely a design task, and redoing software architecture is a kin to rewriting the whole project.
- jandrewrogers 2y ago
  Most non-trivial systems software has minimal unit testing. It doesn’t measure anything important in those systems. Integration tests are everything because the main functional properties being designed are the dynamic behavior of interactions between components and the hardware. In performance-engineered systems, a component’s behavior in isolation is not its behavior in the system, and that behavior determines correctness. Basically the same fallacy as most micro-benchmarking. Resource availability and locality vary dynamically, and the software design has to account for this, which is extremely non-trivial to do without code you can measure in the system context. And yes, modifying software architectures to address design issues that show up in implementation details or to slice some non-obvious tradeoffs differently is expensive, but is almost always required to some extent. That’s a cost of business if you are doing non-trivial greenfield software, it comes with the territory.
epolanski 2y ago
Disagree, in fact TDD forces you to face your API ergonomics very soon. If anything it helps.
- jandrewrogers 2y ago
  There is a lot of software that isn’t an API, or the API is just a thin wrapper around the real implementation, or the API is one of the least important aspects of the system from a customer perspective. For example, systems software is mostly about testing system behavior under various loads and resource constraints. The ability to even measure that reliably without the measurement interfering with the software operation is dependent on a lot of implementation detail.
hitchstory 2y ago
This is why on greenfield projects I start with an end to end test and do red-green-refactor with it. This lets me defer many architectural decisions (those decisions are usually of a better quality when done during the refactor step) while locking in behavior up front. Call it what you like. Some people would be insistent that this is TDD. Some would say it isnt because it's not test driven design. Some say it is TFD. I dont really care what it's called, for me it doesnt make sense to develop serious non-spike/non-POC code any other way. It's one of the few things in tech where my opinion has hardened and not softened over the years. "Unit tests" on the other hand, blech, sometimes useful, very overused.
binary132 2y ago
Good TDD is mainly about API semantics specification, not implementation details. Of course, everything has an API, so you can take it down to whatever level of detail is needed and useful, but specifying the API of your application without knowing the implementation details is basically the entire problem you just described.

lll-o-lll 2y ago

Designing the tests means you have to design the software. This is a low-key way of enforcing some upfront design, which is good. Also, this is not specific to software. Defining how you will validate a system, before you go build the system, is simply good engineering.

deleted 2y ago
[deleted]
jandrewrogers 2y ago
The design of some software, even in substantial detail, is not sufficient to design tests of an implementation. Good tests can be completely dependent on the implementation details far below the level of design because those implementation details determine how the thing you are testing can even be measured. How it is measured isn’t actually important. You can get closer with extreme (over-)specification of the implementation, which almost no one does because it comes with an inordinate cost, but even that can fail because software properties are also a function of the hardware it runs on. To some extent, modeling the interaction of software on hardware for the purposes of building good tests is by actually measuring running code on real hardware. Hardcoding specific implementation detail assumptions into the design of complex non-trivial systems so that you can write a test is a brittle way to go about writing software. There is a reason that waterfall-style never seems to produce good systems in a reasonable amount of time or money. I write non-trivial greenfield systems and put inordinate amounts of design work into them (literally years in some cases). I have never found a case where that allowed me to write useful tests prior to implementation. At a minimum you need a complete architectural skeleton that runs and has had its non-functional properties verified analytically before you can write tests that you’ll likely use. I’m not against writing tests. I invest far more in testing than most devs due to atypically high reliability and robustness requirements. I’m against the complete waste of time that would be writing tests before there is sufficient implementation to design maximally efficient and appropriate tests. Testing of some software is extremely expensive in both time and money, so optimizing test effectiveness and efficiency becomes valuable.
TeMPOraL 2y ago
Designing tests at the unit level requires you to design low-level/"tactical" aspects of your software, which automatically forces you to commit to a certain general/"strategic" design of the overall program. This is often the opposite of what you want in greenfield work - you want to first rapidly explore the space of high-level designs, not precommit to the first one you thought of that makes tests easier. Starting with TDD is like being a general whose recommendation to every international politics issue is to commence training for a combined arms assault on the capital of the country's historical enemy, because that's something that can be iterated to provide good estimates and error-free execution. Except, that general misses other, possibly much more effective approaches, such as airstrikes, assassinations, supplying domestic terrorists in enemy territory, or... not shooting anything at all because we're not at war and the problem can be trivially solved by respective VPs over some drinks. Or the problem isn't even a problem. Keep the TDD General around for when you're actually fighting a land war - but for the love of $deity, don't let him drive international policy in peace time.
- hitchstory 2y ago
  TDD != starting at unit level. It's about red-green-refactor. Or, it should be anyway. People talk about how it "drives" design but I dont think it should. I think tests should be as design-agnostic as possible, providing freedom to redesign. I always start with an end to end test. These dont force you to commit to anything.
  - voiceofunreason 2y ago
    There is definitely room for confusion as to whether the "design" that is "driven" by TDD is design-the-noun or design-the-verb.

motorest 2y ago

> TDD often doesn’t work for greenfield development because you have no idea how the software will be designed and structured until after you’ve done substantial implementation to prove out the viability of the software architecture and how implementation details interact. I don't agree. TDD is only relevant regarding the interfaces used by units of behavior. Software architecture isn't something that you design or verify or even assess with unit tests. Software architecture is purely a design task, and redoing software architecture is a kin to rewriting the whole project.

jandrewrogers 2y ago
Most non-trivial systems software has minimal unit testing. It doesn’t measure anything important in those systems. Integration tests are everything because the main functional properties being designed are the dynamic behavior of interactions between components and the hardware. In performance-engineered systems, a component’s behavior in isolation is not its behavior in the system, and that behavior determines correctness. Basically the same fallacy as most micro-benchmarking. Resource availability and locality vary dynamically, and the software design has to account for this, which is extremely non-trivial to do without code you can measure in the system context. And yes, modifying software architectures to address design issues that show up in implementation details or to slice some non-obvious tradeoffs differently is expensive, but is almost always required to some extent. That’s a cost of business if you are doing non-trivial greenfield software, it comes with the territory.

epolanski 2y ago

Disagree, in fact TDD forces you to face your API ergonomics very soon. If anything it helps.

jandrewrogers 2y ago
There is a lot of software that isn’t an API, or the API is just a thin wrapper around the real implementation, or the API is one of the least important aspects of the system from a customer perspective. For example, systems software is mostly about testing system behavior under various loads and resource constraints. The ability to even measure that reliably without the measurement interfering with the software operation is dependent on a lot of implementation detail.

hitchstory 2y ago

This is why on greenfield projects I start with an end to end test and do red-green-refactor with it. This lets me defer many architectural decisions (those decisions are usually of a better quality when done during the refactor step) while locking in behavior up front. Call it what you like. Some people would be insistent that this is TDD. Some would say it isnt because it's not test driven design. Some say it is TFD. I dont really care what it's called, for me it doesnt make sense to develop serious non-spike/non-POC code any other way. It's one of the few things in tech where my opinion has hardened and not softened over the years. "Unit tests" on the other hand, blech, sometimes useful, very overused.

binary132 2y ago

Good TDD is mainly about API semantics specification, not implementation details. Of course, everything has an API, so you can take it down to whatever level of detail is needed and useful, but specifying the API of your application without knowing the implementation details is basically the entire problem you just described.