3 ms·

This is good advice... but it's hard to follow if you don't understand the costs of particular forms of testing, or what benefits they provide. This was part of

This is good advice... but it's hard to follow if you don't understand the costs of particular forms of testing, or what benefits they provide. This was part of the motivation for creating the model I referenced in a comment above.

brudgers 10y ago
Your earlier question does not make a business case and Markham's answer might be somewhat similar to mine here (unsurprisingly perhaps since I sort of recycle things I read on Hacker News when it comes to startup specific advice and my own experience when it comes ordinary business advice). This sort of falls in the category of both: there is a startup component (survival) and a general business component (YAGNI based cash flow). The tricky thing about TDD is that Uncle Bob packages it up with morals and ethics and craftsmanship. It's probably a big reason why he is a successful consultant in addition to having coding chops...and I don't disagree with his moral and ethical opinions but they are about writing code rather than running a business and here the important decision is running a business and that's what Markham emphasizes and emphasizes particularly well in his reply to Cooper's response. I'd put it this way, the first order benefit of TDD is confidence that refactoring will pass fewer regressions to production. But a successful startup does not depend on refactorings because growth drives rearchitecting and pivoting and those mean that many/much/all existing tests may become irrelevant. Riffing a bit, it might be that in a startup the way to determine the benefit of tests is do they get the company to the point where those tests become irrelevant more quickly? Most startups default on technical debt by either running out of money or by throwing out the code base because they acquire different operating parameters due to scale.
- itamarst 10y ago
  Right, so in my model, there's four goals you might have for testing: understanding users, understanding runtime behavior (of your software), correct functionality, stable functionality. Which set you choose depends on business goals. In a startup environment your goal is often understanding users (does anyone care about this product/feature? is it usable?) followed by correctness. And as you say, stability isn't an immediate concern if you're rewriting everything every two weeks, which is the case for many layers of code. "I'd put it this way, the first order benefit of TDD is confidence that refactoring will pass fewer regressions to production." - I'd say automated tests, including unit tests and therefore TDD, only give you stable functionality. Not correctness, since only a human can determine that (see essay for fleshed out argument - https://codewithoutrules.com/2017/03/26/why-how-test-software/ https://codewithoutrules.com/2017/03/26/why-how-test-softwar..., section on "What means of testing can you use?"). But unit tests (and thus TDD) for code that isn't going to be a stable is actually a negative, because you're trying to enforce stability on something that keeps changing. -- Could you link to the Cooper/Markham discussion?
  - brudgers 10y ago
    Cooper and Markham were at the top of the link in the question. {edit: I have been mistaking you for the OP sandGorgon} -- I can see how A/B testing might help with understanding users, but to me, A/B testing is orthogonal to Test Driven Development and early on A/B testing would probably not have enough test cases to generate deep statistically significant insights due to small sample sizes and the need to make gross movements rather than fine tuning a working process. -- For Uncle Bob, TDD is a business process because he is selling consulting at some hourly/weekly/monthly rate. His client base will tend to be looking at improving stable business processes not those taking a spin of startup roulette. For a technology startup, what gets developed in house will tend to be strategic in the sense that it has a big effect on the company's velocity. Getting 'there' in three months is more than eight times as good as getting there in two years.
    - itamarst 10y ago
      Ah, thanks, will read through. A/B testing is orthogonal to TDD, yes, and that's really the point: there's many forms of testing (usability testing, testing user interest, as well as A/B testing) whose goal is completely different than verifying a specification. Often they are useful when creating the specification. And maintaining stable software is very different than spinning up new product, yes. So you need to start with business goals, work down to testing goals, and then choose testing techniques based on those.
- itamarst 10y ago
  I should add that pure craftsmanship strikes me as a limited point of view: what is the point of "high quality" if no one uses your software? https://blog.ometer.com/2016/05/04/professional-corner-cutting/ https://blog.ometer.com/2016/05/04/professional-corner-cutti... has a better take than just "quality all the time" that TDD advocates sometimes take on.
  - brudgers 10y ago
    My impression is that Uncle Bob works for clients that have software that people are already using at some meaningful volume (volume rather than scale). I could be wrong.
    - itamarst 10y ago
      Well, for TDD specifically I tend to think of it as a testing methodology for libraries. Application logic tends to be hard to test that way since they're about human interaction. Of course, you want applications built out of libraries. But eventually if you do that you end up at the point of "this is pure UI code and unit tests will do me no good." So I don't think it's just "stable, mature product" vs. "new product". It's also about code structure and where the abstraction layers are, and which abstraction layers are stable and therefore amenable to automated tests. A UI can change in ways that a human will not even notice but will break brittle automated tests. There's workarounds, of course, and probably it's possible to build much better tooling than actually exists.

brudgers 10y ago

Your earlier question does not make a business case and Markham's answer might be somewhat similar to mine here (unsurprisingly perhaps since I sort of recycle things I read on Hacker News when it comes to startup specific advice and my own experience when it comes ordinary business advice). This sort of falls in the category of both: there is a startup component (survival) and a general business component (YAGNI based cash flow). The tricky thing about TDD is that Uncle Bob packages it up with morals and ethics and craftsmanship. It's probably a big reason why he is a successful consultant in addition to having coding chops...and I don't disagree with his moral and ethical opinions but they are about writing code rather than running a business and here the important decision is running a business and that's what Markham emphasizes and emphasizes particularly well in his reply to Cooper's response. I'd put it this way, the first order benefit of TDD is confidence that refactoring will pass fewer regressions to production. But a successful startup does not depend on refactorings because growth drives rearchitecting and pivoting and those mean that many/much/all existing tests may become irrelevant. Riffing a bit, it might be that in a startup the way to determine the benefit of tests is do they get the company to the point where those tests become irrelevant more quickly? Most startups default on technical debt by either running out of money or by throwing out the code base because they acquire different operating parameters due to scale.

itamarst 10y ago
Right, so in my model, there's four goals you might have for testing: understanding users, understanding runtime behavior (of your software), correct functionality, stable functionality. Which set you choose depends on business goals. In a startup environment your goal is often understanding users (does anyone care about this product/feature? is it usable?) followed by correctness. And as you say, stability isn't an immediate concern if you're rewriting everything every two weeks, which is the case for many layers of code. "I'd put it this way, the first order benefit of TDD is confidence that refactoring will pass fewer regressions to production." - I'd say automated tests, including unit tests and therefore TDD, only give you stable functionality. Not correctness, since only a human can determine that (see essay for fleshed out argument - https://codewithoutrules.com/2017/03/26/why-how-test-software/ https://codewithoutrules.com/2017/03/26/why-how-test-softwar..., section on "What means of testing can you use?"). But unit tests (and thus TDD) for code that isn't going to be a stable is actually a negative, because you're trying to enforce stability on something that keeps changing. -- Could you link to the Cooper/Markham discussion?
- brudgers 10y ago
  Cooper and Markham were at the top of the link in the question. {edit: I have been mistaking you for the OP sandGorgon} -- I can see how A/B testing might help with understanding users, but to me, A/B testing is orthogonal to Test Driven Development and early on A/B testing would probably not have enough test cases to generate deep statistically significant insights due to small sample sizes and the need to make gross movements rather than fine tuning a working process. -- For Uncle Bob, TDD is a business process because he is selling consulting at some hourly/weekly/monthly rate. His client base will tend to be looking at improving stable business processes not those taking a spin of startup roulette. For a technology startup, what gets developed in house will tend to be strategic in the sense that it has a big effect on the company's velocity. Getting 'there' in three months is more than eight times as good as getting there in two years.
  - itamarst 10y ago
    Ah, thanks, will read through. A/B testing is orthogonal to TDD, yes, and that's really the point: there's many forms of testing (usability testing, testing user interest, as well as A/B testing) whose goal is completely different than verifying a specification. Often they are useful when creating the specification. And maintaining stable software is very different than spinning up new product, yes. So you need to start with business goals, work down to testing goals, and then choose testing techniques based on those.
itamarst 10y ago
I should add that pure craftsmanship strikes me as a limited point of view: what is the point of "high quality" if no one uses your software? https://blog.ometer.com/2016/05/04/professional-corner-cutting/ https://blog.ometer.com/2016/05/04/professional-corner-cutti... has a better take than just "quality all the time" that TDD advocates sometimes take on.
- brudgers 10y ago
  My impression is that Uncle Bob works for clients that have software that people are already using at some meaningful volume (volume rather than scale). I could be wrong.
  - itamarst 10y ago
    Well, for TDD specifically I tend to think of it as a testing methodology for libraries. Application logic tends to be hard to test that way since they're about human interaction. Of course, you want applications built out of libraries. But eventually if you do that you end up at the point of "this is pure UI code and unit tests will do me no good." So I don't think it's just "stable, mature product" vs. "new product". It's also about code structure and where the abstraction layers are, and which abstraction layers are stable and therefore amenable to automated tests. A UI can change in ways that a human will not even notice but will break brittle automated tests. There's workarounds, of course, and probably it's possible to build much better tooling than actually exists.