11 ms·

He's right, but also wrong. It really depends on what you are developing. TDD is awesome when fixing bugs (as opposed to writing features), when writing straig

He's right, but also wrong. It really depends on what you are developing. TDD is awesome when fixing bugs (as opposed to writing features), when writing straight up business logic (especially when there are a lot of edge cases) and when you know exactly how something is going to work. It's really bad when doing anything that requires exploration and research (which is the example in the article).

AstralStorm 7y ago
It is still useful, just a different kind of test. High level quality/performance check, rather than testing functions. Produces values, not pass fail. Not really a unit test, an automated integration test of a module. Simple test run to detect crashes or utter failures in CI fashion helps with speed of development too. Also not a unit test. Once we had a research solution and were setting up performance improvements, we have put regression tests in place for the replaced functions and the whole system, derived from earlier module quality tests. Added a few well chosen property based unit tests. This has caught edge cases in the fast implementations.
- hikarudo 7y ago
  We do the same at work. No unit tests, only integration and regression tests. See metamorphic tests (comparing outputs for mutated inputs). One thing I've been trying out recently is adding asserts to code (design by contract): encode function preconditions, postconditions, invariants. Now there are many more ways to "crash" the system while testing. The gist of it is: if you know that there is a bug if a particular condition happens, add an assert encoding this. Now all code that exercises the code with the assert automatically become tests. I've found preconditions to be more useful than the other properties.
  - tempguy9999 7y ago
    No offence but I'm amazed you weren't using assertions (and assertions as pre/post conditions, invars) from the start. They work so well. And true, you get many new ways to blow up the code; all to good. (Edit: pre.post/assertions/invars etc are what I've used in my own code. There's little push in most places I've worked for any testing, never mind anything systematic like this).
    - hikarudo 7y ago
      You're absolutely right. Assertions are so easy to use and there's no barrier to start using them. You can start small, one assertion at a time, no large refactorings required.
      - tempguy9999 7y ago
        Thanks for taking it constructively! For anyone reading this, I originally was taught assertions etc. at university but really got to use them after reading this ppt presentation by Tony Hoare (PDF here. Only took 1/2 hour to find, thanks for getting crappier, google). Really recommended! http://www.cs.cornell.edu/courses/cs2110/2014sp/assignments/a0/assertearlyassertoften.pdf http://www.cs.cornell.edu/courses/cs2110/2014sp/assignments/... Honestly, it's short, please everyone give this 10 minutes of your time. It was one of the main things that started my move from rando programmer to something closer to engineering practice. I just wish companies I worked for gave a damn for it.
simion314 7y ago
He said that too, and sometimes is better to write the tests after you have the functionality finished and working because you know exactly what you expect and you add the tests to prevent feature updates to cause unexpected changes. IMO TDD works if your input and output is clear from the start and the way to get the output is convoluted enough that you need the test.2
moksly 7y ago
I work for a Danish municipality and we buy quite a lot of development from various software houses. Being the public sector we track and benchmark almost everything, and we actually have a dataset on automated testing that’s been running for two decades. It’s hard to use the data, because we’re comparing different projects, teams and suppliers but our data shows no advantage in choosing the companies that are very test-focused. They are often slower, more expensive but have the same amount of incident reports as the companies which tests less or doesn’t do automated test at all.
- okl 7y ago
  TDD = tests before code. That's not what you describe.
  - moksly 7y ago
    I’m not really describing anything, am I? Companies that sell themselves on TDD (among other things) are part of our dataset. Maybe “automated tests” isn’t the exact literal description of that, but I didn’t mean it as such.
- thrax 7y ago
  Any metrics on typescript vs vanilla js (assuming ya'll consume it:) ?
- elviejo 7y ago
  Your dataset is very important. Is there a place where we can publicly analyze it??
  - jacques_chester 7y ago
    Agreed. I'm pro-TDD, but it does nobody any good to ignore honest-to-god data. I would love to know how folks can get access to the data for study.
- Pamar 7y ago
  One question: are they somehow more efficient in answering to an incident report? My conjecture: I am not so surprised at these having the same % of failures - after all the TDD will not magically lead everyone to think of all the possible ramifications; you write tests for the case that you can "imagine" just like a non-TDD developer will implement logic to cover cases they can "imagine". But on the other hand, if after fixing the bug you need to prove that you haven't caused some other side effect, then having a full test suite in place should be more efficient than retest everything again (or less risky than "ok, fixing module W should not impact X, Y and Z, so we test X only and ship...").
tomxor 7y ago
> He's right, but also wrong. It really depends on what you are developing. The more useful thing to notice beyond TDD is how this is true of almost every development model and programming pattern (including the patterns built into languages that pretend they are not) - development processes and programming patterns are almost never generally applicable - The problems come when people who don't understand this attempt to apply them generally and blindly, e.g the classic go a read a patterns book or development strategy book and now everything is a nail for their preferred choice of hammer. I'm not being defensive of TDD though, just because it's subjectively useful does not make it equal, to me TDD looks to have very narrow utility in general for the vast majority of projects it's going to get in the way.
simonh 7y ago
One thing to consider I think is that TDD isn't just about the tests. It's also about architecting the software into small functions and lightweight classes, which are easy to write concise tests for and then implement incrementally. To TDD advocates, this resulting characteristic of the code is a valuable goal in itself, even just as a side effect of TDD.
- blunte 7y ago
  Right, so as the author suggests, maybe TDD's real benefit is that it makes you think and plan a bit more before jumping into coding. Therefore, we should have MDD - miniplan-driven-development: make a mini plan for the feature being developed, not so dissimilarly from what you would do with the first pass of TDD. The difference is, once you start implementing that plan and realize the problem is much different, you don't have to go back and keep modifying your original miniplan. You just do what you would normally do. What you gain is probably a partial improvement in your development process without much cost.
  - jacques_chester 7y ago
    You may find A Discipline for Software Engineering by Watts Humphreys to be an interesting read.
- cc81 7y ago
  My experience is that the resulting architecture will sometimes end up with characteristics that makes it easy to test but more complex, exposes things it should not and too many layers of indirection. DHH wrote some about it here: https://dhh.dk/2014/test-induced-design-damage.html https://dhh.dk/2014/test-induced-design-damage.html I do like automated tests (unit and integration) and often TDD is appropriate. However tests are not the end goal of an architecture. The code should be maintainable, easy to follow and simple to reduce number of bugs. If TDD starts to change that by adding layers of indirection or changing the purpose of something just so you can do a fast unit test then something is wrong.
kissgyorgy 7y ago
Exactly! Well put.
Gehinnn 7y ago
> It's really bad when doing anything that requires exploration and research I wouldn't say that, I have an counterexample ([1]): It is clear what the refactoring (that is to be implemented) should do and how to state that ([2]), but it usually requires exploration and research how to work with the AST and which methods to use. I would even say that TDD is awesome for exploring, researching, for getting a feeling of the problem's context, what to do and what the caveats are, as long as neither of that is required for stating the test itself. Having tests first, you can simply start the debugger, try things out and with hot reloading you can write the code while running the specific test that you want to concentrate on. Visualization tools like [3] can greatly help exploring how to solve and implement the current test while you debug it. [1] https://blog.hediet.de/post/implementing_typescript_refactorings_with_hot_reloading https://blog.hediet.de/post/implementing_typescript_refactor... [2] https://github.com/hediet/hediet-ts-refactoring-lsp/blob/master/language-service-plugin/test/main.test.ts#L32 https://github.com/hediet/hediet-ts-refactoring-lsp/blob/mas... [3] https://marketplace.visualstudio.com/items?itemName=hediet.debug-visualizer https://marketplace.visualstudio.com/items?itemName=hediet.d...
cryptica 7y ago
TDD can work if the tests are integration tests and each test case covers an entire feature (one which can be clearly described using non-technical language). Integration tests only lock down the high level interfaces and behaviors so they allow you to keep reworking the guts of the system without ever having to rewrite the test cases. Unit tests are mostly only good in the very final stages of the project when you have a fully working, essentially perfect, bug-free system and you just want to lock it all down so that nobody can make even the tiniest changes to the code without breaking all the tests and becoming insecure about their abilities in the face of such a remarkable code base. If you're writing unit tests for poorly architected code, you're essentially locking down this horrible architecture and forcing all future developers to find complex ways to work around it instead of allowing them to confidently make the necessary restructurings to allow them to cleanly build on top (this kind of "I'm not breaking any features" confidence can only come from integration tests). When it comes to complex systems, writing unit tests too early in the project is a bad idea; especially when combined with an agile development methodology. It's like building a skyscraper over several years and perfectly analysing every steel rod, every brick and every pane of glass as it is added to the structure but never actually stepping back to check that the structure as a whole is actually pointing upwards.
dgb23 7y ago
Very well put. It is a great paradigm for problems that you understand very well. But I think we can also define it more inclusive and still be fine: TDD is a good paradigm for programming against well defined APIs. For example if you are handed an API specification then locking down your interface(s) with tests in advance can be very helpful. What is even more helpful is to have a higher level language or abstraction that encodes the specification in detail and generates the tests for you (Clojure spec comes to mind). edit: typo
everdev 7y ago
Or a classic take from DHH 7 years ago, just test the big things: https://signalvnoise.com/posts/3159-testing-like-the-tsa https://signalvnoise.com/posts/3159-testing-like-the-tsa
dnautics 7y ago
I don't typically tdd, but I'll say TDD is also awesome when you are in a crunch and working on some feature that has, say more than one dependency and more than say four corner cases and you are all amuddle about which one you should do first. Especially, london-style? Is it? Where you write out your trivial stub implementation first, helps me sort it out and not worry about thinking of everything.

AstralStorm 7y ago

It is still useful, just a different kind of test. High level quality/performance check, rather than testing functions. Produces values, not pass fail. Not really a unit test, an automated integration test of a module. Simple test run to detect crashes or utter failures in CI fashion helps with speed of development too. Also not a unit test. Once we had a research solution and were setting up performance improvements, we have put regression tests in place for the replaced functions and the whole system, derived from earlier module quality tests. Added a few well chosen property based unit tests. This has caught edge cases in the fast implementations.

hikarudo 7y ago
We do the same at work. No unit tests, only integration and regression tests. See metamorphic tests (comparing outputs for mutated inputs). One thing I've been trying out recently is adding asserts to code (design by contract): encode function preconditions, postconditions, invariants. Now there are many more ways to "crash" the system while testing. The gist of it is: if you know that there is a bug if a particular condition happens, add an assert encoding this. Now all code that exercises the code with the assert automatically become tests. I've found preconditions to be more useful than the other properties.
- tempguy9999 7y ago
  No offence but I'm amazed you weren't using assertions (and assertions as pre/post conditions, invars) from the start. They work so well. And true, you get many new ways to blow up the code; all to good. (Edit: pre.post/assertions/invars etc are what I've used in my own code. There's little push in most places I've worked for any testing, never mind anything systematic like this).
  - hikarudo 7y ago
    You're absolutely right. Assertions are so easy to use and there's no barrier to start using them. You can start small, one assertion at a time, no large refactorings required.
    - tempguy9999 7y ago
      Thanks for taking it constructively! For anyone reading this, I originally was taught assertions etc. at university but really got to use them after reading this ppt presentation by Tony Hoare (PDF here. Only took 1/2 hour to find, thanks for getting crappier, google). Really recommended! http://www.cs.cornell.edu/courses/cs2110/2014sp/assignments/a0/assertearlyassertoften.pdf http://www.cs.cornell.edu/courses/cs2110/2014sp/assignments/... Honestly, it's short, please everyone give this 10 minutes of your time. It was one of the main things that started my move from rando programmer to something closer to engineering practice. I just wish companies I worked for gave a damn for it.

simion314 7y ago

He said that too, and sometimes is better to write the tests after you have the functionality finished and working because you know exactly what you expect and you add the tests to prevent feature updates to cause unexpected changes. IMO TDD works if your input and output is clear from the start and the way to get the output is convoluted enough that you need the test.2

moksly 7y ago

I work for a Danish municipality and we buy quite a lot of development from various software houses. Being the public sector we track and benchmark almost everything, and we actually have a dataset on automated testing that’s been running for two decades. It’s hard to use the data, because we’re comparing different projects, teams and suppliers but our data shows no advantage in choosing the companies that are very test-focused. They are often slower, more expensive but have the same amount of incident reports as the companies which tests less or doesn’t do automated test at all.

okl 7y ago
TDD = tests before code. That's not what you describe.
- moksly 7y ago
  I’m not really describing anything, am I? Companies that sell themselves on TDD (among other things) are part of our dataset. Maybe “automated tests” isn’t the exact literal description of that, but I didn’t mean it as such.
thrax 7y ago
Any metrics on typescript vs vanilla js (assuming ya'll consume it:) ?
elviejo 7y ago
Your dataset is very important. Is there a place where we can publicly analyze it??
- jacques_chester 7y ago
  Agreed. I'm pro-TDD, but it does nobody any good to ignore honest-to-god data. I would love to know how folks can get access to the data for study.
Pamar 7y ago
One question: are they somehow more efficient in answering to an incident report? My conjecture: I am not so surprised at these having the same % of failures - after all the TDD will not magically lead everyone to think of all the possible ramifications; you write tests for the case that you can "imagine" just like a non-TDD developer will implement logic to cover cases they can "imagine". But on the other hand, if after fixing the bug you need to prove that you haven't caused some other side effect, then having a full test suite in place should be more efficient than retest everything again (or less risky than "ok, fixing module W should not impact X, Y and Z, so we test X only and ship...").

tomxor 7y ago

> He's right, but also wrong. It really depends on what you are developing. The more useful thing to notice beyond TDD is how this is true of almost every development model and programming pattern (including the patterns built into languages that pretend they are not) - development processes and programming patterns are almost never generally applicable - The problems come when people who don't understand this attempt to apply them generally and blindly, e.g the classic go a read a patterns book or development strategy book and now everything is a nail for their preferred choice of hammer. I'm not being defensive of TDD though, just because it's subjectively useful does not make it equal, to me TDD looks to have very narrow utility in general for the vast majority of projects it's going to get in the way.

simonh 7y ago

One thing to consider I think is that TDD isn't just about the tests. It's also about architecting the software into small functions and lightweight classes, which are easy to write concise tests for and then implement incrementally. To TDD advocates, this resulting characteristic of the code is a valuable goal in itself, even just as a side effect of TDD.

blunte 7y ago
Right, so as the author suggests, maybe TDD's real benefit is that it makes you think and plan a bit more before jumping into coding. Therefore, we should have MDD - miniplan-driven-development: make a mini plan for the feature being developed, not so dissimilarly from what you would do with the first pass of TDD. The difference is, once you start implementing that plan and realize the problem is much different, you don't have to go back and keep modifying your original miniplan. You just do what you would normally do. What you gain is probably a partial improvement in your development process without much cost.
- jacques_chester 7y ago
  You may find A Discipline for Software Engineering by Watts Humphreys to be an interesting read.
cc81 7y ago
My experience is that the resulting architecture will sometimes end up with characteristics that makes it easy to test but more complex, exposes things it should not and too many layers of indirection. DHH wrote some about it here: https://dhh.dk/2014/test-induced-design-damage.html https://dhh.dk/2014/test-induced-design-damage.html I do like automated tests (unit and integration) and often TDD is appropriate. However tests are not the end goal of an architecture. The code should be maintainable, easy to follow and simple to reduce number of bugs. If TDD starts to change that by adding layers of indirection or changing the purpose of something just so you can do a fast unit test then something is wrong.

kissgyorgy 7y ago

Exactly! Well put.

Gehinnn 7y ago

> It's really bad when doing anything that requires exploration and research I wouldn't say that, I have an counterexample ([1]): It is clear what the refactoring (that is to be implemented) should do and how to state that ([2]), but it usually requires exploration and research how to work with the AST and which methods to use. I would even say that TDD is awesome for exploring, researching, for getting a feeling of the problem's context, what to do and what the caveats are, as long as neither of that is required for stating the test itself. Having tests first, you can simply start the debugger, try things out and with hot reloading you can write the code while running the specific test that you want to concentrate on. Visualization tools like [3] can greatly help exploring how to solve and implement the current test while you debug it. [1] https://blog.hediet.de/post/implementing_typescript_refactorings_with_hot_reloading https://blog.hediet.de/post/implementing_typescript_refactor... [2] https://github.com/hediet/hediet-ts-refactoring-lsp/blob/master/language-service-plugin/test/main.test.ts#L32 https://github.com/hediet/hediet-ts-refactoring-lsp/blob/mas... [3] https://marketplace.visualstudio.com/items?itemName=hediet.debug-visualizer https://marketplace.visualstudio.com/items?itemName=hediet.d...

cryptica 7y ago

TDD can work if the tests are integration tests and each test case covers an entire feature (one which can be clearly described using non-technical language). Integration tests only lock down the high level interfaces and behaviors so they allow you to keep reworking the guts of the system without ever having to rewrite the test cases. Unit tests are mostly only good in the very final stages of the project when you have a fully working, essentially perfect, bug-free system and you just want to lock it all down so that nobody can make even the tiniest changes to the code without breaking all the tests and becoming insecure about their abilities in the face of such a remarkable code base. If you're writing unit tests for poorly architected code, you're essentially locking down this horrible architecture and forcing all future developers to find complex ways to work around it instead of allowing them to confidently make the necessary restructurings to allow them to cleanly build on top (this kind of "I'm not breaking any features" confidence can only come from integration tests). When it comes to complex systems, writing unit tests too early in the project is a bad idea; especially when combined with an agile development methodology. It's like building a skyscraper over several years and perfectly analysing every steel rod, every brick and every pane of glass as it is added to the structure but never actually stepping back to check that the structure as a whole is actually pointing upwards.

dgb23 7y ago

Very well put. It is a great paradigm for problems that you understand very well. But I think we can also define it more inclusive and still be fine: TDD is a good paradigm for programming against well defined APIs. For example if you are handed an API specification then locking down your interface(s) with tests in advance can be very helpful. What is even more helpful is to have a higher level language or abstraction that encodes the specification in detail and generates the tests for you (Clojure spec comes to mind). edit: typo

everdev 7y ago

Or a classic take from DHH 7 years ago, just test the big things: https://signalvnoise.com/posts/3159-testing-like-the-tsa https://signalvnoise.com/posts/3159-testing-like-the-tsa

dnautics 7y ago

I don't typically tdd, but I'll say TDD is also awesome when you are in a crunch and working on some feature that has, say more than one dependency and more than say four corner cases and you are all amuddle about which one you should do first. Especially, london-style? Is it? Where you write out your trivial stub implementation first, helps me sort it out and not worry about thinking of everything.