9 ms·

From the article: Because you want to ensure that you always pass the majority of tests, you tend to think about this when you change and extend the program. Yo

by ekidd 11y ago

From the article: Because you want to ensure that you always pass the majority of tests, you tend to think about this when you change and extend the program. You therefore are more reluctant to make large-scale changes that will lead to the failure of lots of tests. Psychologically, you become conservative to avoid breaking lots of tests. Interesting. I've often found that the lack of tests leaves me absolutely terrified of making changes to large Ruby and JavaScript applications. I mean, if a test breaks, then I get a nice message right on the spot. But if I don't have tests, then the application itself breaks, and I may not find out until it's been put into production on a high-volume site or shipped to users. Once an application crosses, say, 25,000 lines of code, it's hard to keep an entire program in my head, especially in a dynamic language and with multiple authors working on the code base. Under these conditions, large scale refactorings or framework upgrades can cause massive test failures, but the only alternative is to cause massive, unknown breakage.

ivanhoe 11y ago
From what I understand author is not questioning the usefulness of testing, just TDD approach of writing tests first. I personally prefer a sandwich approach, I start with code first, and then write tests for it as soon as I have that little piece of logic finished (usually a method or two). Then I add more code, followed by more test, and so on. Works great for me and my team.
- drewhk 11y ago
  I nowadays do an approach where I don't write tests first, but I write down the test titles (I use a language where the testing framework gives a nice DSL to write nicely readable test names compared to method names) and I just mark them as pending (which will report as no test failures or passes but ignored cases). I rarely end up with the same test ideas and names which I started from, but since I only start with names which do not hinder refactorings this worked out surprisingly nicely for me.
- atom-morgan 11y ago
  I agree. If people started viewing coding as an art instead of a strict science I think people would feel more comfortable doing something like this instead of a strict Test First or Test After.
- hsitz 11y ago
  My impression, too, the author explicitly says he understands value of unit testing and will continue to do it, but he's going to give up on the idea of writing tests before he writes the actual program code. Strange, from what I can tell 90+ percent of comments in this thread treat the author as if he said he was giving up tests entirely.
BonsaiDen 11y ago
One good way to the limit breakage in such cases is to solely perform black box tests on the API level. In case of our Node.js based Backends we don't ever write a single classical unit test, instead we have a custom Framework built on top of Mocha which performs tests on the HTTP layer against all of our endpoints. This works remarkable well in practice and allows for large scale refactorings under the hood with little to no impact on the tests. We can also mock databases, memcached, redis and graylog on their respective http/tcp/udp level. This in turn means no custom build mocks which could break when refactoring. The tests itself also contain no logic, they are pretty much just chained method calls with data that should go in and an expected response that should come out, along with a specification of all external resource our API fetches during the request and their responses etc. Any unexpected outgoing HTTP request from our server will actually result in a test failure. As for scaling this approach, from our experience it works quite well, especially when you have lots of complicated interactions with customer APIs during your requests since the flows are super quick to set up.
- raverbashing 11y ago
  This is a good idea, hence, expect hate from TDD purists
- cousin_it 11y ago
  Yeah, I mostly prefer end-to-end tests as well. Though to be fair, they are often slower than unit tests, because you need to start up the whole system. And they are worse at pinpointing problems, though that doesn't seem to be a big deal in practice.
  - BonsaiDen 11y ago
    I in part agree with the pinpointing, though in my experience this really boils down to an issue of scope and how much of the data you want to test in each of your tests. E.g. an API returning user data, do you have one test for the whole set of data or one test per field. For our use case we have some pretty "fancy" deep-equals logic for nested structures and allow to specific fields as "to be ignored" in our test expectations. When it comes to speed, the most important part is to cut out everything you don't need, for us this means that our testing framework completely throws away the internals of Node.js HTTP layer. We never create a single TCP socket in our tests, which gives an incredible speed up. As a bonus this also allows to test timeouts and low level HTTP errors in < 1ms, since we can just return the timeout directly from the low level APIs and Node.js will invoke the timeout event on the http client handle. > Running "noir:mock" (noir) task > ...................................................... > 1048 passing (10s) And yes, that really reads 10 seconds and yes I always get a bit bored when I have to run the tests for one of our Django backends, feels like an eternity... :)
adrianN 11y ago
If one test breaks you get a nice message. If 20% of your tests break because they depend on functionality that you just changed intentionally, you increased the cost of your feature a lot.
- quaunaut 11y ago
  I don't really see how? If you changed that functionality intentionally, then the other things it interacts with will require a change regardless. Furthermore, if it's that small of a change(as usually, the smallest changes are the ones that impact the most tests), it usually doesn't require any change to the test, aside from adding/subtracting a parameter or something. Major functionality changes should carry weight. It's like saying that a motorcycle runs so much faster without a person on it- it defeats the purpose.
plinkplonk 11y ago
I suspect the author of the article is only giving up on test first or test driven development where you write tests before writing each bit of the program. I doubt he is against having a corpus of tests for your program.
deleted 11y ago
[deleted]
Illniyar 11y ago
To be fair the author was abandoning TDD not tests. He explictly said that he'll add tests after he writes the code.
- organsnyder 11y ago
  Perhaps the author is more disciplined than me, but I'd have an extremely hard time actually following through and writing tests after-the-fact. After the code is "done", it's hard to keep working on it—no matter how much the "extra" work is needed. I find tests to be extremely helpful for designing APIs. I write tests to enforce the contract my API is making with its consumers. Therefore, TDD forces me to consider the API from the user's perspective, rather than designing an API based on implementation details. That also helps to cut down on the cost of refactoring—unless I'm making an API-incompatible change, my tests should keep passing. This gives me more confidence as I'm refactoring, allowing me to make more sweeping changes.
  - Ensorceled 11y ago
    You just need an accountability buddy. I do code reviews for my team and reject any pull request with insufficient test coverage.
  - pkolaczk 11y ago
    This is a great way to create APIs that are hard or nearly impossible to implement properly or efficiently. I can imagine how this could easily lead to horrible API design like a well-known stdlib `gets` function. From the user perspective, gets is a perfectly fine function. Unfortunately there is no way you could write a safe implementation, because in C you have no way of checking arbitrary buffer size. Low level implementation details do affect how we design APIs.
  - Retra 11y ago
    That works well when your API is small and understood. But when you're exporting someone else's results (e.g., returning the result of a floating point operation,) your tests could start to become so exacting that they are more difficult to write and less reliable than the code you're testing itself. In that case, writing tests is basically writing an ad-hoc waterfall-style software specification. I tend to write tests when I'm not confident in the implementation or when the tests are trivial to write. At least, that's how I do it for new developments. If you're releasing code that people are relying on, you should have a whole suite of user tests and regression tests.
sago 11y ago
It's not a good argument against testing, I agree. Automated testing is a huge boon for software quality (though by no means a panacea). But TDD, in my experience, gives a slightly different dynamic. Because you generated the code being motivated by the tests, there are a lot of unit tests that don't test functional units - tests that are essentially testing implementation decisions. I've seen situations where that big refactoring would involve scrapping or rewriting tests, and that causes its own kind of architectural conservatism. I ended up in a compromise position. I use a lot of tests, but I only do TDD in niche cases where it suits the problem I'm solving.
- ncphillips 11y ago
  > Because you generated the code being motivated by the tests, there are a lot of unit tests that dot test functional units - tests that are essentially testing implementation decisions. I've seen this happen to myself numerous times as well, but I don't think this is good argument against TDD. If a tests fails because the implementation details change, that tells us that we wrote a bad test, not that TDD is bad. As I get better at writing tests, I am better able to catch these kinds of errors ahead of time. There are a ton of little techniques you can use and questions you can ask yourself about your code that can make your tests better. Furthermore, test code is just as important as production code, and we should be just as ruthless in refactoring and peer-reviewing each others tests (within reason). TDD is NOT suitable for every problem, this is true. But I think too many people give up on it because they see "the problems of TDD" which are actually just "the problems of writing bad tests". Note: I'm not trying to target you sago, but merely making a general observation. I'm talking more about people who don't write tests even close to the time of writing the code, or don't write any tests at all because the their tests kept breaking for no good reason and were slowing them down.
  - sago 11y ago
    > that tells us that we wrote a bad test, not that TDD is bad I strongly disagree. The only thing that matters to how software works for an end user is its boundary. You simply can't do TDD by only testing the boundary. You have to write a test before each unit of code, so you have to test those internal interfaces, the way units of code work. But that is exactly the stuff that should be allowed to change. It's a mistake to assume 'implementation details' only exist within a function. That stuff you can avoid testing, but the behavior of the function is itself an implementation detail for whatever is using that function. I simply cannot imagine TDD for anything other than a toy example where you don't end up writing tests that depend on the software's implementation, and are therefore inertia to changing that. > we should be just as ruthless in refactoring and peer-reviewing each others tests I strongly agree. But this is rarely part of the TDD ethos. Saying "I refactored the code and threw away 100 tests." is likely to send TDD folks twitching, in my experience.
Ensorceled 11y ago
That's different, I'm against TDD for the same reason the author is. I'm FOR automated testing for exactly the same reasons you are. Think of it this way: pre-testing makes you afraid to change you mind because you throw out your tests, not post-testing makes you afraid to change your mind because you might break something. If you think changing your mind is good the path forward is clear.
josephlord 11y ago
I think the language and in particular the type system matters a lot. So making big changes in the absence of tests in a Swift or even the somewhat more loosely typed Objective C is less scary than in Ruby and Javascript application.
mikehollinger 11y ago
Agreed. I'd argue that if a large number of tests break when a "large" change occurs in an application, those tests are either testing the wrong thing, or written with bad starting assumptions. Yes, there's a transitional period where the responsibility may shift from the caller to the call-ee to take care of some resource, and yes, stuff will break, but the amount of test breakage should be proportional to the impact to the application. I greatly-prefer the warm fuzziness of knowing that I have a large number of unit tests as a "safety net" to detect if my own thoughts on what is "correct" are in some way wrong.
catnaroek 11y ago
> Under these conditions, large scale refactorings or framework upgrades can cause massive test failures An even bigger problem is that large scale refactorings can result in the tests themselves no longer being correct (which isn't the same thing as “not passing”), if you test at too fine a level of granularity. However, when you wrote these tests, you couldn't have possibly foreseen a large scale refactoring 18 months into the future, so how do you tell in advance what the right level of granularity for your tests is?
kazagistar 11y ago
When tests function as a type system as well, this is true. Buy a good type system will guide refactoring and a good module system and IDE and will guaretee isolation .
mamon 11y ago
Ruby and Javascript are dynamic languages, so you need all those tests to make up for missing compiler type checks. With staticaly typed languages like Java or Scala things are much different.
choward 11y ago
> I've often found that the lack of tests leaves me absolutely terrified of making changes to large Ruby and JavaScript applications. I agree. If I'm working on some part of the app that doesn't have any test coverage, I might add some high level tests before I start.
justicezyx 11y ago
I concur your feeling. In my observation and feeling, the statement: " Because you want to ensure that you always pass the majority of tests, you tend to think about this when you change and extend the program. You therefore are more reluctant to make large-scale changes that will lead to the failure of lots of tests. Psychologically, you become conservative to avoid breaking lots of tests." is most likely due to the low quality and low test coverage of the exiting code base. So that the tests are fragile, seemingly hinders changes. If things are started with TDD, I would be really surprised that tests hinders change.
vannevar 11y ago
While I understand that the conservatism the author refers to might be detrimental to the individual working alone, in a team environment it is actually an advantage. The module the team member is working on is likely a dependency for some other team, and any test-breaking refactoring---however beneficial it might seem---needs to be conscientiously approached and coordinated with other people. There may be very good reasons not to pursue refactoring at the moment, reasons that an individual narrowly focused on the module at hand might not be aware of.
iamleppert 11y ago
Don't expect just because there are no test failures that you haven't broken anything. I can't stand developers who have this mindset. Just because there are unit tests, doesn't mean they are covering all the code foremost, or that they are written properly to truly detect when things have changed for the worst. Tests are not a replacement for actually testing and verifying your changes in whatever application gradient you're working in.
hinkley 11y ago
See, I find the reverse to be true. I too don't have the disposition to patiently wade through fixing tons of tons of broken tests over and over again. IF I have to I can do it, but I know better than to unreservedly trust my judgement about how things are progressing. But what this has taught me is to stop trying to push water uphill. If testing is hard because of questionably coupled code, refactor it NOW instead of waiting for things to get bad. The refactoring almost always suggests new features we could add to the code (or makes me backpedal on pronouncements that certain things were 'impossible'), and the number and kind of tests that break is reduced. I came across a quote recently from Bertrand Meyer (the Design By Contract guy), where he suggested that code for making decisions and code for acting on those decisions should be separated. I found myself nodding along to this because it was something I knew intuitively but had never articulated: Decisions without actions don't need mocks, and actions without decisions need at most one or two (often none). Decisions can be tested (possibly in parallel) with unit tests, and actions tested with functional tests. Then all your integration testing is that decisions lead to actions (basically that event dispatch works) and that catastrophic failures are handled gracefully. A proper testing pyramid. Now I want to figure out which of his books or interviews this was in because I want to read the rest of what he had to say on the subject.

by ekidd 11y ago

ivanhoe 11y ago

From what I understand author is not questioning the usefulness of testing, just TDD approach of writing tests first. I personally prefer a sandwich approach, I start with code first, and then write tests for it as soon as I have that little piece of logic finished (usually a method or two). Then I add more code, followed by more test, and so on. Works great for me and my team.

drewhk 11y ago
I nowadays do an approach where I don't write tests first, but I write down the test titles (I use a language where the testing framework gives a nice DSL to write nicely readable test names compared to method names) and I just mark them as pending (which will report as no test failures or passes but ignored cases). I rarely end up with the same test ideas and names which I started from, but since I only start with names which do not hinder refactorings this worked out surprisingly nicely for me.
atom-morgan 11y ago
I agree. If people started viewing coding as an art instead of a strict science I think people would feel more comfortable doing something like this instead of a strict Test First or Test After.
hsitz 11y ago
My impression, too, the author explicitly says he understands value of unit testing and will continue to do it, but he's going to give up on the idea of writing tests before he writes the actual program code. Strange, from what I can tell 90+ percent of comments in this thread treat the author as if he said he was giving up tests entirely.

BonsaiDen 11y ago

One good way to the limit breakage in such cases is to solely perform black box tests on the API level. In case of our Node.js based Backends we don't ever write a single classical unit test, instead we have a custom Framework built on top of Mocha which performs tests on the HTTP layer against all of our endpoints. This works remarkable well in practice and allows for large scale refactorings under the hood with little to no impact on the tests. We can also mock databases, memcached, redis and graylog on their respective http/tcp/udp level. This in turn means no custom build mocks which could break when refactoring. The tests itself also contain no logic, they are pretty much just chained method calls with data that should go in and an expected response that should come out, along with a specification of all external resource our API fetches during the request and their responses etc. Any unexpected outgoing HTTP request from our server will actually result in a test failure. As for scaling this approach, from our experience it works quite well, especially when you have lots of complicated interactions with customer APIs during your requests since the flows are super quick to set up.

raverbashing 11y ago
This is a good idea, hence, expect hate from TDD purists
cousin_it 11y ago
Yeah, I mostly prefer end-to-end tests as well. Though to be fair, they are often slower than unit tests, because you need to start up the whole system. And they are worse at pinpointing problems, though that doesn't seem to be a big deal in practice.
- BonsaiDen 11y ago
  I in part agree with the pinpointing, though in my experience this really boils down to an issue of scope and how much of the data you want to test in each of your tests. E.g. an API returning user data, do you have one test for the whole set of data or one test per field. For our use case we have some pretty "fancy" deep-equals logic for nested structures and allow to specific fields as "to be ignored" in our test expectations. When it comes to speed, the most important part is to cut out everything you don't need, for us this means that our testing framework completely throws away the internals of Node.js HTTP layer. We never create a single TCP socket in our tests, which gives an incredible speed up. As a bonus this also allows to test timeouts and low level HTTP errors in < 1ms, since we can just return the timeout directly from the low level APIs and Node.js will invoke the timeout event on the http client handle. > Running "noir:mock" (noir) task > ...................................................... > 1048 passing (10s) And yes, that really reads 10 seconds and yes I always get a bit bored when I have to run the tests for one of our Django backends, feels like an eternity... :)

adrianN 11y ago

If one test breaks you get a nice message. If 20% of your tests break because they depend on functionality that you just changed intentionally, you increased the cost of your feature a lot.

quaunaut 11y ago
I don't really see how? If you changed that functionality intentionally, then the other things it interacts with will require a change regardless. Furthermore, if it's that small of a change(as usually, the smallest changes are the ones that impact the most tests), it usually doesn't require any change to the test, aside from adding/subtracting a parameter or something. Major functionality changes should carry weight. It's like saying that a motorcycle runs so much faster without a person on it- it defeats the purpose.

plinkplonk 11y ago

I suspect the author of the article is only giving up on test first or test driven development where you write tests before writing each bit of the program. I doubt he is against having a corpus of tests for your program.

deleted 11y ago

[deleted]

Illniyar 11y ago

To be fair the author was abandoning TDD not tests. He explictly said that he'll add tests after he writes the code.

organsnyder 11y ago
Perhaps the author is more disciplined than me, but I'd have an extremely hard time actually following through and writing tests after-the-fact. After the code is "done", it's hard to keep working on it—no matter how much the "extra" work is needed. I find tests to be extremely helpful for designing APIs. I write tests to enforce the contract my API is making with its consumers. Therefore, TDD forces me to consider the API from the user's perspective, rather than designing an API based on implementation details. That also helps to cut down on the cost of refactoring—unless I'm making an API-incompatible change, my tests should keep passing. This gives me more confidence as I'm refactoring, allowing me to make more sweeping changes.
- Ensorceled 11y ago
  You just need an accountability buddy. I do code reviews for my team and reject any pull request with insufficient test coverage.
- pkolaczk 11y ago
  This is a great way to create APIs that are hard or nearly impossible to implement properly or efficiently. I can imagine how this could easily lead to horrible API design like a well-known stdlib `gets` function. From the user perspective, gets is a perfectly fine function. Unfortunately there is no way you could write a safe implementation, because in C you have no way of checking arbitrary buffer size. Low level implementation details do affect how we design APIs.
- Retra 11y ago
  That works well when your API is small and understood. But when you're exporting someone else's results (e.g., returning the result of a floating point operation,) your tests could start to become so exacting that they are more difficult to write and less reliable than the code you're testing itself. In that case, writing tests is basically writing an ad-hoc waterfall-style software specification. I tend to write tests when I'm not confident in the implementation or when the tests are trivial to write. At least, that's how I do it for new developments. If you're releasing code that people are relying on, you should have a whole suite of user tests and regression tests.

sago 11y ago

It's not a good argument against testing, I agree. Automated testing is a huge boon for software quality (though by no means a panacea). But TDD, in my experience, gives a slightly different dynamic. Because you generated the code being motivated by the tests, there are a lot of unit tests that don't test functional units - tests that are essentially testing implementation decisions. I've seen situations where that big refactoring would involve scrapping or rewriting tests, and that causes its own kind of architectural conservatism. I ended up in a compromise position. I use a lot of tests, but I only do TDD in niche cases where it suits the problem I'm solving.

ncphillips 11y ago
> Because you generated the code being motivated by the tests, there are a lot of unit tests that dot test functional units - tests that are essentially testing implementation decisions. I've seen this happen to myself numerous times as well, but I don't think this is good argument against TDD. If a tests fails because the implementation details change, that tells us that we wrote a bad test, not that TDD is bad. As I get better at writing tests, I am better able to catch these kinds of errors ahead of time. There are a ton of little techniques you can use and questions you can ask yourself about your code that can make your tests better. Furthermore, test code is just as important as production code, and we should be just as ruthless in refactoring and peer-reviewing each others tests (within reason). TDD is NOT suitable for every problem, this is true. But I think too many people give up on it because they see "the problems of TDD" which are actually just "the problems of writing bad tests". Note: I'm not trying to target you sago, but merely making a general observation. I'm talking more about people who don't write tests even close to the time of writing the code, or don't write any tests at all because the their tests kept breaking for no good reason and were slowing them down.
- sago 11y ago
  > that tells us that we wrote a bad test, not that TDD is bad I strongly disagree. The only thing that matters to how software works for an end user is its boundary. You simply can't do TDD by only testing the boundary. You have to write a test before each unit of code, so you have to test those internal interfaces, the way units of code work. But that is exactly the stuff that should be allowed to change. It's a mistake to assume 'implementation details' only exist within a function. That stuff you can avoid testing, but the behavior of the function is itself an implementation detail for whatever is using that function. I simply cannot imagine TDD for anything other than a toy example where you don't end up writing tests that depend on the software's implementation, and are therefore inertia to changing that. > we should be just as ruthless in refactoring and peer-reviewing each others tests I strongly agree. But this is rarely part of the TDD ethos. Saying "I refactored the code and threw away 100 tests." is likely to send TDD folks twitching, in my experience.

Ensorceled 11y ago

That's different, I'm against TDD for the same reason the author is. I'm FOR automated testing for exactly the same reasons you are. Think of it this way: pre-testing makes you afraid to change you mind because you throw out your tests, not post-testing makes you afraid to change your mind because you might break something. If you think changing your mind is good the path forward is clear.

josephlord 11y ago

I think the language and in particular the type system matters a lot. So making big changes in the absence of tests in a Swift or even the somewhat more loosely typed Objective C is less scary than in Ruby and Javascript application.

mikehollinger 11y ago

Agreed. I'd argue that if a large number of tests break when a "large" change occurs in an application, those tests are either testing the wrong thing, or written with bad starting assumptions. Yes, there's a transitional period where the responsibility may shift from the caller to the call-ee to take care of some resource, and yes, stuff will break, but the amount of test breakage should be proportional to the impact to the application. I greatly-prefer the warm fuzziness of knowing that I have a large number of unit tests as a "safety net" to detect if my own thoughts on what is "correct" are in some way wrong.

catnaroek 11y ago

> Under these conditions, large scale refactorings or framework upgrades can cause massive test failures An even bigger problem is that large scale refactorings can result in the tests themselves no longer being correct (which isn't the same thing as “not passing”), if you test at too fine a level of granularity. However, when you wrote these tests, you couldn't have possibly foreseen a large scale refactoring 18 months into the future, so how do you tell in advance what the right level of granularity for your tests is?

kazagistar 11y ago

When tests function as a type system as well, this is true. Buy a good type system will guide refactoring and a good module system and IDE and will guaretee isolation .

mamon 11y ago

Ruby and Javascript are dynamic languages, so you need all those tests to make up for missing compiler type checks. With staticaly typed languages like Java or Scala things are much different.

choward 11y ago

> I've often found that the lack of tests leaves me absolutely terrified of making changes to large Ruby and JavaScript applications. I agree. If I'm working on some part of the app that doesn't have any test coverage, I might add some high level tests before I start.

justicezyx 11y ago

I concur your feeling. In my observation and feeling, the statement: " Because you want to ensure that you always pass the majority of tests, you tend to think about this when you change and extend the program. You therefore are more reluctant to make large-scale changes that will lead to the failure of lots of tests. Psychologically, you become conservative to avoid breaking lots of tests." is most likely due to the low quality and low test coverage of the exiting code base. So that the tests are fragile, seemingly hinders changes. If things are started with TDD, I would be really surprised that tests hinders change.

vannevar 11y ago

While I understand that the conservatism the author refers to might be detrimental to the individual working alone, in a team environment it is actually an advantage. The module the team member is working on is likely a dependency for some other team, and any test-breaking refactoring---however beneficial it might seem---needs to be conscientiously approached and coordinated with other people. There may be very good reasons not to pursue refactoring at the moment, reasons that an individual narrowly focused on the module at hand might not be aware of.

iamleppert 11y ago

Don't expect just because there are no test failures that you haven't broken anything. I can't stand developers who have this mindset. Just because there are unit tests, doesn't mean they are covering all the code foremost, or that they are written properly to truly detect when things have changed for the worst. Tests are not a replacement for actually testing and verifying your changes in whatever application gradient you're working in.

hinkley 11y ago

See, I find the reverse to be true. I too don't have the disposition to patiently wade through fixing tons of tons of broken tests over and over again. IF I have to I can do it, but I know better than to unreservedly trust my judgement about how things are progressing. But what this has taught me is to stop trying to push water uphill. If testing is hard because of questionably coupled code, refactor it NOW instead of waiting for things to get bad. The refactoring almost always suggests new features we could add to the code (or makes me backpedal on pronouncements that certain things were 'impossible'), and the number and kind of tests that break is reduced. I came across a quote recently from Bertrand Meyer (the Design By Contract guy), where he suggested that code for making decisions and code for acting on those decisions should be separated. I found myself nodding along to this because it was something I knew intuitively but had never articulated: Decisions without actions don't need mocks, and actions without decisions need at most one or two (often none). Decisions can be tested (possibly in parallel) with unit tests, and actions tested with functional tests. Then all your integration testing is that decisions lead to actions (basically that event dispatch works) and that catastrophic failures are handled gracefully. A proper testing pyramid. Now I want to figure out which of his books or interviews this was in because I want to read the rest of what he had to say on the subject.