4 ms·

I agree with this article. Good stuff. One minor gripe: > if you care about the code, you need to have unit tests for it. That's not really true. You can test

I agree with this article. Good stuff. One minor gripe: > if you care about the code, you need to have unit tests for it. That's not really true. You can test at a differently granularity and still have the same confidence in your code. Scores of brittle unit tests crowding your productivity is not what you do to code you care about. I wish people would just stop propagating this detrimental rumor that is backed by zero evidence. I prefer to keep my unit test suite lite and sweet (try to test what is very sensitive at the unit level - easy to miss or algorithmic things) and then try to get the most out of my acceptance test suite. Sorry, but I just worked on too many systems where the unit tests suite breaks no matter what you touch and you spend more time updating the tests than actually getting shit done.

a_wild_dandan 7y ago
"Write tests. Not too many. Mostly integration."
vp8989 7y ago
+1, though this is an unpopular opinion. IMO appreciation for automated test ROI is a great litmus test for a "good programmer".
leeoniya 7y ago
agree. Acceptance Tests hit the sweet spot for me in terms of productivity and quickly catching causes of regressions (coupled with a well-groomed, atomic commit log).
eranation 7y ago
If you write a library, or expose a public interface, I do expect a lot of unit tests. Unit tests can run offline, fast, during build time. I don’t mind integ-unit tests that perhaps simulate a database in memory to avoid mocking the entire storage layer. I rather have tests that are technically integration tests as they include more than, well, a unit, as long as they are idempotent, test mainly one thing, and can run fast and offline. Not sure if there is a term for it but I call these integunit tests... (probably the worst name possible) But unit / unit++ tests are not just for you, they are for me, they give me confidence I can touch your code without inadvertently breaking something critical that you may have missed in your acceptance tests. It gives you peace of mind to be able to refactor. An acceptance test may tell you they something broke, but a good unit test will show you exactly where.
- externalreality 7y ago
  > breaking something critical that you may have missed in your acceptance tests. So you can miss things in your acceptance tests but can't miss them in your unit tests? Why do you feel more confident with unit tests than with acceptance tests? Maybe because unit tests break more often so they are providing you with a false sense of security? Do you know that when a test fails for a reason other than the specified assertion its a called a false negative and the failure is meaningless? Do you know if your tests fail and no functional or non-functional requirements of the program has changed the failure is also meaningless (you are testing implementation)? Ask yourself how often does your group give you time to do these hypothetical large refactoring that give merit to large unit test suites? How many times, even, do you do small refactorings and find all the test have to go away or significantly change anyway because they made some implementation assumption that is no longer true? How many times have you gone into a test suite and have seen that its been patched up so many times that you can swear its testing something but you just don't know what? I know that Uncle Bob tells us about test for refactoring and this and that, but I just don't buy it any more. The emperor is naked god dammit. Uncle Bob and the like are very famous because people swallow their anecdote without due scrutiny.
  - discreteevent 7y ago
    One thing I can't find much evidence of is Uncle Bob's experience in delivering commercial software. I get the feeling he works mainly on toy examples which might explain some of the things he advocates that, like you, don't match my experience at all.
- int_19h 7y ago
  If you write a library or expose a public interface, I expect functional tests that cover that interface. Unit testing would require going below that level, and test implementation details - which is exactly why those tests tend to be so brittle, and the overhead of maintaining them is so high.
Spartan-S63 7y ago
Biggest benefit of having a compiler in your toolchain is that it often serves as the majority of your integration testing layer. That said, you should proliferate unit tests for algorithmic logic. That can be a small subset of your code, but a large part of your domain (depending on richness). Integration testing can be written manually, or you can pick a compiled language that does a lot of the lifting by type checking and producing the binary. Acceptance tests guide the overall happy path and can protect against weird regressions.
ufmace 7y ago
I agree. I think it's important that your test suite gives you something useful with every test, instead of writing a bunch of pointless extra tests just to get to 100% coverage because you read somewhere that that's good. I feel like I ought to write more on this one of these days, but I have a few testing pet peeves, as far as tests you shouldn't write. Don't write tests that are a copy-paste of the code you're testing. Don't write tests to validate things that should be proven by your type system and lack of compile/parse errors. Beware of tests so tied to implementation details that they make your code harder to refactor. Don't write tests for things talking to external services - the only real test is that it correctly handles the actual service responses.
- ChristianGeek 7y ago
  Tests should be testing behavior, not implementation (this was the original intent of unit testing but it got sidetracked over time).
stemmlerjs 7y ago
Thanks for reading the article. Perhaps I should have been more specific about what needs testing. In The Clean Coder by Uncle Bob, he says that your unit tests should approach the asymptote of 100% test coverage. Here's where I agree with his statement: On the backend, I agree that all of the domain-layer code should be tested. This is a hard requirement for me. It's also code that has 0 dependencies so it should be easy to test. TDD pairs really nicely with DDD. Unit tests give you the needed confidence in order to do refactoring. You can't refactor code without tests. If you do, there's a risk involved. Therefore, in order to safely improve the design of existing code, it needs to have tests. This "detrimental rumor backed by zero evidence" is one of the fundamental takeaways from Martin Fowler's book on "Refactoring" in addition to Uncle Bob's chapter in The Clean Coder on Unit Testing. Here's where I disagree with his statement on 100% test coverage. I used to spend a lot of time writing brittle tests by testing front-end UI code. I used tools like Selenium and Cypress. Because the front-end is the most susceptible part of a system to change, I found myself spending an equivalent amount of time maintaining these tests in addition to adding new code. This is a hard place to find a balance. In Angular, I merely ensure that I write tests for services. In React, I spend a lot less time writing Enzyme tests on rendering and a lot more time testing the redux operators. Uncle Bob clears this up in his book by saying USUALLY, it's not necessary to write UI tests. I'd say by UI tests, he means "rendering tests". His solution is to ensure that you have a way to run acceptance tests that work through the API, as it should be a lot less susceptible to constant regressions. This way, you're essentially testing the features that the API is executing. If you're using DTOs, the inputs and outputs of your system should remain relatively the same anyways, and you should spend less time changing old tests, and more time adding new ones.
- externalreality 7y ago
  > You can't refactor code without tests See this is where the conversation goes south. You will use the phrase "unit tests" when it suits you, and then, like a magician using indirection, generalize to the word "tests" when it suits you. Why are you listening to Uncle Bob so much? I mean you can do as you like. However, the strong "Uncle Bob" style assertions should probably be left out of most articles in favor of a more humble "this is what worked for me/us" approach. Universal rules for any discipline are very hard to come by. Sharing what worked for you on your projects is great but it becomes somewhat obnoxious when you try to generalize it to universal rules that works in all environments for everyone. Also, I think that large unit test suites, where most of the tests are redundant, come about in competitive environments where if you try to make a commit without a test some other competitive coder will try to use it as a "I know better" stepping stone and call you out on the change - "Where is the test". After 3 years of this behavior what do you think your going to have. A nice clean suite of tests or a monstrous big ball of stubs, mocks, faked, copy pasta that resulted from defensive social coding. Honestly, write a light suite of tests that get to the point, and keep your ability to code swiftly and make changes quickly. As the code matures you'll find the trouble spots and focus testing on those areas. Don't blindly follow methodologies that are going to have you writing large test suite for version pre-alpha 0.0, and now your updating a large test suite for every micro-change just to make it to beta 1. Here is an old article by DHH where he rails against unit testing dogma. Ironically it was ideas from the Ruby community that formed a lot of the basis of the mad dog McCarthyan style unit test dogma. https://dhh.dk/2014/tdd-is-dead-long-live-testing.html https://dhh.dk/2014/tdd-is-dead-long-live-testing.html

a_wild_dandan 7y ago

"Write tests. Not too many. Mostly integration."

vp8989 7y ago

+1, though this is an unpopular opinion. IMO appreciation for automated test ROI is a great litmus test for a "good programmer".

leeoniya 7y ago

agree. Acceptance Tests hit the sweet spot for me in terms of productivity and quickly catching causes of regressions (coupled with a well-groomed, atomic commit log).

eranation 7y ago

If you write a library, or expose a public interface, I do expect a lot of unit tests. Unit tests can run offline, fast, during build time. I don’t mind integ-unit tests that perhaps simulate a database in memory to avoid mocking the entire storage layer. I rather have tests that are technically integration tests as they include more than, well, a unit, as long as they are idempotent, test mainly one thing, and can run fast and offline. Not sure if there is a term for it but I call these integunit tests... (probably the worst name possible) But unit / unit++ tests are not just for you, they are for me, they give me confidence I can touch your code without inadvertently breaking something critical that you may have missed in your acceptance tests. It gives you peace of mind to be able to refactor. An acceptance test may tell you they something broke, but a good unit test will show you exactly where.

externalreality 7y ago
> breaking something critical that you may have missed in your acceptance tests. So you can miss things in your acceptance tests but can't miss them in your unit tests? Why do you feel more confident with unit tests than with acceptance tests? Maybe because unit tests break more often so they are providing you with a false sense of security? Do you know that when a test fails for a reason other than the specified assertion its a called a false negative and the failure is meaningless? Do you know if your tests fail and no functional or non-functional requirements of the program has changed the failure is also meaningless (you are testing implementation)? Ask yourself how often does your group give you time to do these hypothetical large refactoring that give merit to large unit test suites? How many times, even, do you do small refactorings and find all the test have to go away or significantly change anyway because they made some implementation assumption that is no longer true? How many times have you gone into a test suite and have seen that its been patched up so many times that you can swear its testing something but you just don't know what? I know that Uncle Bob tells us about test for refactoring and this and that, but I just don't buy it any more. The emperor is naked god dammit. Uncle Bob and the like are very famous because people swallow their anecdote without due scrutiny.
- discreteevent 7y ago
  One thing I can't find much evidence of is Uncle Bob's experience in delivering commercial software. I get the feeling he works mainly on toy examples which might explain some of the things he advocates that, like you, don't match my experience at all.
int_19h 7y ago
If you write a library or expose a public interface, I expect functional tests that cover that interface. Unit testing would require going below that level, and test implementation details - which is exactly why those tests tend to be so brittle, and the overhead of maintaining them is so high.

Spartan-S63 7y ago

Biggest benefit of having a compiler in your toolchain is that it often serves as the majority of your integration testing layer. That said, you should proliferate unit tests for algorithmic logic. That can be a small subset of your code, but a large part of your domain (depending on richness). Integration testing can be written manually, or you can pick a compiled language that does a lot of the lifting by type checking and producing the binary. Acceptance tests guide the overall happy path and can protect against weird regressions.

ufmace 7y ago

I agree. I think it's important that your test suite gives you something useful with every test, instead of writing a bunch of pointless extra tests just to get to 100% coverage because you read somewhere that that's good. I feel like I ought to write more on this one of these days, but I have a few testing pet peeves, as far as tests you shouldn't write. Don't write tests that are a copy-paste of the code you're testing. Don't write tests to validate things that should be proven by your type system and lack of compile/parse errors. Beware of tests so tied to implementation details that they make your code harder to refactor. Don't write tests for things talking to external services - the only real test is that it correctly handles the actual service responses.

ChristianGeek 7y ago
Tests should be testing behavior, not implementation (this was the original intent of unit testing but it got sidetracked over time).

stemmlerjs 7y ago

Thanks for reading the article. Perhaps I should have been more specific about what needs testing. In The Clean Coder by Uncle Bob, he says that your unit tests should approach the asymptote of 100% test coverage. Here's where I agree with his statement: On the backend, I agree that all of the domain-layer code should be tested. This is a hard requirement for me. It's also code that has 0 dependencies so it should be easy to test. TDD pairs really nicely with DDD. Unit tests give you the needed confidence in order to do refactoring. You can't refactor code without tests. If you do, there's a risk involved. Therefore, in order to safely improve the design of existing code, it needs to have tests. This "detrimental rumor backed by zero evidence" is one of the fundamental takeaways from Martin Fowler's book on "Refactoring" in addition to Uncle Bob's chapter in The Clean Coder on Unit Testing. Here's where I disagree with his statement on 100% test coverage. I used to spend a lot of time writing brittle tests by testing front-end UI code. I used tools like Selenium and Cypress. Because the front-end is the most susceptible part of a system to change, I found myself spending an equivalent amount of time maintaining these tests in addition to adding new code. This is a hard place to find a balance. In Angular, I merely ensure that I write tests for services. In React, I spend a lot less time writing Enzyme tests on rendering and a lot more time testing the redux operators. Uncle Bob clears this up in his book by saying USUALLY, it's not necessary to write UI tests. I'd say by UI tests, he means "rendering tests". His solution is to ensure that you have a way to run acceptance tests that work through the API, as it should be a lot less susceptible to constant regressions. This way, you're essentially testing the features that the API is executing. If you're using DTOs, the inputs and outputs of your system should remain relatively the same anyways, and you should spend less time changing old tests, and more time adding new ones.

externalreality 7y ago
> You can't refactor code without tests See this is where the conversation goes south. You will use the phrase "unit tests" when it suits you, and then, like a magician using indirection, generalize to the word "tests" when it suits you. Why are you listening to Uncle Bob so much? I mean you can do as you like. However, the strong "Uncle Bob" style assertions should probably be left out of most articles in favor of a more humble "this is what worked for me/us" approach. Universal rules for any discipline are very hard to come by. Sharing what worked for you on your projects is great but it becomes somewhat obnoxious when you try to generalize it to universal rules that works in all environments for everyone. Also, I think that large unit test suites, where most of the tests are redundant, come about in competitive environments where if you try to make a commit without a test some other competitive coder will try to use it as a "I know better" stepping stone and call you out on the change - "Where is the test". After 3 years of this behavior what do you think your going to have. A nice clean suite of tests or a monstrous big ball of stubs, mocks, faked, copy pasta that resulted from defensive social coding. Honestly, write a light suite of tests that get to the point, and keep your ability to code swiftly and make changes quickly. As the code matures you'll find the trouble spots and focus testing on those areas. Don't blindly follow methodologies that are going to have you writing large test suite for version pre-alpha 0.0, and now your updating a large test suite for every micro-change just to make it to beta 1. Here is an old article by DHH where he rails against unit testing dogma. Ironically it was ideas from the Ruby community that formed a lot of the basis of the mad dog McCarthyan style unit test dogma. https://dhh.dk/2014/tdd-is-dead-long-live-testing.html https://dhh.dk/2014/tdd-is-dead-long-live-testing.html