5 ms·

I find it strange that so much time is spent trying to convince people that some aspect of testing (coverage / tdd) is not a panacea and so little around improv

I find it strange that so much time is spent trying to convince people that some aspect of testing (coverage / tdd) is not a panacea and so little around improving how to test. As a student / recent grad I remember thinking that testing was maybe some thing you maybe had to do for super hardcore projects. Now I see it as one of the first things to think about on a serious project and something that is going to take ~30 - 50% of the cost / effort.

vetler 11y ago
From experience, it's amazingly easy to break something unintentionally when you're making changes, and even with a relatively small codebase it quickly becomes difficult to keep everything in your head. I'm not an advocate of TDD though, and I agree that more effort should be put into how to test.
lubonay 11y ago
IMO testing's biggest strength is in allowing you to delegate remembering all the use cases of an API to some code, instead of having to look them up in a checklist, or worse, having to remember them all. Even if the tests are covering mundane, easy functions and execution paths, the benefit of not worrying if you broke something far outweighs the time to write tests.
- Fradow 11y ago
  Isn't "remembering use cases" something that should be done with comments or documentation ?
  - jacques_chester 11y ago
    Comments and documentation become stale because they require deliberate human intervention to maintain their truthfulness. They also rely on the author of the documentation or comments correctly mentally executing the code being marked up. Tests are compared with the API every time they're run, as seen by the computer's model of execution, giving immediate feedback on detected discrepancies. (I believe that Pythonistas have clever docs that can encode snippets of testing into documentation, which splits the difference a bit, but is no substitute for a robust test suite).
lmm 11y ago
Many companies have introduced counterproductive test coverage policies which are harmful to good testing practice. And any improvement has to start with measures - it's hard to make the case for some improved testing practice to someone who thinks that coverage alone is a good guarantee of test quality.
- daleharvey 11y ago
  This somewhat makes my point, it is the type of comment that shows up every time somebody mentions testing but nobody has ever thought that coverage alone is a good guarantee of test quality, it is however very obviously a useful metric.
  - lmm 11y ago
    > nobody has ever thought that coverage alone is a good guarantee of test quality Wrong. I could name three managers who have from personal experience. > it is however very obviously a useful metric. Disagree. I think it's an actively counterproductive metric, and one is better off not measuring it at all.
knughit 11y ago
Try weird thing is the more you test, the larger fraction of your time is needed from testing, because testing avoid the blowups of investigations and bugfixes and workarounds.

vetler 11y ago

From experience, it's amazingly easy to break something unintentionally when you're making changes, and even with a relatively small codebase it quickly becomes difficult to keep everything in your head. I'm not an advocate of TDD though, and I agree that more effort should be put into how to test.

lubonay 11y ago

IMO testing's biggest strength is in allowing you to delegate remembering all the use cases of an API to some code, instead of having to look them up in a checklist, or worse, having to remember them all. Even if the tests are covering mundane, easy functions and execution paths, the benefit of not worrying if you broke something far outweighs the time to write tests.

Fradow 11y ago
Isn't "remembering use cases" something that should be done with comments or documentation ?
- jacques_chester 11y ago
  Comments and documentation become stale because they require deliberate human intervention to maintain their truthfulness. They also rely on the author of the documentation or comments correctly mentally executing the code being marked up. Tests are compared with the API every time they're run, as seen by the computer's model of execution, giving immediate feedback on detected discrepancies. (I believe that Pythonistas have clever docs that can encode snippets of testing into documentation, which splits the difference a bit, but is no substitute for a robust test suite).

lmm 11y ago

Many companies have introduced counterproductive test coverage policies which are harmful to good testing practice. And any improvement has to start with measures - it's hard to make the case for some improved testing practice to someone who thinks that coverage alone is a good guarantee of test quality.

daleharvey 11y ago
This somewhat makes my point, it is the type of comment that shows up every time somebody mentions testing but nobody has ever thought that coverage alone is a good guarantee of test quality, it is however very obviously a useful metric.
- lmm 11y ago
  > nobody has ever thought that coverage alone is a good guarantee of test quality Wrong. I could name three managers who have from personal experience. > it is however very obviously a useful metric. Disagree. I think it's an actively counterproductive metric, and one is better off not measuring it at all.

knughit 11y ago

Try weird thing is the more you test, the larger fraction of your time is needed from testing, because testing avoid the blowups of investigations and bugfixes and workarounds.