4 ms·

I stopped thinking about "unit testing" and just started thinking "when I write some code, I should write some other code that runs the code automatically". Th

by dools 3y ago

I stopped thinking about "unit testing" and just started thinking "when I write some code, I should write some other code that runs the code automatically". There's every likelihood that both my original code and the code I write to test it will have bugs, but it's somewhat unlikely that the bugs will be perfectly complementary. However that will happen too, and so it is not the role of automated testing to eliminate bugs. It is the role of quality control testing to eliminate bugs. The role of automated testing is to reduce the cost of quality control (by reducing regressions and reducing the amount of time required to run manual quality control testing). I have to credit Sebastian Bergmann with the genesis of this attitude. I took a course on unit testing with him somewhere around 2008 and I said "it seems that test code is just more code, how is it any different from writing code in the rest of your project?" and he said "it isn't, tests are just part of your codebase like all your other code". That was a watershed moment for me -- I always thought I was missing something.

eddtries 3y ago
The difference between Quality Control and Quality Assurance is really important but most developers don’t really know the difference, and their only form of quality is QC catching some bugs on the way out rather than QA catching them earlier in the pipeline. I’m a contract test engineer & very rarely see the latter - for example test design alongside feature design. It’s amazing how quickly you can improve a products quality metrics (defect leakage, customer reports, defect costs, etc) by focusing a little on the QA side of development. It’s one of the issues companies have when they think they can just replace QA with developers automated tests - it misses every other part of quality except a subset of QC.
jjice 3y ago
> I stopped thinking about "unit testing" and just started thinking "when I write some code, I should write some other code that runs the code automatically". I like that perspective. Personally, the difference of "unit" vs "integration" testing from an organization/thinking perspective for me is what has to be available when it runs. We currently have .test.ts and .int.test.ts files in our code base. The former gets to run really fast without any dependencies, and the latter takes longer to run, but has full access to DBs and Redis. That separation just helps split it up for CI. Thinking about it, you can probably combine them and just require everything is always available, but I've found that the distinction has helped how I think about writing pure functions and allows me to iterate on them with tests easily.
- dools 3y ago
  Yeah I sometimes write tests that are separate from external dependencies still, I think the freedom I gained through this attitude is that I don’t stress too much about it, especially in the early stages. This gives me “good enough” coverage most of the time without the sorts of extreme bending over backwards to mock out every little dependency that a purist unit testing approach would entail. As a side note I really liked using Lettuce when I was working with Django not so much because of the whole “BDD” thing which was lost on me but because of the way they did database fixtures. I wrote a little automated test runner in PHP that uses the same approach, and later wrote a little test runner in node with the same name (but never implemented database testing in it because I’m not masochistic enough to use asynchronous code to access a database): https://github.com/iaindooley/murphy https://github.com/iaindooley/murphy

by dools 3y ago

eddtries 3y ago

The difference between Quality Control and Quality Assurance is really important but most developers don’t really know the difference, and their only form of quality is QC catching some bugs on the way out rather than QA catching them earlier in the pipeline. I’m a contract test engineer & very rarely see the latter - for example test design alongside feature design. It’s amazing how quickly you can improve a products quality metrics (defect leakage, customer reports, defect costs, etc) by focusing a little on the QA side of development. It’s one of the issues companies have when they think they can just replace QA with developers automated tests - it misses every other part of quality except a subset of QC.

jjice 3y ago

> I stopped thinking about "unit testing" and just started thinking "when I write some code, I should write some other code that runs the code automatically". I like that perspective. Personally, the difference of "unit" vs "integration" testing from an organization/thinking perspective for me is what has to be available when it runs. We currently have .test.ts and .int.test.ts files in our code base. The former gets to run really fast without any dependencies, and the latter takes longer to run, but has full access to DBs and Redis. That separation just helps split it up for CI. Thinking about it, you can probably combine them and just require everything is always available, but I've found that the distinction has helped how I think about writing pure functions and allows me to iterate on them with tests easily.

dools 3y ago
Yeah I sometimes write tests that are separate from external dependencies still, I think the freedom I gained through this attitude is that I don’t stress too much about it, especially in the early stages. This gives me “good enough” coverage most of the time without the sorts of extreme bending over backwards to mock out every little dependency that a purist unit testing approach would entail. As a side note I really liked using Lettuce when I was working with Django not so much because of the whole “BDD” thing which was lost on me but because of the way they did database fixtures. I wrote a little automated test runner in PHP that uses the same approach, and later wrote a little test runner in node with the same name (but never implemented database testing in it because I’m not masochistic enough to use asynchronous code to access a database): https://github.com/iaindooley/murphy https://github.com/iaindooley/murphy