9 ms·
I can't believe that in 2017 we still need to discuss if the benefits of tests outweigh their costs. Anyway, the article seems like a brain-storming session fr
by kkapelon 10y ago
I can't believe that in 2017 we still need to discuss if the benefits of tests outweigh their costs.
Anyway, the article seems like a brain-storming session from the author, so I am not sure that I need to answer all the points. Those that stick out
1)As other people said already you always write a unit test against a spec. If you have no spec, then unit tests are not really helpful. When you prototype something you don't need unit tests. Most startups also would not need a vast test suite in their early stages
2) The company that had only UI tests forgot to make them resistant against ui changes. I don't know the tool they were using but most GUI tools support the Object/pages pattern. If the UI was changing so much that even that was not enough, then they did not have spec (see point 1)
3) A/B testing is not unit testing because by definition you don't have a spec (if you are still trying to find if A or B are ok)
4) Code review is not unit testing because you will catch only obvious stuff.
Code review is more about the structure of the code than its correctness. It far easier to judge correctness by automation than by visual inspection.
Basically I would suggest you to remove the word "unit" from your blog post title.
Unit testing has a very clear definition and you are confusing it with code reviews, A/B tests and other unrelated stuff
google also "the testing pyramid" for a sound model of unit testing. The tax preparation company would do fine to follow this.
- itamarst 10y agoThe blog post isn't about defining unit testing, it's about defining testing. I'll make that more clear when revising, yes. The testing pyramid, IIRC, starts with the presumption that the goal of testing is quality. There are other goals for testing, though, and I say in the intro quality is too hand wavy to be useful in decision making.
- kkapelon 10y agoThe title clearly starts with "unit testing".
- itamarst 10y agoPyCon talk title (at least, what I proposed) is "Big picture software testing: unit testing, Lean Startup, and everything in between." Which seemed cumbersome, so I tried this iteration without the prefix. Will need to come up with something better if it's confusing to readers/listeners, though.
- deleted 10y ago[deleted]
- qwer 10y agoThere are more words after that though.
- dkersten 10y agoIt also ends with "and everything in between". The title gave a spectrum of testing which the article then discusses.
- yomly 10y agoSo you've got a working prototype, which is 80% of the way towards your spec (which changes by the day, mind you) where and when do the tests then get written? Do you start from scratch when you go from prototype to product? Is that feasible in a tiny lean startup with few developers and real hard fast deadlines?
- itamarst 10y agoGoing from prototype to tested code is possible. Here's how I'm doing it, for command line tool that provides local development environment for remote Kubernetes cluster (https://datawire.github.io/telepresence/ https://datawire.github.io/telepresence/): 1. Build prototype, test manually. 2. Have coworkers try it out. 3. Fix UX based on feedback, test manually. 4. Once initial UX was nailed down, write end-to-end tests. Beyond adding features, next I will rewrite the core using test-driven development (TDD), so that the internals are maintainable. and the end-to-end tests will ensure the basic functionality continues to work. The UI is fairly simple, and at this point I have confidence that it won't change multiple times in a day, so it's worth the effort to stop relying solely on manual testing and write some automated UI tests.
- kkapelon 10y agoIf you have a spec that changes by the _day_ = it means you don't have a spec. You are still prototyping. So yes, start writing unit tests when you no longer prototype.