4 ms·

By the way, could someone provide any real statistics about unit tests? I've already read quadrupillion times that they're great because they lets you save tim

by maurycy 18y ago

By the way, could someone provide any real statistics about unit tests? I've already read quadrupillion times that they're great because they lets you save time, provide a testing history, simplify maintaince, and take care about your daughter. Let's say that I'm convinced. But still, I would like to see some numbers, some carefully performed study that proves that such statements are right. The problem is that they're very confirmation bias prone. People see a situation when the unit tests found a bug, but they don't see thousands of situaton when a bug wasn't found ("hey, it does not make sure your code is correct"), or pointless hours spent on rewriting the tests because the code changed significantly. Though, could be I'm prone to antybuzzwords and antyhype bias.

nostrademons 18y ago
"when a bug wasn't found ... or pointless hours spent on rewriting the tests" Both of those are very real problems with unit tests. You should expect that thoroughly unit-testing your code roughly doubles the amount of time you need to spend on it. The real benefit of unit testing isn't in catching bugs, though: it's that it lets you refactor without worrying about breaking things. Without a complete set of automated tests, most companies simply institute a "Don't touch that code; something might break" rule. Then their codebase becomes a sprawling mess, nobody touches it because they don't know what'll happen, and nothing ever gets done. Virtually all development "best practices" do not speed you up: most slow you down by a significant amount. The point of them is so that you can make any forward progress at all. There's sort of a brick wall that happens when a piece of software gets bigger than one head can contain, and unless you adopt techniques to mitigate that, you'll find that you can't do anything other than rewrite the code from scratch.
- jamongkad 18y ago
  Once again I find myself agreeing with you on a lot of aspects. I initially thought Unit Testing was a means to speed of software development time. But in reality it actually slowed me down a whole lot. But looking down the road 6 months to a year from now. As our product grows in design and new features. I can see the wisdom in how a little pain now can save me alot of headaches in the future.
- maurycy 18y ago
  If your application is so large, it would be good to ask first if it really needs to be so. We favor small methods, which do only one thing. We favor two pizzas teams, which are small. Why large applications are not considered harmful? You might want to move some of your functionality into different modules, libraries or separate applications. The false premise lying behind the 100% coverage philosophy is that instead of avoiding bad practices and being careful with your every step, trying to make things as obvious as possible, tests let you not to think about it. You can stop worrying. They work like a committee, freeing you from responsibility. Especially if you're a mediocre programmer looking for constant excuses. Articles about tests are mostly abstract. Kool-Aid's hectoliters why tests are great, yet nearly zero discussion how to get them right. Tests are not created equal. I follow Rails blogosphere, which is a very test oriented community, and I don't remember too many, if any, writings about how to write good tests. For instance, let's raise confirmation bias again. People usually write tests to make sure the things work as they think they should. I've not seen too many agressive tests that tried to check exteme cases, to, literally, falsify the method. If you deal with things more complicated than Rails controllers, it becomes the real issue. I also try to perceive tests in terms of ROI. If you don't plan to make your application big, why bother? If you don't plan to do any non-standard things, why bother? If it took five minutes to come up with an immediate and obvious solution, why bother? If you change your frequently ("writing is rewriting"), why bother right now? I'm not saying that tests are all bad. I just don't get the whole hype, saying "write tests, be happy" without raising all risks involved.
  - nostrademons 18y ago
    "If you don't plan to make your application big, why bother? If you don't plan to do any non-standard things, why bother? If it took five minutes to come up with an immediate and obvious solution, why bother? If you change your frequently ("writing is rewriting"), why bother right now?" Don't bother. When one of my programming mentors was explaining unit tests to me, she said "They don't have to be a huge deal. Just enough to let you have confidence in your code." That's the real issue - having confidence that your code does what you think it does and will continue doing so. I have a pretty simple litmus test for when I need unit tests: if I ever have to say "Well, it should work..." then it's time to write a test and find out if it really does. But I don't unit test things like Django models or CSS positioning, unless I'm doing something funky or complicated. I do unit test a lot of my JavaScript, since doing anything non-trivial is complicated.

by maurycy 18y ago

nostrademons 18y ago

"when a bug wasn't found ... or pointless hours spent on rewriting the tests" Both of those are very real problems with unit tests. You should expect that thoroughly unit-testing your code roughly doubles the amount of time you need to spend on it. The real benefit of unit testing isn't in catching bugs, though: it's that it lets you refactor without worrying about breaking things. Without a complete set of automated tests, most companies simply institute a "Don't touch that code; something might break" rule. Then their codebase becomes a sprawling mess, nobody touches it because they don't know what'll happen, and nothing ever gets done. Virtually all development "best practices" do not speed you up: most slow you down by a significant amount. The point of them is so that you can make any forward progress at all. There's sort of a brick wall that happens when a piece of software gets bigger than one head can contain, and unless you adopt techniques to mitigate that, you'll find that you can't do anything other than rewrite the code from scratch.

jamongkad 18y ago
Once again I find myself agreeing with you on a lot of aspects. I initially thought Unit Testing was a means to speed of software development time. But in reality it actually slowed me down a whole lot. But looking down the road 6 months to a year from now. As our product grows in design and new features. I can see the wisdom in how a little pain now can save me alot of headaches in the future.
maurycy 18y ago
If your application is so large, it would be good to ask first if it really needs to be so. We favor small methods, which do only one thing. We favor two pizzas teams, which are small. Why large applications are not considered harmful? You might want to move some of your functionality into different modules, libraries or separate applications. The false premise lying behind the 100% coverage philosophy is that instead of avoiding bad practices and being careful with your every step, trying to make things as obvious as possible, tests let you not to think about it. You can stop worrying. They work like a committee, freeing you from responsibility. Especially if you're a mediocre programmer looking for constant excuses. Articles about tests are mostly abstract. Kool-Aid's hectoliters why tests are great, yet nearly zero discussion how to get them right. Tests are not created equal. I follow Rails blogosphere, which is a very test oriented community, and I don't remember too many, if any, writings about how to write good tests. For instance, let's raise confirmation bias again. People usually write tests to make sure the things work as they think they should. I've not seen too many agressive tests that tried to check exteme cases, to, literally, falsify the method. If you deal with things more complicated than Rails controllers, it becomes the real issue. I also try to perceive tests in terms of ROI. If you don't plan to make your application big, why bother? If you don't plan to do any non-standard things, why bother? If it took five minutes to come up with an immediate and obvious solution, why bother? If you change your frequently ("writing is rewriting"), why bother right now? I'm not saying that tests are all bad. I just don't get the whole hype, saying "write tests, be happy" without raising all risks involved.
- nostrademons 18y ago
  "If you don't plan to make your application big, why bother? If you don't plan to do any non-standard things, why bother? If it took five minutes to come up with an immediate and obvious solution, why bother? If you change your frequently ("writing is rewriting"), why bother right now?" Don't bother. When one of my programming mentors was explaining unit tests to me, she said "They don't have to be a huge deal. Just enough to let you have confidence in your code." That's the real issue - having confidence that your code does what you think it does and will continue doing so. I have a pretty simple litmus test for when I need unit tests: if I ever have to say "Well, it should work..." then it's time to write a test and find out if it really does. But I don't unit test things like Django models or CSS positioning, unless I'm doing something funky or complicated. I do unit test a lot of my JavaScript, since doing anything non-trivial is complicated.