6 ms·

Pledging for unit tests

35 points by koski 16y ago

ericHosick 16y ago
I also can not believe how hard it is to convince people of the advantages of testing. I often see stories that are not accepted multiple times simply because the product owner has changed the intent of the original story. Tests really help to define what it is that the product owner wants before a line of code is even written. In the bigger picture, how much "more time" does it really take? Dumb story, but my dad taught me how to do word working: making European Cabinets. So many times he would say "Measure twice, cut once." This, in some convoluted way, could apply to software engineering.
- tomjen3 16y ago
  Software is free, so if you make a wrong cut, it doesn't matter. Measuring software is expensive, so you are better of not doing it.
  - Lewisham 16y ago
    Software is free, so if you make a wrong cut, it doesn't matter. Untrue; tell that to the person who's paying the QA teams who have to validate your code. Or the patch teams that work 48 hours straight to get a hotfix out because something broke after deployment. Software is anything but free.
- gaius 16y ago
  I also can not believe how hard it is to convince people of the advantages of testing. It's about as hard as convincing people of the advantages of static typing. Not to pick on Ruby particularly, but those guys go crazy for writing their own tests for things a compiler could tell them "for free".
- jasonkester 16y ago
  There's a reason they say that though. Unless you happen to have a board stretcher in the truck, cutting something too short ranges from "expensive" to "irrecoverably fails the entire project." Compare that to writing code. Fixing something you did wrong is so trivial that I'll often find myself writing 3 variants of a bit of code and testing rather than pulling out a sheet of paper and doing a bit of math to make sure I code it correctly. "Should this be cos(theta) or -cos(theta)?"... code, compile, check, whoops, delete, compile, check, done. That's waaay better than "134 inches, I think", cut, fit, whoops, phone Jerry: Hey, could you pick me up a new piece of #4 crown moulding on the way in?
  - ZeroGravitas 16y ago
    I'm intrigued to know whether your "code, compile, check" anecdote was meant to argue for or against TDD as it could go either way. I guess the point you're responding to meant that "eyeballing" whether something actually works is the equivalent of "estimating" a length, a couple of easy tests is the equivalent of "measuring once" and "measuring twice" is taking the extra effort to build a small suite of tests with corner cases, a small sign of having learned from bitter experience. Of course with automated tests you are literally measuring twice (indeed, measuring many times) even as the tools and materials change out from under you.
nadam 16y ago
From a bit philosophical point of view a program is the specification of what the computer should do. Unit tests and type annotations both are some kind of checksums. Generally I am for brevity: the program should be short. So I prefer that the 'checksum' also should be short. Unit testing is a really elegant and good tool if you can test a quite big chunk of functionality with a short test. If the test is for example longer than the tested functionality then its value is more controversial. Ok, for extremely important code, like the control of an airplane, put there as much redundancy in the checksum as you think is needed. But otherwise, it can be questionable. The issue is quite similar with static typing. I am a static typing fan, but dynamic language guys always complain that you have to type too much in static languages. And they are right to some extent, although better static languages have good type inference, so it is not that big of an issue. In fact I don't really understand dynamic language guys. They always state that they like terse code, but with modern statically typed languages like Scala you really don't have to type much more than in dynamic languages. The difference is negligible compared to that those same guys are usually unit testing fans and their unit tests sometimes really make the code base significantly bigger. TL;DR;: I value terseness, and I prefer to use both static typing and unit tests only if it does not increase the code size significantly.
- jefffoster 16y ago
  When I'm working in a static language such as Haskell I very rarely write traditional unit tests, instead preferring to use QuickCheck (http://en.wikipedia.org/wiki/QuickCheck http://en.wikipedia.org/wiki/QuickCheck). I like the adversarial nature - I specify invariants and I challenge the computer to prove me wrong! This seems more likely to find bugs than me writing my own tests. Wanting to find bugs seems to be an important factor in writing quality tests. It's easy to write soft tests when the same person writes the code and does the test. Maybe that's why pair programming and TDD seem to go well together, as the other person is more motivated to try and break your code?
mhd 16y ago
Again with the "professional" slander, as with the TDD article… I hereby declare that anyone who doesn't follow Jackson Structured Programming is a two-bit script kiddy!
tomjen3 16y ago
Unlike, it seems, everybody else I find unit tests to have very little use, but hey if you want them do so. TDD on the other hand is both an unprofessional waste of time (write code that you know isn't going to work just to make a test that you know isn't comprehensive enough pass? Really?), and an unprofessional way to structure your design - instead of a bunch of well behaved classes that abstracts and neatly encapsulate their behavior, you have to insert hooks so that you can inject your mocks. At the end of the day, you end up testing software and find some, but far from all, of your bugs. If you are concerned about software quality, great - write better code, do root cause analysis, but most of all: use a language with a good static type system.
- gurraman 16y ago
  If you see no added value in unit tests you're already doing something to ensure that your programs run as expected when you refactor, try on different platforms, try different input data etc. It would be interesting to hear more about how you test your software. Either we can learn from you, or you can learn from us (we/us being people who use formalized tests). Cheers!
- DeusExMachina 16y ago
  I really agree with this. The main guideline I keep in mind when I write code is the old adage from Abelson & Sussman: "Programs must be written for people to read, and only incidentally for machines to execute". Readability is the main reason I get code that is very maintainable and with very few bugs. For those that still get into the code (they always do), it's very easy to find them thanks to the good structure and readability of the code. Unit testable code becomes ugly and unreadable very fast. The main goal becomes to make it testable instead of making it readable. It pained me every time I had to do this in the past, and I did it because some company wanted it no matter what. I don't do this in my personal projects and they are almost bug free anyway (no unit testing does not mean no testing at all). In TDD you spend time making your code ugly and you probably introduce bugs in doing this, and then you spend other time to find and fix those bugs. And when you need to go through that code again for fixing or refactoring it's harder to understand because of all those unit testing hooks. For very big systems or for applications with a certain structure it probably makes sense to have unit tests. In functional languages they are a lot better because you usually don't have to change the code to adapt to it. But for a lot of applications they make no sense at all. So please, stop selling them like a panacea for everything. Even better, please stop selling anything like it is a magic medicine for every illness.
henrikschroder 16y ago
"It will make our work so much more efficient" The common complaints he lists at the start and acknowledge actually all boil down to the fact that unit tests do not always make work more efficient. It's a tool in the toolbox. Sometimes they are great for your project. Sometimes they are a waste of time. It's still not magic pixie dust. In my current project we have a test suite for our public webservice APIs and various clients that interface against those. Those tests are invaluable. For other parts of the project unittests don't make sense, so we don't have them, since making them would be a colossal waste of time. So yes, you still need to defend unit tests, just like you need to defend every other choice you make during development. No specific tech gets carte blanche.
moconnor 16y ago
More time spent writing unit tests does not imply the product will be better for it. http://news.ycombinator.com/item?id=1846841 http://news.ycombinator.com/item?id=1846841
RBerenguel 16y ago
Completely unrelated, but just this made my day. After upvoting: Pledging for unit tests (madebymonsieur.com) NaN points by koski 1 hour ago | 11 comments
brown9-2 16y ago
There's a bit of a solution to this out there: if you are a developer who really cares about unit tests, make sure to ask any prospective employers how they feel about them during the interview process.
bergie 16y ago
Importance of unit tests varies with the projected lifetime of the software you're writing. If you're writing libraries or system services, then unit testing is smart, as those are bound to live on for many years. Simple end-user applications might be a different case. On the systems side (in this case, a PHP framework), I've found that once test coverage passes 70% you can make optimizations and refactor the code without having to actually run it - the tests will tell whether the changes you did still maintain API stability.
ollysb 16y ago
As both a TDD and HN fan I'm really quite surprised to hear the anti TDD rhetoric here. To the detractors here are some honest questions: 1) How many projects have you worked on that were 100% test driven and written outside-in? 2) Do you have the knowledge to test any code, regardless of web APIs, filesystem etc.? 3) When you deploy an app without 100% coverage how can you be confident that there are no regressions? 4) When you add new functionality are you comfortable changing the existing code or do you try to leave it alone and build on top of it?
- britman 16y ago
  I take a pragmatic approach to Unit tests so find some use but don't feel the 100% gold standard of coverage is realistic or even sensible to aspire too but in response to Q3) never - but then I'd return the question When you deploy an app WITH 100% coverage how can you be confident that there are no regressions? I'm pretty sure the answer is never too as 100% test coverage doesn't prevent 100% of bugs.
- moconnor 16y ago
  I like TDD, but it has its place. I think you'll find the rhetoric here is a reaction to the inflammatory tone in the article more than anything else. It's perfectly possible to develop and maintain large, complex systems without 100% code coverage. Clearly some level of testing coverage between 0% and 100% is optimal. The case for it being 100% is, to my mind, extremely weak.
  - tuomas-mbm 16y ago
    A bit out of topic, but what is it that makes the article to have "inflammatory tone"? Would love to hear some feedback about this too to improve my writing skills.
    - ollysb 16y ago
      It's effectively said that anyone that doesn't practice TDD isn't professional. Telling people they're unprofessional tends to piss them off :) edit: In response to child post - That was my interpretation of this "It will make our work so much more efficient, motivating and a bit more professional."
tuomas-mbm 16y ago
Hi all, I'm the author of the blog post. I'm very glad that it has gained already so much opinions and feedback. As user ollysb, I'm quite surprised to notice that there seems to be more against the post than for it. And that was one of the reasons why I actually wrote it: You are smart guys, please proof me wrong about unit tests. I can only find good things in unit testing, so it seems that I have not fully thought it through. ps. Did the post have "inflammatory tone"? It was definitely not my intention. Sorry if it did. I have done computer programming for living only just about 10 years and I am definitely not a professional in it yet. Which parts made it sound cocky? I would love to hear feedback for that too improve my writing.
- henrikschroder 16y ago
  "I can only find good things in unit testing" Then you are simply not very experienced using it. You're in the love-phase where everything about some new technology seems wonderful, and since it's solved so many previous problems for you, you are blind to the warts and problems that come with it. We understand your enthusiasm for it, unit testing and TDD are good tools, but they're just that. Tools. That you can choose to use if they fit the project. Your blog post (and the previous one) both said that unit tests and TDD are always awesome for all projects. This is not true, and this is what people here are reacting to.
  - tuomas-mbm 16y ago
    Thanks for the nice reply Henrik. Yes, most likely the truth is that I'm not experienced enough of using it and the type of systems I build are limiting my view. I also understand unit testing is a tool with it's limitation. However, for me, unit testing is a good tool that helps me to proof that the code I did works. And I hope I would see more of unit tests on the field. But as you said, probably the way to find the dark side of unit testing is just doing it and finding the problem. So I'll keep using the tool until I do. I think I also really need to learn to write better. :) Where did I do such a claim that the TDD and unit tests are always awesome for all projects? I think I even did the opposite.
stevenwei 16y ago
I wonder if the split between pro/anti unit testing comes from folks in traditional developer positions vs startups (since there is a decent mix of both here). As a pure developer working on an established codebase, the benefits of automated testing are quite clear, especially if you have paying customers and the cost of failure is high. But as a startup, every extra line of test code you write slows you down from launching, and the most important thing at this point is launching and figuring out whether you actually have a market for your product. After all, what's the point of having 100% test coverage for a product that no one wants to use? It's also entirely likely that you're going to end up throwing large sections of your codebase out as you pivot towards something else. Here's the way I see it: It's rather pointless to have 100% test coverage of a product that no one is using, but it's also dangerous to have 0% coverage of a product with live, paying customers. Intelligent developers will have the find the right balance between the two (as it shifts over time).
- eridev 16y ago
  While it is true that a start up needs to get a product or prototype up and running as quickly as possible once they have something working, it would be devastating for the program to stop working soon afterwards so even thought the startup company will not be spending time testing until they finish a prototype it would be in their best interest to generate some automatic unit tests as soon as the prototype is working so that the current state can be captured and future regressions can be detected.Check this out: http://www.parasoft.com/jsp/technologies/unit_testing.jsp?itemId=326 http://www.parasoft.com/jsp/technologies/unit_testing.jsp?it...
- regularfry 16y ago
  If you're ever intending to pivot, chances are that there's going to be a refactoring involved, just as much as you're likely to throw things away. I have difficulty seeing how that's going to be quick and easy without either a rock-solid type system, good test coverage, awesome IDE support, or all three.
deleted 16y ago
[deleted]

35 points by koski 16y ago

I also can not believe how hard it is to convince people of the advantages of testing. I often see stories that are not accepted multiple times simply because the product owner has changed the intent of the original story. Tests really help to define what it is that the product owner wants before a line of code is even written. In the bigger picture, how much "more time" does it really take? Dumb story, but my dad taught me how to do word working: making European Cabinets. So many times he would say "Measure twice, cut once." This, in some convoluted way, could apply to software engineering.

tomjen3 16y ago
Software is free, so if you make a wrong cut, it doesn't matter. Measuring software is expensive, so you are better of not doing it.
- Lewisham 16y ago
  Software is free, so if you make a wrong cut, it doesn't matter. Untrue; tell that to the person who's paying the QA teams who have to validate your code. Or the patch teams that work 48 hours straight to get a hotfix out because something broke after deployment. Software is anything but free.
gaius 16y ago
I also can not believe how hard it is to convince people of the advantages of testing. It's about as hard as convincing people of the advantages of static typing. Not to pick on Ruby particularly, but those guys go crazy for writing their own tests for things a compiler could tell them "for free".
jasonkester 16y ago
There's a reason they say that though. Unless you happen to have a board stretcher in the truck, cutting something too short ranges from "expensive" to "irrecoverably fails the entire project." Compare that to writing code. Fixing something you did wrong is so trivial that I'll often find myself writing 3 variants of a bit of code and testing rather than pulling out a sheet of paper and doing a bit of math to make sure I code it correctly. "Should this be cos(theta) or -cos(theta)?"... code, compile, check, whoops, delete, compile, check, done. That's waaay better than "134 inches, I think", cut, fit, whoops, phone Jerry: Hey, could you pick me up a new piece of #4 crown moulding on the way in?
- ZeroGravitas 16y ago
  I'm intrigued to know whether your "code, compile, check" anecdote was meant to argue for or against TDD as it could go either way. I guess the point you're responding to meant that "eyeballing" whether something actually works is the equivalent of "estimating" a length, a couple of easy tests is the equivalent of "measuring once" and "measuring twice" is taking the extra effort to build a small suite of tests with corner cases, a small sign of having learned from bitter experience. Of course with automated tests you are literally measuring twice (indeed, measuring many times) even as the tools and materials change out from under you.

nadam 16y ago

From a bit philosophical point of view a program is the specification of what the computer should do. Unit tests and type annotations both are some kind of checksums. Generally I am for brevity: the program should be short. So I prefer that the 'checksum' also should be short. Unit testing is a really elegant and good tool if you can test a quite big chunk of functionality with a short test. If the test is for example longer than the tested functionality then its value is more controversial. Ok, for extremely important code, like the control of an airplane, put there as much redundancy in the checksum as you think is needed. But otherwise, it can be questionable. The issue is quite similar with static typing. I am a static typing fan, but dynamic language guys always complain that you have to type too much in static languages. And they are right to some extent, although better static languages have good type inference, so it is not that big of an issue. In fact I don't really understand dynamic language guys. They always state that they like terse code, but with modern statically typed languages like Scala you really don't have to type much more than in dynamic languages. The difference is negligible compared to that those same guys are usually unit testing fans and their unit tests sometimes really make the code base significantly bigger. TL;DR;: I value terseness, and I prefer to use both static typing and unit tests only if it does not increase the code size significantly.

jefffoster 16y ago
When I'm working in a static language such as Haskell I very rarely write traditional unit tests, instead preferring to use QuickCheck (http://en.wikipedia.org/wiki/QuickCheck http://en.wikipedia.org/wiki/QuickCheck). I like the adversarial nature - I specify invariants and I challenge the computer to prove me wrong! This seems more likely to find bugs than me writing my own tests. Wanting to find bugs seems to be an important factor in writing quality tests. It's easy to write soft tests when the same person writes the code and does the test. Maybe that's why pair programming and TDD seem to go well together, as the other person is more motivated to try and break your code?

mhd 16y ago

Again with the "professional" slander, as with the TDD article… I hereby declare that anyone who doesn't follow Jackson Structured Programming is a two-bit script kiddy!

tomjen3 16y ago

Unlike, it seems, everybody else I find unit tests to have very little use, but hey if you want them do so. TDD on the other hand is both an unprofessional waste of time (write code that you know isn't going to work just to make a test that you know isn't comprehensive enough pass? Really?), and an unprofessional way to structure your design - instead of a bunch of well behaved classes that abstracts and neatly encapsulate their behavior, you have to insert hooks so that you can inject your mocks. At the end of the day, you end up testing software and find some, but far from all, of your bugs. If you are concerned about software quality, great - write better code, do root cause analysis, but most of all: use a language with a good static type system.

gurraman 16y ago
If you see no added value in unit tests you're already doing something to ensure that your programs run as expected when you refactor, try on different platforms, try different input data etc. It would be interesting to hear more about how you test your software. Either we can learn from you, or you can learn from us (we/us being people who use formalized tests). Cheers!
DeusExMachina 16y ago
I really agree with this. The main guideline I keep in mind when I write code is the old adage from Abelson & Sussman: "Programs must be written for people to read, and only incidentally for machines to execute". Readability is the main reason I get code that is very maintainable and with very few bugs. For those that still get into the code (they always do), it's very easy to find them thanks to the good structure and readability of the code. Unit testable code becomes ugly and unreadable very fast. The main goal becomes to make it testable instead of making it readable. It pained me every time I had to do this in the past, and I did it because some company wanted it no matter what. I don't do this in my personal projects and they are almost bug free anyway (no unit testing does not mean no testing at all). In TDD you spend time making your code ugly and you probably introduce bugs in doing this, and then you spend other time to find and fix those bugs. And when you need to go through that code again for fixing or refactoring it's harder to understand because of all those unit testing hooks. For very big systems or for applications with a certain structure it probably makes sense to have unit tests. In functional languages they are a lot better because you usually don't have to change the code to adapt to it. But for a lot of applications they make no sense at all. So please, stop selling them like a panacea for everything. Even better, please stop selling anything like it is a magic medicine for every illness.

henrikschroder 16y ago

"It will make our work so much more efficient" The common complaints he lists at the start and acknowledge actually all boil down to the fact that unit tests do not always make work more efficient. It's a tool in the toolbox. Sometimes they are great for your project. Sometimes they are a waste of time. It's still not magic pixie dust. In my current project we have a test suite for our public webservice APIs and various clients that interface against those. Those tests are invaluable. For other parts of the project unittests don't make sense, so we don't have them, since making them would be a colossal waste of time. So yes, you still need to defend unit tests, just like you need to defend every other choice you make during development. No specific tech gets carte blanche.

moconnor 16y ago

More time spent writing unit tests does not imply the product will be better for it. http://news.ycombinator.com/item?id=1846841 http://news.ycombinator.com/item?id=1846841

RBerenguel 16y ago

Completely unrelated, but just this made my day. After upvoting: Pledging for unit tests (madebymonsieur.com) NaN points by koski 1 hour ago | 11 comments

brown9-2 16y ago

There's a bit of a solution to this out there: if you are a developer who really cares about unit tests, make sure to ask any prospective employers how they feel about them during the interview process.

bergie 16y ago

Importance of unit tests varies with the projected lifetime of the software you're writing. If you're writing libraries or system services, then unit testing is smart, as those are bound to live on for many years. Simple end-user applications might be a different case. On the systems side (in this case, a PHP framework), I've found that once test coverage passes 70% you can make optimizations and refactor the code without having to actually run it - the tests will tell whether the changes you did still maintain API stability.

ollysb 16y ago

As both a TDD and HN fan I'm really quite surprised to hear the anti TDD rhetoric here. To the detractors here are some honest questions: 1) How many projects have you worked on that were 100% test driven and written outside-in? 2) Do you have the knowledge to test any code, regardless of web APIs, filesystem etc.? 3) When you deploy an app without 100% coverage how can you be confident that there are no regressions? 4) When you add new functionality are you comfortable changing the existing code or do you try to leave it alone and build on top of it?

britman 16y ago
I take a pragmatic approach to Unit tests so find some use but don't feel the 100% gold standard of coverage is realistic or even sensible to aspire too but in response to Q3) never - but then I'd return the question When you deploy an app WITH 100% coverage how can you be confident that there are no regressions? I'm pretty sure the answer is never too as 100% test coverage doesn't prevent 100% of bugs.
moconnor 16y ago
I like TDD, but it has its place. I think you'll find the rhetoric here is a reaction to the inflammatory tone in the article more than anything else. It's perfectly possible to develop and maintain large, complex systems without 100% code coverage. Clearly some level of testing coverage between 0% and 100% is optimal. The case for it being 100% is, to my mind, extremely weak.
- tuomas-mbm 16y ago
  A bit out of topic, but what is it that makes the article to have "inflammatory tone"? Would love to hear some feedback about this too to improve my writing skills.
  - ollysb 16y ago
    It's effectively said that anyone that doesn't practice TDD isn't professional. Telling people they're unprofessional tends to piss them off :) edit: In response to child post - That was my interpretation of this "It will make our work so much more efficient, motivating and a bit more professional."

tuomas-mbm 16y ago

Hi all, I'm the author of the blog post. I'm very glad that it has gained already so much opinions and feedback. As user ollysb, I'm quite surprised to notice that there seems to be more against the post than for it. And that was one of the reasons why I actually wrote it: You are smart guys, please proof me wrong about unit tests. I can only find good things in unit testing, so it seems that I have not fully thought it through. ps. Did the post have "inflammatory tone"? It was definitely not my intention. Sorry if it did. I have done computer programming for living only just about 10 years and I am definitely not a professional in it yet. Which parts made it sound cocky? I would love to hear feedback for that too improve my writing.

henrikschroder 16y ago
"I can only find good things in unit testing" Then you are simply not very experienced using it. You're in the love-phase where everything about some new technology seems wonderful, and since it's solved so many previous problems for you, you are blind to the warts and problems that come with it. We understand your enthusiasm for it, unit testing and TDD are good tools, but they're just that. Tools. That you can choose to use if they fit the project. Your blog post (and the previous one) both said that unit tests and TDD are always awesome for all projects. This is not true, and this is what people here are reacting to.
- tuomas-mbm 16y ago
  Thanks for the nice reply Henrik. Yes, most likely the truth is that I'm not experienced enough of using it and the type of systems I build are limiting my view. I also understand unit testing is a tool with it's limitation. However, for me, unit testing is a good tool that helps me to proof that the code I did works. And I hope I would see more of unit tests on the field. But as you said, probably the way to find the dark side of unit testing is just doing it and finding the problem. So I'll keep using the tool until I do. I think I also really need to learn to write better. :) Where did I do such a claim that the TDD and unit tests are always awesome for all projects? I think I even did the opposite.

stevenwei 16y ago

I wonder if the split between pro/anti unit testing comes from folks in traditional developer positions vs startups (since there is a decent mix of both here). As a pure developer working on an established codebase, the benefits of automated testing are quite clear, especially if you have paying customers and the cost of failure is high. But as a startup, every extra line of test code you write slows you down from launching, and the most important thing at this point is launching and figuring out whether you actually have a market for your product. After all, what's the point of having 100% test coverage for a product that no one wants to use? It's also entirely likely that you're going to end up throwing large sections of your codebase out as you pivot towards something else. Here's the way I see it: It's rather pointless to have 100% test coverage of a product that no one is using, but it's also dangerous to have 0% coverage of a product with live, paying customers. Intelligent developers will have the find the right balance between the two (as it shifts over time).

eridev 16y ago
While it is true that a start up needs to get a product or prototype up and running as quickly as possible once they have something working, it would be devastating for the program to stop working soon afterwards so even thought the startup company will not be spending time testing until they finish a prototype it would be in their best interest to generate some automatic unit tests as soon as the prototype is working so that the current state can be captured and future regressions can be detected.Check this out: http://www.parasoft.com/jsp/technologies/unit_testing.jsp?itemId=326 http://www.parasoft.com/jsp/technologies/unit_testing.jsp?it...
regularfry 16y ago
If you're ever intending to pivot, chances are that there's going to be a refactoring involved, just as much as you're likely to throw things away. I have difficulty seeing how that's going to be quick and easy without either a rock-solid type system, good test coverage, awesome IDE support, or all three.

deleted 16y ago

[deleted]