4 ms·

The author rejects unit tests because they don't help find bugs. I don't think his reasoning adds up to much, more like rationalizations for already-held belie

The author rejects unit tests because they don't help find bugs. I don't think his reasoning adds up to much, more like rationalizations for already-held beliefs. But if you want to take the argument seriously, I would say that "finding bugs" is not the primary reason I write unit tests anyway. It's to prevent regressions, and make me feel safe that I haven't broken anything after a refactoring. If not for my unit tests, I'd be scared to change much of anything, for fear of stirring up a whole bunch of trouble that I'll have to fix later on.

jgrahamc 18y ago
I agree. Unit tests are what make refactoring safe. Adding known bugs to the unit test suite is a good way to prevent regressions.
abstractbill 18y ago
"preventing regressions" is a subset of "finding bugs", and I've found the other methods I listed to be more effective (from an ROI perspective) at doing that too.
- allenbrunson 18y ago
  Apparently you work at Justin.TV. I don't know how big your individual projects are, but personally, I didn't get unit-testing religion until I started working on really big stuff, close to the limits of what I'm capable of. I worked for a year on a 60,000 line program, and it started getting painful. I worked for another year on a 100,000 line program, and that's when I finally had to relent and start writing unit tests. Here's a common scenario that made me want to write tests. I want to implement feature A, but to make it work, I'm first going to have to write library B and C. So I write library B, but there is no way to test it in the program yet, because feature A doesn't exist. Right after I've finished the code, I have a pretty good idea of where things might go wrong, and what corner cases I want to exercise. But by the time I finish library C and then get around to implementing feature A, I've forgotten that stuff. Pain points go unaddressed. In summary, unit tests really start to show their value when you're working on a program that's too big to fit into your head all at once.
  - abstractbill 18y ago
    Couldn't you have tried to break that 100,000 line program up into ten 10,000 line programs? You might have found the additional benefit that each of the ten programs could be useful on its own. That's how I would usually attack this kind of problem, rather than throwing unit tests at it.
    - mrtron 18y ago
      I think it would be preferable to do both. Have a few really solid and well contained modules that are well tested. Testing is only a piece.
- chuckm 18y ago
  It seems like you're still using some form of automation to find bugs, regardless of whether these would be classified as unit tests, correct?
  - abstractbill 18y ago
    Sure, some of the monitoring systems I've written do some fairly detailed testing. For example, I wrote the Justin.TV chat system. There's a group of bots that periodically log in to the production chat server, send messages to each other, and email me if anything doesn't work. That's certainly not unit testing though.
    - there 18y ago
      do they ever say anything interesting to eachother?
      - abstractbill 18y ago
        heh, no they're awfully dull - they just say things like "testing 123" and "received 'testing 123' from bot456".
    - jerf 18y ago
      I speak only for myself, but I would say the following: The real key is fully automated testing, no matter how it is done. The finer the grain, the better, generally. Matching an actual business case is near ideal. Fully automated is necessary so you can run them frequently, automatically, near-continuously. But beyond that, I think you get into religious territory, and I think people getting dogmatic about the definition of "unit test" tends to mistake definition for virtue (a common failing). If you've got automated testing, great! You win. Doesn't matter how it works. Or, rather, it does matter, but only within your context, which nobody else is really competent to judge you on.
      - joe_the_user 18y ago
        Yeah, the only thing that matters to me is whether the pro-unit testing position has become more religious or whether the anti-unit testing position has become more religious. I'll take whatever seems serious. I liked what Joel Spolsky had to say yesterday but I disagree with the linked blog post today.
axod 18y ago
>> "If not for my unit tests, I'd be scared to change much of anything, for fear of stirring up a whole bunch of trouble that I'll have to fix later on." For me, if I had unit tests everywhere, I'd be scared to change much of anything, knowing that after the refactoring, and usual testing/checking things work, I'd then have to rewrite/fix unit tests.
- darkxanthos 18y ago
  That's why I stopped for a while. Then I discovered BDD style testing (or context/specification, or...). Whatever it's called, it's not quite as fine grained as a unit test and it's not as large in scope as integration testing. It's at the level of your business specs. So the only time you'll break your tests is if you've jacked up a spec. Honestly, if my changes cause new behavior in the system but the people paying me won't care, then I don't care either.
- jon_dahl 18y ago
  Unit testing can be this way, but it doesn't have to be. At a really low level, unit testing can be "testing the implementation", which may be useful while you're writing code, but isn't useful during a refactoring. But slightly higher level tests can be written to almost never have this problem. Let's say you've written code to ensure that a CC transaction doesn't go through with an invalid number. No matter what the implementation looks like - no matter how much you refactor - this test should always work. The same is true for a wide range of useful tests. (These aren't strictly "unit" tests, but they fall into a similar category.) My personal experience is that a good unit test suite is 99% useful when refactoring, and 1% painful.
swombat 18y ago
Also, TDD/BDD are most useful as design practices - not as bug-finding practices. They help you think more carefully about your design and make it more modular and clear.
- tow21 18y ago
  I agree entirely. Whenever I leave out writing unit tests, I end up with a web of messy interdependencies which ends up far too hard to extend, and will always need refactored as soon as I need to add the smallest feature. Writing unit tests up front forces me to write the code in a way where inputs & outputs can easily be stubbed out, which in turn makes writing messy interdependencies simply too hard! Basically, if I write with unit tests, it becomes easier to write maintainable code, and harder to write unmaintainable code.

jgrahamc 18y ago

I agree. Unit tests are what make refactoring safe. Adding known bugs to the unit test suite is a good way to prevent regressions.

abstractbill 18y ago

"preventing regressions" is a subset of "finding bugs", and I've found the other methods I listed to be more effective (from an ROI perspective) at doing that too.

allenbrunson 18y ago
Apparently you work at Justin.TV. I don't know how big your individual projects are, but personally, I didn't get unit-testing religion until I started working on really big stuff, close to the limits of what I'm capable of. I worked for a year on a 60,000 line program, and it started getting painful. I worked for another year on a 100,000 line program, and that's when I finally had to relent and start writing unit tests. Here's a common scenario that made me want to write tests. I want to implement feature A, but to make it work, I'm first going to have to write library B and C. So I write library B, but there is no way to test it in the program yet, because feature A doesn't exist. Right after I've finished the code, I have a pretty good idea of where things might go wrong, and what corner cases I want to exercise. But by the time I finish library C and then get around to implementing feature A, I've forgotten that stuff. Pain points go unaddressed. In summary, unit tests really start to show their value when you're working on a program that's too big to fit into your head all at once.
- abstractbill 18y ago
  Couldn't you have tried to break that 100,000 line program up into ten 10,000 line programs? You might have found the additional benefit that each of the ten programs could be useful on its own. That's how I would usually attack this kind of problem, rather than throwing unit tests at it.
  - mrtron 18y ago
    I think it would be preferable to do both. Have a few really solid and well contained modules that are well tested. Testing is only a piece.
chuckm 18y ago
It seems like you're still using some form of automation to find bugs, regardless of whether these would be classified as unit tests, correct?
- abstractbill 18y ago
  Sure, some of the monitoring systems I've written do some fairly detailed testing. For example, I wrote the Justin.TV chat system. There's a group of bots that periodically log in to the production chat server, send messages to each other, and email me if anything doesn't work. That's certainly not unit testing though.
  - there 18y ago
    do they ever say anything interesting to eachother?
    - abstractbill 18y ago
      heh, no they're awfully dull - they just say things like "testing 123" and "received 'testing 123' from bot456".
  - jerf 18y ago
    I speak only for myself, but I would say the following: The real key is fully automated testing, no matter how it is done. The finer the grain, the better, generally. Matching an actual business case is near ideal. Fully automated is necessary so you can run them frequently, automatically, near-continuously. But beyond that, I think you get into religious territory, and I think people getting dogmatic about the definition of "unit test" tends to mistake definition for virtue (a common failing). If you've got automated testing, great! You win. Doesn't matter how it works. Or, rather, it does matter, but only within your context, which nobody else is really competent to judge you on.
    - joe_the_user 18y ago
      Yeah, the only thing that matters to me is whether the pro-unit testing position has become more religious or whether the anti-unit testing position has become more religious. I'll take whatever seems serious. I liked what Joel Spolsky had to say yesterday but I disagree with the linked blog post today.

axod 18y ago

>> "If not for my unit tests, I'd be scared to change much of anything, for fear of stirring up a whole bunch of trouble that I'll have to fix later on." For me, if I had unit tests everywhere, I'd be scared to change much of anything, knowing that after the refactoring, and usual testing/checking things work, I'd then have to rewrite/fix unit tests.

darkxanthos 18y ago
That's why I stopped for a while. Then I discovered BDD style testing (or context/specification, or...). Whatever it's called, it's not quite as fine grained as a unit test and it's not as large in scope as integration testing. It's at the level of your business specs. So the only time you'll break your tests is if you've jacked up a spec. Honestly, if my changes cause new behavior in the system but the people paying me won't care, then I don't care either.
jon_dahl 18y ago
Unit testing can be this way, but it doesn't have to be. At a really low level, unit testing can be "testing the implementation", which may be useful while you're writing code, but isn't useful during a refactoring. But slightly higher level tests can be written to almost never have this problem. Let's say you've written code to ensure that a CC transaction doesn't go through with an invalid number. No matter what the implementation looks like - no matter how much you refactor - this test should always work. The same is true for a wide range of useful tests. (These aren't strictly "unit" tests, but they fall into a similar category.) My personal experience is that a good unit test suite is 99% useful when refactoring, and 1% painful.

swombat 18y ago

Also, TDD/BDD are most useful as design practices - not as bug-finding practices. They help you think more carefully about your design and make it more modular and clear.

tow21 18y ago
I agree entirely. Whenever I leave out writing unit tests, I end up with a web of messy interdependencies which ends up far too hard to extend, and will always need refactored as soon as I need to add the smallest feature. Writing unit tests up front forces me to write the code in a way where inputs & outputs can easily be stubbed out, which in turn makes writing messy interdependencies simply too hard! Basically, if I write with unit tests, it becomes easier to write maintainable code, and harder to write unmaintainable code.