5 ms·

I agree, in theory. However, when you're quickly iterating, redesigning, changing how things work, having to completely rework the unit tests as well is pretty

by axod 18y ago

I agree, in theory. However, when you're quickly iterating, redesigning, changing how things work, having to completely rework the unit tests as well is pretty much wasted effort. I think it depends on the project - is it a 'design up front, go away code once' project, or an iterative constantly changing thing.

swombat 18y ago
'Fraid not, axod. It's when you're quickly iterating and changing lots of stuff that quality unit testing helps you the most. Let's look at 2 possible scenarios: 1) You don't bother testing much. You make lots of changes, iterate quickly. Before the changes, you have code that, let's say, you know works. After the changes, you know there's probably bugs, but you don't know where. 2) You test fairly thoroughly. You make lots of changes, iterate quickly. Before the changes you have code that you know works. After the changes, you know where most of the bugs are because your tests tell you where they are. It's the difference between walking with your eyes open or closed... it may be more work to process all that information about obstacles, but it sure is helpful if you want to avoid those obstacles.
- fizx 18y ago
  I'll chime in from recent experience. I had to pull a dependency in my project because it was causing heisenbugs that I couldn't root out. Without tests, I wouldn't have known. With tests, it took 2 hours to replace (diff -200 +250), and I'm confident the library still works. Without tests, it would have taken several more, at least.
  - mattm 18y ago
    How long did it take to write the tests?
    - fizx 18y ago
      I write tests as I develop, and they take the place of going to the browser or a repl and poking around at my work. In some sense, testing takes zero or negative net time. It would be hard to measure gross time, because it occurs in such small increments.
- axod 18y ago
  I disagree. If I'm constantly changing my mind about how best to do something, I don't want the massive overhead of having to change unit tests all the time as well. I'll test as I go, in the code.
  - gdee 18y ago
    "If I'm constantly changing my mind about how best to do something" then you're finding /more/ not /best/ ways.
  - Retric 18y ago
    If your testing in the code then your doing unit tests. But probably in a poor fashion. Unit tests should hook into the application in such a way that you can reorder the GUI and change nothing. If you are using Unit tests correctly then any change to the code base at worst requires 10% of that effort to change the test cases. If you can think of any meaningful change that is worse then that your something is wrong with how you're testing.
- davidw 18y ago
  > Before the changes you have code that you know works. After the changes, you know where most of the bugs are because your tests tell you where they are. The tests tell you where the tests fail. Some of those failures may be due to bugs, and others may simply due to the way your code now works.
- trapper 18y ago
  There are two schools of thought in programming I have seen. 1. Design it right the first time, tests and all. 2. Build a throwaway, no tests, little refactoring as you quickly iterate Programmers who fall into category 1 in my experience mistakenly believe that they understand the requirements of the software they are writing. Category 2 programmers tend to be ones who actually deal with users/real world, and know that they don't understand the requirements. My position is that if you are building something for the first time, you are going to get it wrong, so get it wrong as quickly and cheaply as possible. This means getting it in front of users as quickly as possible, bugs and all.
  - swombat 18y ago
    You're assuming that you'll get something in front of users more quickly without writing tests. I find that's not the case in my experience.
    - nostrademons 18y ago
      I find there's a balance to be struck. You want to get stuff in front of users quickly, but if that stuff sucks so much that it's unusable, you aren't going to release it (and if you did, nobody would use it). So you want enough tests that you can be reasonably confident of the code and not spend a lot of time debugging, since debugging is the real timesink that'll kill you. I often find that I unit test all but the latest layer. As soon as I start depending on code from something else, it's time to write some tests. But if I just wrote something, it's probably in flux and tests would just prematurely lock down the interface. Write the code, write the client (which probably requires that you rewrite the code), and then write the tests in lieu of debugging the code.
    - trapper 18y ago
      I'd love to see some research on this. I would hypothesize that there is a point of problem complexity where the lack of tests becomes a negative for productivity.
      - mrandle 18y ago
- DougBTX 18y ago
  > After the changes, you know there's probably bugs, but you don't know where. Which highlights one of the "problems" with unit tests: if something fails, you have to take time to go and fix it. If you have no tests, you can assume there are no bugs because you are such a great programmer, then go home after a job well done. That there actually are bugs there doesn't bother you, because you don't know about them. It takes less time to write buggy code.
  - axod 18y ago
    The alternative, is to build you code in a way which tells you quickly about bugs... decent logging, etc
    - swombat 18y ago
      I do think he was being sarcastic.
- axod 18y ago
  >> "Before the changes you have code that you know works. After the changes, you know where most of the bugs are because your tests tell you where they are." No. More often than not, if you change was a large architectural one, which is often the case in young software, your tests are now irrelevant, or broken. >> "It's the difference between walking with your eyes open or closed... it may be more work to process all that information about obstacles, but it sure is helpful if you want to avoid those obstacles." Do you seriously think people are just blindly coding and never testing anything here? while(true) { writeABitOfCode(); while (!compile()) fixCompilationErrors(); testThingsYouMightHaveChanged(); checkLogsForIssues(); }
  - DougBTX 18y ago
    testThingsYouMightHaveChanged requires running your tests manually, which is slow and hard to fully reproduce later. It also depends on being able to make strong predictions about what your change affects, unit testing assumes that anything could have broken.
  - swombat 18y ago
    * testThingsYouMightHaveChanged();* If you change that into writeAnAutomatedTestForItAndRunYourTestSuite() you get the following benefits: 1) Every test you write accrues value onto your test suite and helps you avoid problems later; 2) You don't need to spend time to figure out what you might have broken with your change - your test suite tells you in seconds; 3) You can say with some measure of certainty that it's extremely unlikely that your change broke anything.
    - axod 18y ago
      BUT you then have to modify: while (!compile()) { fixCompilationErrors(); fixUnitTests(); rewriteUnitTestsThatNoLongerMakeSense(); WriteNewUnitTests(); } Your iterate loop just became maybe twice or 3 times as long. Less iterations = worse code.
      - Retric 18y ago
shiro 18y ago
> I think it depends on the project - is it a 'design up front, go away code once' project, or an iterative constantly changing thing. I think your line is too coarse. I do heavily rely on unit tests even in "iterative constant changing projects". But in the very early "sketching" stage of development I don't bother to write tests. I just play with code on REPL (I work with Common Lisp or Scheme most of the time), in a flat namespace. Then, once I start putting code snippets together into packages (CL) or modules (Scheme), that's about the time I start writing tests as well. After that, even I constantly change stuff, sometimes drastically, unit tests do help me a lot. It is true that you need effort on reworking on the tests. Actually I think writing tests needs more work than writing actual code. And I think that's the right thing. Writing something that works is an easy part. Making sure it doesn't break in every possible situation is much harder. In language like CL, you can fairly quickly write something that takes typical input and reasonable output without serious thinking. To make sure it works in every corner case you have to think much harder and precisely, and test code reflects that thinking. (NB: I know some programmers who seem to think out those corner cases in their brain and just spit out a beautiful code at the first attempt. They may not need support from test code, I guess.)

by axod 18y ago

swombat 18y ago

'Fraid not, axod. It's when you're quickly iterating and changing lots of stuff that quality unit testing helps you the most. Let's look at 2 possible scenarios: 1) You don't bother testing much. You make lots of changes, iterate quickly. Before the changes, you have code that, let's say, you know works. After the changes, you know there's probably bugs, but you don't know where. 2) You test fairly thoroughly. You make lots of changes, iterate quickly. Before the changes you have code that you know works. After the changes, you know where most of the bugs are because your tests tell you where they are. It's the difference between walking with your eyes open or closed... it may be more work to process all that information about obstacles, but it sure is helpful if you want to avoid those obstacles.

fizx 18y ago
I'll chime in from recent experience. I had to pull a dependency in my project because it was causing heisenbugs that I couldn't root out. Without tests, I wouldn't have known. With tests, it took 2 hours to replace (diff -200 +250), and I'm confident the library still works. Without tests, it would have taken several more, at least.
- mattm 18y ago
  How long did it take to write the tests?
  - fizx 18y ago
    I write tests as I develop, and they take the place of going to the browser or a repl and poking around at my work. In some sense, testing takes zero or negative net time. It would be hard to measure gross time, because it occurs in such small increments.
axod 18y ago
I disagree. If I'm constantly changing my mind about how best to do something, I don't want the massive overhead of having to change unit tests all the time as well. I'll test as I go, in the code.
- gdee 18y ago
  "If I'm constantly changing my mind about how best to do something" then you're finding /more/ not /best/ ways.
- Retric 18y ago
  If your testing in the code then your doing unit tests. But probably in a poor fashion. Unit tests should hook into the application in such a way that you can reorder the GUI and change nothing. If you are using Unit tests correctly then any change to the code base at worst requires 10% of that effort to change the test cases. If you can think of any meaningful change that is worse then that your something is wrong with how you're testing.
davidw 18y ago
> Before the changes you have code that you know works. After the changes, you know where most of the bugs are because your tests tell you where they are. The tests tell you where the tests fail. Some of those failures may be due to bugs, and others may simply due to the way your code now works.
trapper 18y ago
There are two schools of thought in programming I have seen. 1. Design it right the first time, tests and all. 2. Build a throwaway, no tests, little refactoring as you quickly iterate Programmers who fall into category 1 in my experience mistakenly believe that they understand the requirements of the software they are writing. Category 2 programmers tend to be ones who actually deal with users/real world, and know that they don't understand the requirements. My position is that if you are building something for the first time, you are going to get it wrong, so get it wrong as quickly and cheaply as possible. This means getting it in front of users as quickly as possible, bugs and all.
- swombat 18y ago
  You're assuming that you'll get something in front of users more quickly without writing tests. I find that's not the case in my experience.
  - nostrademons 18y ago
    I find there's a balance to be struck. You want to get stuff in front of users quickly, but if that stuff sucks so much that it's unusable, you aren't going to release it (and if you did, nobody would use it). So you want enough tests that you can be reasonably confident of the code and not spend a lot of time debugging, since debugging is the real timesink that'll kill you. I often find that I unit test all but the latest layer. As soon as I start depending on code from something else, it's time to write some tests. But if I just wrote something, it's probably in flux and tests would just prematurely lock down the interface. Write the code, write the client (which probably requires that you rewrite the code), and then write the tests in lieu of debugging the code.
  - trapper 18y ago
    I'd love to see some research on this. I would hypothesize that there is a point of problem complexity where the lack of tests becomes a negative for productivity.
    - mrandle 18y ago
DougBTX 18y ago
> After the changes, you know there's probably bugs, but you don't know where. Which highlights one of the "problems" with unit tests: if something fails, you have to take time to go and fix it. If you have no tests, you can assume there are no bugs because you are such a great programmer, then go home after a job well done. That there actually are bugs there doesn't bother you, because you don't know about them. It takes less time to write buggy code.
- axod 18y ago
  The alternative, is to build you code in a way which tells you quickly about bugs... decent logging, etc
  - swombat 18y ago
    I do think he was being sarcastic.
axod 18y ago
>> "Before the changes you have code that you know works. After the changes, you know where most of the bugs are because your tests tell you where they are." No. More often than not, if you change was a large architectural one, which is often the case in young software, your tests are now irrelevant, or broken. >> "It's the difference between walking with your eyes open or closed... it may be more work to process all that information about obstacles, but it sure is helpful if you want to avoid those obstacles." Do you seriously think people are just blindly coding and never testing anything here? while(true) { writeABitOfCode(); while (!compile()) fixCompilationErrors(); testThingsYouMightHaveChanged(); checkLogsForIssues(); }
- DougBTX 18y ago
  testThingsYouMightHaveChanged requires running your tests manually, which is slow and hard to fully reproduce later. It also depends on being able to make strong predictions about what your change affects, unit testing assumes that anything could have broken.
- swombat 18y ago
  * testThingsYouMightHaveChanged();* If you change that into writeAnAutomatedTestForItAndRunYourTestSuite() you get the following benefits: 1) Every test you write accrues value onto your test suite and helps you avoid problems later; 2) You don't need to spend time to figure out what you might have broken with your change - your test suite tells you in seconds; 3) You can say with some measure of certainty that it's extremely unlikely that your change broke anything.
  - axod 18y ago
    BUT you then have to modify: while (!compile()) { fixCompilationErrors(); fixUnitTests(); rewriteUnitTestsThatNoLongerMakeSense(); WriteNewUnitTests(); } Your iterate loop just became maybe twice or 3 times as long. Less iterations = worse code.
    - Retric 18y ago

shiro 18y ago

> I think it depends on the project - is it a 'design up front, go away code once' project, or an iterative constantly changing thing. I think your line is too coarse. I do heavily rely on unit tests even in "iterative constant changing projects". But in the very early "sketching" stage of development I don't bother to write tests. I just play with code on REPL (I work with Common Lisp or Scheme most of the time), in a flat namespace. Then, once I start putting code snippets together into packages (CL) or modules (Scheme), that's about the time I start writing tests as well. After that, even I constantly change stuff, sometimes drastically, unit tests do help me a lot. It is true that you need effort on reworking on the tests. Actually I think writing tests needs more work than writing actual code. And I think that's the right thing. Writing something that works is an easy part. Making sure it doesn't break in every possible situation is much harder. In language like CL, you can fairly quickly write something that takes typical input and reasonable output without serious thinking. To make sure it works in every corner case you have to think much harder and precisely, and test code reflects that thinking. (NB: I know some programmers who seem to think out those corner cases in their brain and just spit out a beautiful code at the first attempt. They may not need support from test code, I guess.)