8 ms·

I agree that testing getters/setters and other libraries functionality is a waste of time, but... You know, the "pragmatic" approach that many programmers take

I agree that testing getters/setters and other libraries functionality is a waste of time, but... You know, the "pragmatic" approach that many programmers take - "writing tests is hard and time consuming, so let's write fewer tests" is a recipe for disaster and frankly the idea that if you just think about the problem more, that you're make fewer mistakes is very much along those same lines. Here is the real problem - humans are fallible and we make mistakes. This is why pilots have flight checklists, why manufacturing has quality checks, why engineers have building codes, and why at hospitals checklists and procedures save lives If pilots and mechanics just "thought more about the problem" they would still make mistakes. You might be a brilliant surgeon, but thinking you're too smart or too good to go through a checklist could cost someone their life because of a small oversight. Would you feel good going under the knife if you knew that your life was in the hands of someone who thought they were too smart to scrub in? We are a young profession with very few actual quality standards. Luckily, most of the software that we build isn't critical to people's safety, but I'd be terrified if the software powering medical robots, life support machines, and airplane autopilot systems were built by people who were "too smart" to write tests. When a software bug costs people their lives because some programmer thought writing those tests was "too expensive" or "a waste of time" how smart is that programmer?

yummyfajitas 13y ago
I agree that testing getters/setters...is a waste of time, but... But it isn't necessarily. Say I have multiple implementations of a class, e.g.: trait TimeSeries { def datapoints: Seq[DataPoint] } case class SequenceTimeSeries(datapoints: Seq[DataPoint]) extends TimeSeries class ArrayTimeSeries(times: Array[Long], values: Array[Double]) extends TimeSeries It's pretty straightforward and nearly pointless to test that `sequenceTimeSeries.datapoints` does what I expect. On the other hand, applying the same test to `arrayTimeSeries.datapoints` exposed an off by one error in my implementation.
- tel 13y ago
  Out of curiosity—I'm not so familiar with Scala—how does an off-by-1 arise here? Aren't you just going to zip (times, values)?
  - yummyfajitas 13y ago
    I did this a while back, so I don't recall the exact off-by-one error. But I don't want to to times.zip(values) - that would involve constructing explicitly a List[DataPoint] which is precisely what I'm trying to avoid with the ArrayTimeSeries. Rather, I defined a custom Seq[DataPoint], which had the apply(i: Int) method construct datapoints on the fly from the arrays. (I admit that the error was entirely stupidity on my part and not the result of anything complicated. If I was as smart as Rich Hickey, I probably wouldn't need so many tests.)
    - tel 13y ago
      Ohh, I see, array-of-structures/structure-of-arrays trouble.
- puredanger 13y ago
  So test that one. Please, I encourage you to engage your brain in the process of choosing what does and does not need to be tested. That is the whole point.
taeric 13y ago
My problem is just how adverse to utilizing a proper QA department many developers are. Sure, unit tests and other automated systems can and should be used to help out in creation of a system. However, realize that there are folks who have the sole job of testing a system. Unsurprisingly, many of them are actually very good at it. Instead of spending a lot of time trying to make a more convoluted model just so you can test it, use the real tools at your disposal to accomplish the same goal. I fully grant that the line between "convoluted model" and "adequately testable" system is a tough one to spot. I am open to some good rules in distinguishing them.
- engrenage 13y ago
  A QA department doesn't locate the bugs for you. They report bugs, and then developers have to repeat the procedure and investigate. This is a costly round-trip. Automated testing is intended to be a much cheaper way to stop bugs from propagating up to QA, and it usually provides much better and more immediate information about the precise nature and location of bugs.
  - taeric 13y ago
    But an automated testing suite does not write itself. Not even in a "statically" typed language. In those worlds, the staffing costs of having a dedicated QA team can be far cheaper than engineering your system to be perfectly modelled for testing. This is especially true in areas that do not already have established models. (And is why early attempts at such endeavors as bridge building focused on prototypes and iterative solutions, and the rigorous mathematical models came later.)
    - engrenage 13y ago
      If you remove the word 'perfectly', you're just saying that a dedicated QA team is cheaper than writing tests. I doubt that very much.
      - taeric 13y ago
puredanger 13y ago
Would you fly an airline that did 24 hours of verification before every flight thus making flights cost 10x as much? (and same argument for all your other examples...) Our job as professionals is to verify that the software we write does what we expect. Our job is also to manage costs appropriate to the application (not just up-front but over the long haul). For me, that means that we want our tests to maximize value per effort. I am not saying a) write fewer tests b) tests are a waste of time or c) tests are too expensive. I'm saying: a) write better tests, b) some tests are a waste of time and c) let's write tests that do more work for us.
- pbiggar 13y ago
  [disclaimer: I work at https://circleci.com https://circleci.com, so I do have a vested interest in disagreeing with you ;) ] I think is a strawman. If running a huge suite of automated verification was really cheap, then airlines would run it before each flight.
  - puredanger 13y ago
    Hey Paul, I totally agree that the cost of running big test suites can be mitigated and CircleCI is doing awesome stuff there. That does nothing to address the problem of maintaining a large suite of tests tightly coupled to your code. That is, the cost of updating many example-based tests when you change the code.
    - pbiggar 13y ago
      I guess that depends on how much of your suite is in flux. We have largely example-driven tests, and our suite is pretty big: about 16kLOC. I can't think of a case where a significant amount of time was spent fixing the test suite (and in those cases, it was generally useful to think about why). Also, I haven't seen anything that works better. Generative tests are nice in theory, but I haven't seem the be that useful for anything other than short, pure, functions.
      - jamii 13y ago
kaonashi 13y ago
Really, what it comes down to is that with a language like Clojure, that has very isolated side effects and works primarily with pure functions, getting to high test coverage requires a lot less testing code.

yummyfajitas 13y ago

I agree that testing getters/setters...is a waste of time, but... But it isn't necessarily. Say I have multiple implementations of a class, e.g.: trait TimeSeries { def datapoints: Seq[DataPoint] } case class SequenceTimeSeries(datapoints: Seq[DataPoint]) extends TimeSeries class ArrayTimeSeries(times: Array[Long], values: Array[Double]) extends TimeSeries It's pretty straightforward and nearly pointless to test that `sequenceTimeSeries.datapoints` does what I expect. On the other hand, applying the same test to `arrayTimeSeries.datapoints` exposed an off by one error in my implementation.

tel 13y ago
Out of curiosity—I'm not so familiar with Scala—how does an off-by-1 arise here? Aren't you just going to zip (times, values)?
- yummyfajitas 13y ago
  I did this a while back, so I don't recall the exact off-by-one error. But I don't want to to times.zip(values) - that would involve constructing explicitly a List[DataPoint] which is precisely what I'm trying to avoid with the ArrayTimeSeries. Rather, I defined a custom Seq[DataPoint], which had the apply(i: Int) method construct datapoints on the fly from the arrays. (I admit that the error was entirely stupidity on my part and not the result of anything complicated. If I was as smart as Rich Hickey, I probably wouldn't need so many tests.)
  - tel 13y ago
    Ohh, I see, array-of-structures/structure-of-arrays trouble.
puredanger 13y ago
So test that one. Please, I encourage you to engage your brain in the process of choosing what does and does not need to be tested. That is the whole point.

taeric 13y ago

My problem is just how adverse to utilizing a proper QA department many developers are. Sure, unit tests and other automated systems can and should be used to help out in creation of a system. However, realize that there are folks who have the sole job of testing a system. Unsurprisingly, many of them are actually very good at it. Instead of spending a lot of time trying to make a more convoluted model just so you can test it, use the real tools at your disposal to accomplish the same goal. I fully grant that the line between "convoluted model" and "adequately testable" system is a tough one to spot. I am open to some good rules in distinguishing them.

engrenage 13y ago
A QA department doesn't locate the bugs for you. They report bugs, and then developers have to repeat the procedure and investigate. This is a costly round-trip. Automated testing is intended to be a much cheaper way to stop bugs from propagating up to QA, and it usually provides much better and more immediate information about the precise nature and location of bugs.
- taeric 13y ago
  But an automated testing suite does not write itself. Not even in a "statically" typed language. In those worlds, the staffing costs of having a dedicated QA team can be far cheaper than engineering your system to be perfectly modelled for testing. This is especially true in areas that do not already have established models. (And is why early attempts at such endeavors as bridge building focused on prototypes and iterative solutions, and the rigorous mathematical models came later.)
  - engrenage 13y ago
    If you remove the word 'perfectly', you're just saying that a dedicated QA team is cheaper than writing tests. I doubt that very much.
    - taeric 13y ago

puredanger 13y ago

Would you fly an airline that did 24 hours of verification before every flight thus making flights cost 10x as much? (and same argument for all your other examples...) Our job as professionals is to verify that the software we write does what we expect. Our job is also to manage costs appropriate to the application (not just up-front but over the long haul). For me, that means that we want our tests to maximize value per effort. I am not saying a) write fewer tests b) tests are a waste of time or c) tests are too expensive. I'm saying: a) write better tests, b) some tests are a waste of time and c) let's write tests that do more work for us.

pbiggar 13y ago
[disclaimer: I work at https://circleci.com https://circleci.com, so I do have a vested interest in disagreeing with you ;) ] I think is a strawman. If running a huge suite of automated verification was really cheap, then airlines would run it before each flight.
- puredanger 13y ago
  Hey Paul, I totally agree that the cost of running big test suites can be mitigated and CircleCI is doing awesome stuff there. That does nothing to address the problem of maintaining a large suite of tests tightly coupled to your code. That is, the cost of updating many example-based tests when you change the code.
  - pbiggar 13y ago
    I guess that depends on how much of your suite is in flux. We have largely example-driven tests, and our suite is pretty big: about 16kLOC. I can't think of a case where a significant amount of time was spent fixing the test suite (and in those cases, it was generally useful to think about why). Also, I haven't seen anything that works better. Generative tests are nice in theory, but I haven't seem the be that useful for anything other than short, pure, functions.
    - jamii 13y ago

kaonashi 13y ago

Really, what it comes down to is that with a language like Clojure, that has very isolated side effects and works primarily with pure functions, getting to high test coverage requires a lot less testing code.