5 ms·

A tangent, and somewhat philosophical: is "The primary purpose of software testing" really "to detect any potential defects in a program before it reaches its i

by tomgp 6y ago

A tangent, and somewhat philosophical: is "The primary purpose of software testing" really "to detect any potential defects in a program before it reaches its intended consumers"? I would argue that whilst that is a commonly held belief, and certainly true for some tests of business logic, the main value and reason for testing is to allow developers to confidently make changes to existing code. In my experience user acceptance testing is a more useful tool for detecting user facing defects as it also tests the developers assumptions about functionality.

FartyMcFarter 6y ago
But if we didn't care about defects, developers could confidently make changes to existing code without having tests. The end goal is still avoiding defects, helping developers is just the "middle-man".
- nickff 6y ago
  I agree with this view, and would like to add that I see improving the tests available to the developer as really speeding up the OODA loop of development.
- hinkley 6y ago
  The interim goal is objective confidence, not Dunning-Kruger confidence. That’s how you go faster. There’s always someone who’s willing to plow ahead, safeties off, and ignore the warning signs they see and the complaints of their peers. Denial is a hell of a drug (“That bug can’t be in my code, la la la can’t hear you look how fast I’m going!”). Some people need a little back pressure.
rocqua 6y ago
I agree, but the two are very similar if you read 'defects in a program' as bugs. Bugs can be hard to cause, and thus not be noticed by user acceptance testing. So for catching them, tests are more useful.
deleted 6y ago
[deleted]
rkangel 6y ago
Let's look at it a different way and start with the assumption that in all cases we're delivering reliable software, it's just a question of how we get there. The simplest way of doing this is to write code with good practices then do a load of manual system testing to find bugs (system testing including user acceptance testing in this example). This is basically what most people were doing 20 years ago. This can work, but doing the system testing is time consuming and so expensive and so makes making changes expensive. Instead you automate some or all of your system testing. This makes them cheaper, but these tests are expensive to maintain and still time consuming to run. If the test fails you don't have that much information about the problem. To complement (or maybe replace) your automated system testing, you add tests for smaller and smaller chunks of software, allowing for tighter feedback loops and more precise diagnosis (there are also some negative tradeoffs). Applied well, this allows you to write features more quickly, refactor more confidently and generally develop more efficiently. Basically, it's not so much about catching bugs as it is about getting to your desired level of reliability in the most cost and time efficient way.
- hinkley 6y ago
  > start with the assumption that in all cases we're delivering reliable software. This is like the physics joke about spherical cows. You cannot assume this. Good software doesn’t just happen, it takes a ton of work, a lot of which is invisible - if you succeed. People might get lucky or cleverly hide problems in version 1 but in a long enough time horizon, the scales fall away and the hot messes generally reveal themselves.
- drewcoo 6y ago
  It's also a question of the there you want to get to. And how fast you want to get there. Testing can be about a lot more than just catching bugs. It can be about any measure of reality that can be turned into useful feedback to make developers more aware. Testing is like the headlights on your car - they're useful even in absence of bugs.
hinkley 6y ago
I was left stupefied by a senior staff member asserting that the point of tests was full code coverage. I thought I knew why he left such a wake behind him but apparently he had whole dimensions of crazy I had yet to discover.
bluGill 6y ago
I prefer to express it as the purpose of tests is to ensure something never changes. I don't care how future maintainer touches the code so long as given this set of inputs this is the output.
- falcolas 6y ago
  People are touching the code because something has to change. Typically in the service of a new business case.
  - bluGill 6y ago
    if your code is correctly designed the business case that change doesn't require most of the tests to change. I have modules that haven't had any changed in 7 years, and I don't expect there to ever be a business case to change them. Even the modules that change, most of them are changing to add a customization in a place/way where we normally add them, so no existing test needs to change to support the new business case. There are exceptions, but tests that change because the business case changed are a design smell in general.
  - TheCoelacanth 6y ago
    And the tests help you avoid accidentally changing something other than the thing you want to change.

by tomgp 6y ago

FartyMcFarter 6y ago

But if we didn't care about defects, developers could confidently make changes to existing code without having tests. The end goal is still avoiding defects, helping developers is just the "middle-man".

nickff 6y ago
I agree with this view, and would like to add that I see improving the tests available to the developer as really speeding up the OODA loop of development.
hinkley 6y ago
The interim goal is objective confidence, not Dunning-Kruger confidence. That’s how you go faster. There’s always someone who’s willing to plow ahead, safeties off, and ignore the warning signs they see and the complaints of their peers. Denial is a hell of a drug (“That bug can’t be in my code, la la la can’t hear you look how fast I’m going!”). Some people need a little back pressure.

rocqua 6y ago

I agree, but the two are very similar if you read 'defects in a program' as bugs. Bugs can be hard to cause, and thus not be noticed by user acceptance testing. So for catching them, tests are more useful.

deleted 6y ago

[deleted]

rkangel 6y ago

Let's look at it a different way and start with the assumption that in all cases we're delivering reliable software, it's just a question of how we get there. The simplest way of doing this is to write code with good practices then do a load of manual system testing to find bugs (system testing including user acceptance testing in this example). This is basically what most people were doing 20 years ago. This can work, but doing the system testing is time consuming and so expensive and so makes making changes expensive. Instead you automate some or all of your system testing. This makes them cheaper, but these tests are expensive to maintain and still time consuming to run. If the test fails you don't have that much information about the problem. To complement (or maybe replace) your automated system testing, you add tests for smaller and smaller chunks of software, allowing for tighter feedback loops and more precise diagnosis (there are also some negative tradeoffs). Applied well, this allows you to write features more quickly, refactor more confidently and generally develop more efficiently. Basically, it's not so much about catching bugs as it is about getting to your desired level of reliability in the most cost and time efficient way.

hinkley 6y ago
> start with the assumption that in all cases we're delivering reliable software. This is like the physics joke about spherical cows. You cannot assume this. Good software doesn’t just happen, it takes a ton of work, a lot of which is invisible - if you succeed. People might get lucky or cleverly hide problems in version 1 but in a long enough time horizon, the scales fall away and the hot messes generally reveal themselves.
drewcoo 6y ago
It's also a question of the there you want to get to. And how fast you want to get there. Testing can be about a lot more than just catching bugs. It can be about any measure of reality that can be turned into useful feedback to make developers more aware. Testing is like the headlights on your car - they're useful even in absence of bugs.

hinkley 6y ago

I was left stupefied by a senior staff member asserting that the point of tests was full code coverage. I thought I knew why he left such a wake behind him but apparently he had whole dimensions of crazy I had yet to discover.

bluGill 6y ago

I prefer to express it as the purpose of tests is to ensure something never changes. I don't care how future maintainer touches the code so long as given this set of inputs this is the output.

falcolas 6y ago
People are touching the code because something has to change. Typically in the service of a new business case.
- bluGill 6y ago
  if your code is correctly designed the business case that change doesn't require most of the tests to change. I have modules that haven't had any changed in 7 years, and I don't expect there to ever be a business case to change them. Even the modules that change, most of them are changing to add a customization in a place/way where we normally add them, so no existing test needs to change to support the new business case. There are exceptions, but tests that change because the business case changed are a design smell in general.
- TheCoelacanth 6y ago
  And the tests help you avoid accidentally changing something other than the thing you want to change.