4 ms·

The problem here is that if it works, how is it superstition? When I knock on wood or throw salt over my shoulder, I don't have a way of verifying that I've av

The problem here is that if it works, how is it superstition? When I knock on wood or throw salt over my shoulder, I don't have a way of verifying that I've averted disaster. However, I can test the code I don't understand, and if it does what it says it does (or does what I want it to do, since that's why I've selected it in the first place), then we're a step removed from superstition. There might be unintended consequences if I truly don't understand the code, but that is something else than superstition.

rbanffy 12y ago
If you do not understand the code, you cannot say you tested it. It may have corner cases you did not anticipate. The test may be incomplete and only cover what you think the program should do.
- tjradcliffe 12y ago
  Nonsense. We test stuff all the time we don't understand "completely" (which is some kind of magical standard, as we obviously don't understand the universe completely, so the very atoms and molecules that the machine is made of are in some sense not well understood.) Certainty is the alchemist's stone of philosophy. People have done all kinds of interesting stuff while attempting the impossible: turning knowledge into certainty. They can't, but that doesn't mean knowledge--which is inherently uncertain--is inferior. It means certainty is inferior, because it can't actually be achieved, regardless of what any pre-Bayesian thinker might have imagined. Knowledge is always uncertain. Tests are always incomplete. This does not mean "knowledge is impossible" or "you cannot say you tested the code" but rather "certainty is a chimera" and "compete test coverage is a futile goal." The whole point if good engineering is to adopt standards that are both achievable and useful, and the kind of test coverage that people run on code they are unfamiliar with to ensure it will do what they want it to is generally adequate for that. Evidence: most shipping code actually works.
  - sheepmullet 12y ago
    "Evidence: most shipping code actually works." The vast majority of time spent on software development is on bug fixing and on many projects the construction/coding defects make up more than half of all the defects.
- mreiland 12y ago
  why write unit tests when we don't understand the microcode in the CPU? We can't have "tested" it because we don't understand said microcode. Or wait... no, maybe your characterization is wrong. Yeah... that's probably it.
  - ackalker 12y ago
    That's why they're called unit tests. They are supposed to test precisely one unit of functionality, while assuming everything else works as specified (using mocks if necessary). Taking into account the effects of microcode execution on the test result is in fact a kind of integration testing.
    - SilasX 12y ago
      I think the GP's point was that you can't mock out the computer chip's opcodes.
  - rbanffy 12y ago
    Where exactly did I say you should understand the code down to how electrons flow through the chip's gates? I said if you do not understand what you are testing (not the compiler, not the runtime libraries, not the OS, not the instruction decode, not the microcode, not the gates), you can't say you actually tested it. You merely observed what could be a side-effect.
    - mreiland 12y ago
      you're drawing an arbitrary line, that was the point. We specialize and abstract for a reason. No one understands everything about everything surrounded their code, and that includes other software. Don't tell me you've personally been through the code of every single project your projects touch. You haven't and that makes you a hypocrite.
      - rbanffy 12y ago
        A line that encompasses the object you are testing but that you don't fully understand (or that is hidden from you) and nothing more is not arbitrary. The best you can do in these circumstances is say it adheres to a spec for the cases you tested (which could, for all you know, be the only ones that yield correct results).
- klibertp 12y ago
  Tests are always like this, no matter how well you know the code. A test that covers every possible case is called a proof.
brudgers 12y ago
The problem here is that if it works, how is it superstition? Consider Bertrand Russell's Chicken: The man who has fed the chicken every day throughout its life at last wrings its neck instead, showing that more refined views as to the uniformity of nature would have been useful to the chicken. [1] Sometimes things working but not in the way we imagine they work is worse than them not working at all. [1]: http://www.ditext.com/russell/rus6.html http://www.ditext.com/russell/rus6.html
- vacri 12y ago
  Had the chicken been living in the wild, it might have lived a long, full life... or it might have been painfully maimed or slaughtered in the myriad of nasty ways that nature has to offer, at any age from before it hatched on up. I don't think it's at all safe to assume that a comfortable life with a definite-but-unknown end point in your prime is definitively worse than a life subject to the random events of the wild. Particularly for something that's down the lower end of the food chain.

rbanffy 12y ago

If you do not understand the code, you cannot say you tested it. It may have corner cases you did not anticipate. The test may be incomplete and only cover what you think the program should do.

tjradcliffe 12y ago
Nonsense. We test stuff all the time we don't understand "completely" (which is some kind of magical standard, as we obviously don't understand the universe completely, so the very atoms and molecules that the machine is made of are in some sense not well understood.) Certainty is the alchemist's stone of philosophy. People have done all kinds of interesting stuff while attempting the impossible: turning knowledge into certainty. They can't, but that doesn't mean knowledge--which is inherently uncertain--is inferior. It means certainty is inferior, because it can't actually be achieved, regardless of what any pre-Bayesian thinker might have imagined. Knowledge is always uncertain. Tests are always incomplete. This does not mean "knowledge is impossible" or "you cannot say you tested the code" but rather "certainty is a chimera" and "compete test coverage is a futile goal." The whole point if good engineering is to adopt standards that are both achievable and useful, and the kind of test coverage that people run on code they are unfamiliar with to ensure it will do what they want it to is generally adequate for that. Evidence: most shipping code actually works.
- sheepmullet 12y ago
  "Evidence: most shipping code actually works." The vast majority of time spent on software development is on bug fixing and on many projects the construction/coding defects make up more than half of all the defects.
mreiland 12y ago
why write unit tests when we don't understand the microcode in the CPU? We can't have "tested" it because we don't understand said microcode. Or wait... no, maybe your characterization is wrong. Yeah... that's probably it.
- ackalker 12y ago
  That's why they're called unit tests. They are supposed to test precisely one unit of functionality, while assuming everything else works as specified (using mocks if necessary). Taking into account the effects of microcode execution on the test result is in fact a kind of integration testing.
  - SilasX 12y ago
    I think the GP's point was that you can't mock out the computer chip's opcodes.
- rbanffy 12y ago
  Where exactly did I say you should understand the code down to how electrons flow through the chip's gates? I said if you do not understand what you are testing (not the compiler, not the runtime libraries, not the OS, not the instruction decode, not the microcode, not the gates), you can't say you actually tested it. You merely observed what could be a side-effect.
  - mreiland 12y ago
    you're drawing an arbitrary line, that was the point. We specialize and abstract for a reason. No one understands everything about everything surrounded their code, and that includes other software. Don't tell me you've personally been through the code of every single project your projects touch. You haven't and that makes you a hypocrite.
    - rbanffy 12y ago
      A line that encompasses the object you are testing but that you don't fully understand (or that is hidden from you) and nothing more is not arbitrary. The best you can do in these circumstances is say it adheres to a spec for the cases you tested (which could, for all you know, be the only ones that yield correct results).
klibertp 12y ago
Tests are always like this, no matter how well you know the code. A test that covers every possible case is called a proof.

brudgers 12y ago

The problem here is that if it works, how is it superstition? Consider Bertrand Russell's Chicken: The man who has fed the chicken every day throughout its life at last wrings its neck instead, showing that more refined views as to the uniformity of nature would have been useful to the chicken. [1] Sometimes things working but not in the way we imagine they work is worse than them not working at all. [1]: http://www.ditext.com/russell/rus6.html http://www.ditext.com/russell/rus6.html

vacri 12y ago
Had the chicken been living in the wild, it might have lived a long, full life... or it might have been painfully maimed or slaughtered in the myriad of nasty ways that nature has to offer, at any age from before it hatched on up. I don't think it's at all safe to assume that a comfortable life with a definite-but-unknown end point in your prime is definitively worse than a life subject to the random events of the wild. Particularly for something that's down the lower end of the food chain.