4 ms·

I firmly agree with your second paragraph and the general point the link made about error message being vastly superior in clang. I don't strictly disagree wit

by nolok 4y ago

I firmly agree with your second paragraph and the general point the link made about error message being vastly superior in clang. I don't strictly disagree with your first point, but I disagree with the attitude from many dev regarding tests that lead to it becoming too common like in this article (and I'm not piling on the guy, it's very very common). It's like people have forgotten what tests are for and thus what you want from them: a test that pass is essentially useless, it completes the green check mark and that's it, it doesn't matter. It's the test that fails that provide you information, that matters to you. One might be tempted to disagree, tell me that passing test also provide information, by ensuring this and that and I will insist that they take a second and realize that no, they don't. Ultimately, the only value they have is in fact that they can fail, it's always the failure that matter. There are indeed tests which fails to fail for other unplanned reasons, by that's not really what I'm getting into, I'm aiming specifically for the attitude of tests for the sake of tests.

TheDong 4y ago
What you're saying isn't all that precise, but I think I disagree with it. > It's like people have forgotten what tests are for and thus what you want from them: a test that pass is essentially useless, it completes the green check mark and that's it, it doesn't matter This is, in my opinion, not true in some cases. Sometimes, I'll write unit tests that never fail, and are purely to allow me to verify one of my assumptions about the code. Basically, the unit test is a tool to force me to think through each line of the code, and that's it. Typically, when I do the above, I also use a debugger and/or line coverage tool to ensure the test actually reaches part of the code I'm trying to reason about, and with the debugger, this sort of "test-writing as a tool to think about code" is in my opinion much more powerful than just reading code. I do this a lot in code review when I don't understand someone's code, since it's easier to run sample values through it and set breakpoints in a test, than it is to do all that in my head looking at the diff view. Now, you might "no true scottsman" argue that those are not tests (and indeed, I often don't bother to check them in), but I think they are. I also think integration and blackbox tests that never fail are fine, and expected. If I add a website uptime monitor (a "foo.com returns 200" type test), should I immediately take my site offline for a while to make sure it fails?
- robertlagrant 4y ago
  > If I add a website uptime monitor (a "foo.com returns 200" type test), should I immediately take my site offline for a while to make sure it fails? Ideally yes, to test the uptime monitor!
- nolok 4y ago
  > This is, in my opinion, not true in some cases. Sometimes, I'll write unit tests that never fail, and are purely to allow me to verify one of my assumptions about the code. Basically, the unit test is a tool to force me to think through each line of the code, and that's it. That is fine and has value, but that is usually called an assertion (and most language have them under some sort of assert() function), which is different from an unit test. > If I add a website uptime monitor (a "foo.com returns 200" type test), should I immediately take my site offline for a while to make sure it fails? How do you know if it works otherwise ? In fact for that specific exemple, the most naive version of that test actually misses many failure cases as we've seen again and again in the last decade with status page that fail to detect any error
  - TheDong 4y ago
    Assertions are to validate things do not happen. They do not let me trace a sample value through code with a debugger, step through lines with that value, and add further breakpoints. unit tests let me pick sample values, and I do what I'm talking about using unit test libraries, not assertion libraries. Perhaps you misunderstood me. > many failure cases as we've seen again and again in the last decade with status page that fail to detect any error That is a political problem, not a testing problem. The majority of status pages that fail to detect errors are because the company decided to have a human update it because the test failed too often, or did not differentiate "error" and "warn" or provide a human description. Status page uptime is often a political matter for a company, and no unit testing practices will fix a political problem.
pizlonator 4y ago
Of course people should write tests. But there are many subtle reasons why a test might not get written. Sometimes it’s deadlines. Sometimes it’s because a component is hard to test in isolation. Sometimes it’s just an honest mistake. So, while I would encourage everyone to write tests I also won’t judge them for not doing it.

by nolok 4y ago

TheDong 4y ago

What you're saying isn't all that precise, but I think I disagree with it. > It's like people have forgotten what tests are for and thus what you want from them: a test that pass is essentially useless, it completes the green check mark and that's it, it doesn't matter This is, in my opinion, not true in some cases. Sometimes, I'll write unit tests that never fail, and are purely to allow me to verify one of my assumptions about the code. Basically, the unit test is a tool to force me to think through each line of the code, and that's it. Typically, when I do the above, I also use a debugger and/or line coverage tool to ensure the test actually reaches part of the code I'm trying to reason about, and with the debugger, this sort of "test-writing as a tool to think about code" is in my opinion much more powerful than just reading code. I do this a lot in code review when I don't understand someone's code, since it's easier to run sample values through it and set breakpoints in a test, than it is to do all that in my head looking at the diff view. Now, you might "no true scottsman" argue that those are not tests (and indeed, I often don't bother to check them in), but I think they are. I also think integration and blackbox tests that never fail are fine, and expected. If I add a website uptime monitor (a "foo.com returns 200" type test), should I immediately take my site offline for a while to make sure it fails?

robertlagrant 4y ago
> If I add a website uptime monitor (a "foo.com returns 200" type test), should I immediately take my site offline for a while to make sure it fails? Ideally yes, to test the uptime monitor!
nolok 4y ago
> This is, in my opinion, not true in some cases. Sometimes, I'll write unit tests that never fail, and are purely to allow me to verify one of my assumptions about the code. Basically, the unit test is a tool to force me to think through each line of the code, and that's it. That is fine and has value, but that is usually called an assertion (and most language have them under some sort of assert() function), which is different from an unit test. > If I add a website uptime monitor (a "foo.com returns 200" type test), should I immediately take my site offline for a while to make sure it fails? How do you know if it works otherwise ? In fact for that specific exemple, the most naive version of that test actually misses many failure cases as we've seen again and again in the last decade with status page that fail to detect any error
- TheDong 4y ago
  Assertions are to validate things do not happen. They do not let me trace a sample value through code with a debugger, step through lines with that value, and add further breakpoints. unit tests let me pick sample values, and I do what I'm talking about using unit test libraries, not assertion libraries. Perhaps you misunderstood me. > many failure cases as we've seen again and again in the last decade with status page that fail to detect any error That is a political problem, not a testing problem. The majority of status pages that fail to detect errors are because the company decided to have a human update it because the test failed too often, or did not differentiate "error" and "warn" or provide a human description. Status page uptime is often a political matter for a company, and no unit testing practices will fix a political problem.

pizlonator 4y ago

Of course people should write tests. But there are many subtle reasons why a test might not get written. Sometimes it’s deadlines. Sometimes it’s because a component is hard to test in isolation. Sometimes it’s just an honest mistake. So, while I would encourage everyone to write tests I also won’t judge them for not doing it.