3 ms·
I think this chart has cause and effect backwards. Places with programmers capable of doing formal code reviews are less likely to have bugs in the first place.
by ars 15y ago
I think this chart has cause and effect backwards. Places with programmers capable of doing formal code reviews are less likely to have bugs in the first place.
It's really hard to do a (proper) code review, a programmer who is capable of it is probably a high caliber programmer (I assume programmers review each others code).
Also I don't like this sentence: "code reading detected 80 percent more faults per hour than testing".
If you look at the article that sentence links to you will find that the problem is poor technique in creating the tests. You can't do "white box" testing without reading and understanding the code in the first place. And if their "black box" testing didn't find bugs, they didn't write the test properly in the first place.
- DarkShikari 15y agoAnd if their "black box" testing didn't find bugs, they didn't write the test properly in the first place. That's practically begging the question: a test isn't a real test unless it catches all possible bugs? Really, any code for which such a magical test can be written is probably so simple that it only exists inside Fizzbuzz interviews. Years of experience have consistently shown me that 10 minutes of looking over code by another developer will find more bugs than days of testing. A huge number of bugs are obvious to the eye, but have high odds of escaping tests. Doubly so in code where there's no "right" output to test for: as in the case of any non-optimal optimization function. This doesn't mean tests are bad. Tests are good for the sort of code that's easily testable, and large-scale regression testing is needed for any project, even if unit tests aren't feasible. But testing is no substitute for having people read the bloody code.
- ars 15y agoYou are thinking unit tests, I am thinking functionality tests. i.e. you known what the code is supposed to do then you test that it really does it. Then you think about edge cases and test those.
- DarkShikari 15y agoi.e. you known what the code is supposed to do then you test that it really does it. Maybe if your code is "Hello World", but what if your code is a motion search, or a facial recognition algorithm, or a band-pass filter, or any other sort of real-world code whose operation can't be summed up exactly in two lines of code?
- ars 15y agoThen you run some samples through it and make sure it works correctly. As many samples as possible, and since you wrote it you know which samples are hardest for it, and which trigger edge cases. I'm not saying never review code, I'm just not convinced that a programmer's own testing works so poorly.
- DarkShikari 15y agoThen you run some samples through it and make sure it works correctly. What if the exact meaning of "correctly" isn't defined? Here's what happens when you blindly apply this approach to real-world applications: you end up with hardcoded "correct" md5sums that are "whatever the function happened to output last time, and we checked and thought it was right". I've seen this happen, repeatedly, with real-world apps, and it is useless at best and usually harmful. For a huge span of real-world applications, there is NO RIGHT ANSWER. You can't blindly apply unit-testing to that in the same way you would for a web framework.
- mitchty 15y agoSo I guess the question is, if the meaning of "correct" isn't defined, how did you arrive at code to do that thing? Are we talking neural networks here or what? Even those get tested, maybe not in an automated way, but you could at least make sure the results are better than your last iteration. Take the facial recognition example. Give the whole thing a few images that it should fail on, and some it should pass. As you learn more about the system and where things fail or just don't add up, add tests there. Its likelier those parts of the code are bad too so refactoring isn't unheard of. To a degree I completely agree with you that some things don't have a easily known answer in the real world. But adding tests to test known things like regressions, just seems like it should be a minimum of testing. I can't count the real world apps I've seen without a single "proof" that they don't regress state as things change. And they do, often and repeatedly. But dealing with external products and teams can be frustrating.
- felipemnoa 15y ago>>10 minutes of looking over code by another developer will find more bugs than days of testing I find this surprising. Unless you are talking about new developers. Automated test cases should be more than unit test cases. They should include system wide test cases as much as possible. i.e. For a game you would write test cases on the physics engine. The physics engine includes collision detection and collision response. Of course, you would also have test cases for each of those modules independently.
- nl 15y agoin Code: static final String HELLO = "helo wolrd"; ... out.println(HELLO); in test: input = in.readln(); // yeah massively simplified, I know assertEqual(Code.HELLO, input); pass. But a developer will (should!) find the error immediately.
- deleted 15y ago[deleted]
- boyter 15y agoThat's a bad test though. Its not brittle but assuming it was written in a Tdd manner there should be at least two chances for the developer to catch the bug if they don't use the constant they are testing in the test. I have no source on this but isn't it bad practice to use constants in your tests? I personally use a new string with the expected value. It forces me to look twice and really consider what I am doing.
- nl 15y agoIt's a comment on hacker news illustrating a point, not a guide of how to write test cases. The point is that it is possible to have good test coverage and still have bugs that are best found by humans.
- eru 15y ago> A huge number of bugs are obvious to the eye, but have high odds of escaping tests. Doubly so in code where there's no "right" output to test for: as in the case of any non-optimal optimization function. I agree in principle, but your example is off. Optimization is testable, and even provable. If your optimizers doesn't rely on ad-hoc techniques, you can also usually give an estimate of how far away your solution is from the true optimum.