3 ms·
That is wrong in general. With enough tests you absolutely can show the absence of bugs for certain programs. It is for example easy to test „hello world“ exhau
by adrianN 2y ago
That is wrong in general. With enough tests you absolutely can show the absence of bugs for certain programs. It is for example easy to test „hello world“ exhaustively.
- mgsouth 2y agoLet's say you've tested this thing 1 million times. Each time the output was automatically checked by five different and independently-developed test suites. You're ready to swear there's no possible way for it to fail. And then someone tries it with a ulimit of 16kB. Does it run? Do you _know_? Do you even know what is correct behavior in this situation?
- adrianN 2y agoThe system it runs on is part of the specification. A program is correct if fulfills all specified requirements. You're saying a car is defective because it breaks when you put sugar in the tank.
- deleted 2y ago[deleted]
- odyssey7 2y agoThis is a no true Scotsman fallacy. Any time it runs incorrectly, it was invoked with the wrong operating conditions — which seem to be defined as any conditions that cause it to run incorrectly. Sugar in the tank is an agreeable example because of how obvious it is, but what about something more subtle? An odd condition that leads to the wrong resonant frequency. An unknown software bug that makes the brakes lock up on Tuesdays at 12:00 AM on one specific backroad in a remote part of Virginia. The combinatorial possibilities of operating conditions are too numerous to exhaustively test. I guess you could say that every function has every quality that it happens to have, so that functions need only exist in order to be “correct.”
- adrianN 2y agoYou typically write down the specification before claiming that your code is done so you can’t use it to claim that any undesired behavior is not a bug. I naturally agree that in general tests are insufficient to show correctness because the state space is of impractical size, but, as I said above, for certain programs they totally can be.
- worthless-trash 2y agoSpoken like someone who has never had to deal with undefined behavior corner cases.
- adrianN 2y agoI have written safety critical software.
- worthless-trash 2y agoI'm now scared.
- odyssey7 2y agoI agree, but the word “certain” really needs to not be overlooked in this statement.