3 ms·
This story I think very neatly points out the difference between verification and validation -- and why it's important to have both of them in a testplan for an
by jwise0 12y ago
This story I think very neatly points out the difference between verification and validation -- and why it's important to have both of them in a testplan for anything new.
In short, verification asks the question: "does it do what we designed it to do?" For instance, if you have a reference encoder of some kind that is a verbatim translation of a specification, and a high-performance encoder that is designed to squeeze every last drop out of the system, you can verify the high-performance version by comparing the output to the reference; if they're the same, then you know that the high-performance version, at least on some level, works as designed.
If verification compares the design against the implementation, then validation asks the question: "is the design what it needs to be?". The story that xiphmont told here is a validation issue: the implementation is perfectly correct, but it's the design that isn't what he'd hoped for! Although verification often takes more of the time than validation, in my experience, validation often seems to take quite a lot more creativity than verification.
Thanks to Monty for writing this up, and providing such an illustrative example of the difference! It's a useful distinction to have, especially when looking at errors that escaped testing and made it to production.
- pm 12y agoAs you put it, validation requires that you ask the right questions, which is half the ball game.
- QuantumRoar 12y agoI've had a lot of these problems where everything is correct but nothing works as intended. Turns out, numerical mathematics excels at making programming more about validation than verification. Breaking the continuum into pieces breaks your symmetries. For example on a finite rectangular grid, a sphere is not rotationally invariant. Even if your maths is valid in the continuum, for the implementation, you still need to prove that it is valid (i.e. fulfills all desired properties) on a discrete grid. I have zero experience with video compression, however, the author states that the issues get less severe when the video is rotated by 90˚. I guess the compression algorithm is designed to work irrespective of orientation. Therefore, my humble guess is that they failed to implement the desired symmetry of the continuum in the compression algorithm that acts upon discrete spatial coordinates. Unfortunately, there's no way to recover all symmetries from the continuum on a discrete grid. You always get some kind of artifacts and you can only work to reduce them for the case you are considering.