2 ms·
Consider the input as a big space of points, where each point is a possible input. These points aren't unrelated; speaking loosely, foo(3.000000000001) is usual
by mysterymath 5y ago
Consider the input as a big space of points, where each point is a possible input. These points aren't unrelated; speaking loosely, foo(3.000000000001) is usually more likely to behave like foo(3) than foo(4) is.
The different behaviors on this space of points are due to the control flow and mathematical expressions within the function. You could thus model the function's behavior as a partition of the input: the different sets of inputs that will behave "identically" WRT the control flow inside the function. You could probably formalize this with a suitable notion of continuity, but I haven't bothered.
Accordingly, testing corresponds to searching this space for problematic partitions. It's a bit like playing battleship: if you know a lot about the possible shapes of errors in this mathematical space, then you can choose a few small points that can totally prove their absence.
Even if you don't know the shapes though, by choosing your points carefully, you can rule out shapes that are "sufficiently large". This is the impetus behind boundary testing; you don't prove that errors don't exist, but you can prove that they are relatively "pointlike"; that there aren't arbitrarily wide swaths of the input space that all exhibit the same erroneous behavior. Since the input space is usually vastly multidimensional, there's usually pretty hard limits on how much you can actually reduce the set of possible "problem shapes," though.
A lot of in-practice software testing out there (I can only really speak about the work I've seen at Google) is statistical in nature: you run a huge amount of tee-d or saved production traffic through a service, and run a massive diff of the output, after painstakingly making your service deterministic. By doing this, you can show that a "typical" query is unlikely to cause a "catastrophic" failure.