3 ms·
There are a few things to know about this. First, the framework you probably want to think about this is Parnas's Trace Assertion Method. It gives you the tool
by madhadron 6y ago
There are a few things to know about this.
First, the framework you probably want to think about this is Parnas's Trace Assertion Method. It gives you the tools to think about the problem without getting caught in particular details.
There was a paper that looked at serious failures in distributed systems, and found that most of them were caused by a small number of factors. So you get a lot of power without going into all possible sequences. I am completely failing to find it, but it's well known so hopefully someone whose memory is doing better than mine can post it. Or if I remember, I'll come back and add a comment.
I like to think of it as separating the input space into boundaries and bulk (I've described it further here: https://madhadron.com/posts/easier_faster_testing.html https://madhadron.com/posts/easier_faster_testing.html ). No actions, empty lists, lists with one element, two actions of different kinds --- these are all the boundary of the input space. For the kind of systems we write as programmers, behavior tends to be fairly uniform once you're into the bulk, so you sample it and then closely test the boundary where stuff changes a lot. Now, you may have boundaries in the middle of the bulk as well (and in numerical work, the boundaries can move around a lot if you don't have a good mathematical understanding of the system), but it is still usually possible to test the various regions of bulk and get good coverage on the boundaries.
QuickCheck and other property based testing libraries have been used for this. The Erlang QuickCheck library was used for testing phone handsets by throwing random event streams at them until they crashed. It's a very powerful tool for this purpose.