2 ms·
FTA: Sometimes, for the most general tests, the only invariant you can assert is that the program doesn't crash. One of the most valuable tests we ever wrote at
by Someone 2y ago
FTA: Sometimes, for the most general tests, the only invariant you can assert is that the program doesn't crash. One of the most valuable tests we ever wrote at Row Zero is the one that just takes the top-level spreadsheet interface, and takes completely random actions on it
That strategy was also part of testing the original Mac. See https://www.folklore.org/Monkey_Lives.html https://www.folklore.org/Monkey_Lives.html.
As that article hunts at (“usually it didn't run for more than 20 minutes, even if it didn't crash, because the Monkey would invariably select the quit command”), there’s a black art in determining what “completely random” means.
You probably want the ability to change that depending on your testing needs. For example, if you’ve been working on the code for array formulas, the randomizer better create and edit them fairly frequently.
- gamegoblin 2y agoWe actually have two versions of this test. One does 100% totally random actions. The other tries to simulate an actual user (i.e. maintains the state of their cursor, which sheet they are looking at, etc), but still quite random. So the former finds crazy obscure edges, but the combinatorics make it less likely to hit any particular combination. The latter is the opposite. They pair really well.