>Social psychology experiments in the laboratory tend to throw up spectacular mind-boggling effects. Many of these fail to replicate and are later discredited.

>Social psychology experiments in the laboratory tend to throw up spectacular mind-boggling effects. Many of these fail to replicate and are later discredited. The ones that do replicate are not always generalizable – sometimes an even slightly different situation will remove the effect or create exactly the opposite effect. Generator-based tests (as in Haskell's Quickcheck) could help here. Standard unit tests run assertions with hardcoded input values such as [0, 1, -1]. Generator based tests instead try to show that properties hold over randomly generated inputs, by replacing 1 with a generator of positive integers, -1 with a generator of negative integers, etc. Imagine if social psychologists ran generator-based studies as a hedge against unknown unknowns. The biggest challenge would be generating and executing valid permutations of the original experiment. Perhaps ad network AB-testing could be repurposed to run experiments as ads with randomly generated values.