4 ms·
If you design an experiment in such a way that outcome A can be explained away by "yes, but the design is flawed", while outcome B cannot, I don't think it is f
by Someone 13y ago
If you design an experiment in such a way that outcome A can be explained away by "yes, but the design is flawed", while outcome B cannot, I don't think it is fair to do the experiment.
- gwern 13y agoExperiments don't have to be 'fair'; your 'fairness' is not required for tests to be informative. Imagine an IQ test which spits out a binary yes/no result based on whether you're in the top 10% or not. This is a useful, meaningful test, which is 'unfair' in your naive sense. Issues of information content, discrimination, false positives and negatives - it's more complex than saying 'it is fair to do the experiment'. In this case, I got a reliable negative, rather than a less reliable positive.