4 ms·

This can't be emphasized enough. For an experiment to be statistically valid, you have to run the experiment. Not part of the experiment. Not most of the experi

• equark 15y ago
  The problem is that this advice completely ignores the motivation for experimenting: optimal decision making. If you run a test that ends inconclusively should you really just throw up your hands? And if you run a test that's quickly conclusive, should you really avoid all the profit that could be gained from immediately exploiting this knowledge?
  • viraptor 15y ago
    Inconclusive result is a kind of result. You can test comparable designs all you want and get an inconclusive result for a long time. This means there's no big difference and that's that. If you get significant result at 0.0005, then it's up to you - might as well stop. There's even a table in the article saying what significance is appropriate after "correction".