3 ms·
> It is nowhere near significance. Stopping it is completely wrong under the procedure we're using, we'll have no idea whether we're making the right decision!
by msellout 14y ago
> It is nowhere near significance. Stopping it is completely wrong under the procedure we're using, we'll have no idea whether we're making the right decision!
I don't know anyone who runs an experiment until they find "significance". That's comparable to only reporting outliers. What if the variable you are testing is simply not causal? That's why you pick the sample size before running the experiment, based on the minimum causal effect you want to detect.
- btilly 14y agoPeople don't as a practical matter because they simply can't. But a lot of people just haphazardly give up, and have little sense of what kinds of errors they may or may not be making. (But they hope that the errors are not too big.) The main point of this article is to give people a sense of what a more organized "giving up" curve could look like. And also to give them some much more concrete information about what their potential for serious error is.