6 ms·

"Why Most Published Research Findings Are False" http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124 http://www.plosmedicine.org/article/i

• carbocation 16y ago
  And the specific reason here (among the ones listed in that article) is almost certainly a lack of data. It's always helpful to estimate your necessary sample size prior to beginning your test, so you don't get too excited when results look odd before you get anywhere near the sample necessary to see an effect of a given size.
  • StavrosK 16y ago
    Sure, but the way you estimate your sample size is working backwards from the confidence level you want to get. In this case, we were waiting for the confidence metric to tell us when the sample size was appropriate, but it didn't work out as planned.
    • jfarmer 16y ago
      No, you're doing it wrong. You say, "How many people should I test before there's a 95% chance I observe a 10% effect size?" That's your sample size. It's easy to compute up front.
      • StavrosK 16y ago
        Hmm, yes, I see now why you're right. I'd like to read a bit more on the theory, though, so we don't make these mistakes again. I'll look around for some resources, thanks again!
        • carbocation 16y ago
          Terms to get you started are "power analysis" and "sample size estimation."