3 ms·

For me the take away is that non-trivial independent replication of results still stands as the gold standard for experimentation. P-hacking shows up clearly wh

by NyxWulf 11y ago

For me the take away is that non-trivial independent replication of results still stands as the gold standard for experimentation. P-hacking shows up clearly when the experiment is replicated. P-values in my experience are only a starting point for further experiments. Take the cold-fusion debacle. When independent groups attempted to replicate the results they were unable to do so. Of course the other big take away is that people really don't understand statistics at a basic level, even the "scientific reporters".

digi_owl 11y ago
In essence "science" reporters have made it their business to not understand statistics. That way they can turn weak correlations into strong causations, rake in the eyeballs, and then go "oops" whenever someone points out problems.
mcguire 11y ago
Replication is hard and expensive, particularly in the medical field, and does not get the researcher any real science points. Besides, it's more fun to do a meta-analysis on a flock of sorta-vaguely-similar studies.
hootener 11y ago
> For me the take away is that non-trivial independent replication of results still stands as the gold standard for experimentation. Agreed. Too bad funding agencies rarely if ever give you the money to do it :(. The takeaway here is that statistics is arguably one of the most nuanced quantitative fields out there. It's really easy to shoot yourself in the foot, particularly with p-values. I think every statistical test has its place, but my personal favorite lambasting of p-value testing is Steiger and Fouladi's 1997 paper on non-centrality interval estimation [1]. As an aside, Steiger was my graduate statistics professor several years ago, and probably the primary reason I know this paper even exists. If you enjoy the harsh treatment of significance testing in the paper, just imagine hearing it straight from the horse's mouth during lecture :). [1]: http://www.statpower.net/Steiger%20Biblio/Steiger&Fouladi97.PDF http://www.statpower.net/Steiger%20Biblio/Steiger&Fouladi97....

by NyxWulf 11y ago

digi_owl 11y ago

In essence "science" reporters have made it their business to not understand statistics. That way they can turn weak correlations into strong causations, rake in the eyeballs, and then go "oops" whenever someone points out problems.

mcguire 11y ago

Replication is hard and expensive, particularly in the medical field, and does not get the researcher any real science points. Besides, it's more fun to do a meta-analysis on a flock of sorta-vaguely-similar studies.

hootener 11y ago

> For me the take away is that non-trivial independent replication of results still stands as the gold standard for experimentation. Agreed. Too bad funding agencies rarely if ever give you the money to do it :(. The takeaway here is that statistics is arguably one of the most nuanced quantitative fields out there. It's really easy to shoot yourself in the foot, particularly with p-values. I think every statistical test has its place, but my personal favorite lambasting of p-value testing is Steiger and Fouladi's 1997 paper on non-centrality interval estimation [1]. As an aside, Steiger was my graduate statistics professor several years ago, and probably the primary reason I know this paper even exists. If you enjoy the harsh treatment of significance testing in the paper, just imagine hearing it straight from the horse's mouth during lecture :). [1]: http://www.statpower.net/Steiger%20Biblio/Steiger&Fouladi97.PDF http://www.statpower.net/Steiger%20Biblio/Steiger&Fouladi97....