2 ms·

Sorry, which parts of the proposal are you responding to? The article is more specific than "more rigor". Are you objecting to the higher p-value threshold?

Sorry, which parts of the proposal are you responding to? The article is more specific than "more rigor". Are you objecting to the higher p-value threshold? The independent confirmation? The author argues that a single higher quality confirmatory experiment will be able to replace gathering lots of statistics for exploratory experiments: > Unlike clinical studies, most preclinical research papers describe a long chain of experiments, all incrementally building support for the same hypothesis. Such papers often include more than a dozen separate in vitro and animal experiments, with each one required to reach statistical significance. We argue that, as long as there is a final, impeccable study that confirms the hypothesis, the earlier experiments in this chain do not need to be held to the same rigid statistical standard. Do you disagree?

untilHellbanned 10y ago
> one that incorporates an independent, statistically rigorous confirmation of a researcher's central hypothesis. We call this large confirmatory study a preclinical trial. These would be more formal and rigorous than the typical preclinical testing conducted in academic labs, and would adopt many practices of a clinical trial. As you can see above and from your quotation (and like many other folks who come in to save the day), this article is heavy on plans and short on who is going to do the work. Of course I support papers where every single experiment doesn't have to play p < 0.05 games, but other parts of the article wander in other directions. That's all I'm reacting to.
- jessriedel 10y ago
  Having a higher threshold for publication can be imposed without detailing who does what work. You might argue that this means less research will get produced, but it's probably worth it. Practitioners underestimate the difficulty of transferring knowledge to outsiders because of frictions due to trust, clarity, and tacit knowledge. http://blog.givewell.org/2016/01/19/the-importance-of-gold-standard-studies-for-consumers-of-research/ http://blog.givewell.org/2016/01/19/the-importance-of-gold-s...
  - mattkrause 10y ago
    Eh... When you call for everyone to scale their experiments up by sixfold, I think you also need to consider the logistics of doing that. I'm totally in favor of better, more rigorous experiments, but I know that we couldn't afford the time, space, or gear needed to do that right now.

untilHellbanned 10y ago

> one that incorporates an independent, statistically rigorous confirmation of a researcher's central hypothesis. We call this large confirmatory study a preclinical trial. These would be more formal and rigorous than the typical preclinical testing conducted in academic labs, and would adopt many practices of a clinical trial. As you can see above and from your quotation (and like many other folks who come in to save the day), this article is heavy on plans and short on who is going to do the work. Of course I support papers where every single experiment doesn't have to play p < 0.05 games, but other parts of the article wander in other directions. That's all I'm reacting to.

jessriedel 10y ago
Having a higher threshold for publication can be imposed without detailing who does what work. You might argue that this means less research will get produced, but it's probably worth it. Practitioners underestimate the difficulty of transferring knowledge to outsiders because of frictions due to trust, clarity, and tacit knowledge. http://blog.givewell.org/2016/01/19/the-importance-of-gold-standard-studies-for-consumers-of-research/ http://blog.givewell.org/2016/01/19/the-importance-of-gold-s...
- mattkrause 10y ago
  Eh... When you call for everyone to scale their experiments up by sixfold, I think you also need to consider the logistics of doing that. I'm totally in favor of better, more rigorous experiments, but I know that we couldn't afford the time, space, or gear needed to do that right now.