4 ms·

> What if this was in place when someone performed the first open heart surgeries? I'm unclear if you're imagining here a scholarly article about a heart surge

> What if this was in place when someone performed the first open heart surgeries? I'm unclear if you're imagining here a scholarly article about a heart surgery, or a press release from the university hospital saying it was done. Research happens all the time without formal publication - it makes sense that there are different standards. I am proposing that there it would be beneficial for science to have journals with a modestly higher standard.

jrumbut 6y ago
The idea is that the first open heart surgury was truly innovative, but how would you describe that method for replication? If someone agreed to do a peer review they're not gonna test out brand new extremely dangerous surgical techniques, but the case studies and outcome data ought to be published as soon as possible so it can be evaluated. This type of higher standard places a heavier burden on the most innovative science. Meanwhile simple surveys or small nudge interventions are easy to replicate.
- deleted 6y ago
  [deleted]
- learnstats2 6y ago
  Given that an open heart surgery is a surgery, and not a published statistical hypothesis, I don't find it to be within the scope of this discussion. But, if you developed a new surgical technique today, and other surgeons were unable to reproduce that technique afterward, that would surely NOT be considered an innovation in surgery.
  - jrumbut 6y ago
    I mentioned in the original comment, open heart surgery at the time that it was a novel technique. The statistical hypothesis is "this new surgical intervention is better than the current standard of care." Instructions in a single paper publishing the results of the pilot cases will not be sufficient for another surgeon to try to replicate. A replication would take perhaps years of training and a massive budget to achieve, meanwhile the field is in the dark about the development because this is all pre-publication. If the heart surgery example isn't working for you just imagine something else that takes years of training and millions of dollars to set up, plenty of those things exist in medicine and elsewhere in science. What I am getting at is that many innovations can not be readily replicated by many or any others (in the early days after it is discovered or invented) or it would be massively expensive to do so and hard to justify when no results have been published and publicly discussed. Meanwhile simple, unambitious interventions are very easy to replicate. This proposed standard increases the burden on researchers in proportion to how truly innovative their research is, thereby incentivizing simple, easy to replicate research. It's possible it would be a good standard in some fields, but would be counterproductive in others. I believe the author should consider this and be more specific about where it should be applied.
  - jrumbut 6y ago
    Oh, I have a better example! A longitudinal trial that follows up with participants for 10+ years.

jrumbut 6y ago

The idea is that the first open heart surgury was truly innovative, but how would you describe that method for replication? If someone agreed to do a peer review they're not gonna test out brand new extremely dangerous surgical techniques, but the case studies and outcome data ought to be published as soon as possible so it can be evaluated. This type of higher standard places a heavier burden on the most innovative science. Meanwhile simple surveys or small nudge interventions are easy to replicate.

deleted 6y ago
[deleted]
learnstats2 6y ago
Given that an open heart surgery is a surgery, and not a published statistical hypothesis, I don't find it to be within the scope of this discussion. But, if you developed a new surgical technique today, and other surgeons were unable to reproduce that technique afterward, that would surely NOT be considered an innovation in surgery.
- jrumbut 6y ago
  I mentioned in the original comment, open heart surgery at the time that it was a novel technique. The statistical hypothesis is "this new surgical intervention is better than the current standard of care." Instructions in a single paper publishing the results of the pilot cases will not be sufficient for another surgeon to try to replicate. A replication would take perhaps years of training and a massive budget to achieve, meanwhile the field is in the dark about the development because this is all pre-publication. If the heart surgery example isn't working for you just imagine something else that takes years of training and millions of dollars to set up, plenty of those things exist in medicine and elsewhere in science. What I am getting at is that many innovations can not be readily replicated by many or any others (in the early days after it is discovered or invented) or it would be massively expensive to do so and hard to justify when no results have been published and publicly discussed. Meanwhile simple, unambitious interventions are very easy to replicate. This proposed standard increases the burden on researchers in proportion to how truly innovative their research is, thereby incentivizing simple, easy to replicate research. It's possible it would be a good standard in some fields, but would be counterproductive in others. I believe the author should consider this and be more specific about where it should be applied.
- jrumbut 6y ago
  Oh, I have a better example! A longitudinal trial that follows up with participants for 10+ years.