4 ms·

i've said it before and i'll say it again, this isn't going to improve until there are more grant dollars and recognition for replication than there is for nove

• glial 5y ago
  100% agree. The NIH could replicating studies a requirement of funding for graduate or postdoctoral training. In addition to the scientific and societal benefit, it would just be good training.
  • a-dub 5y ago
    your first task in lab will be to reproduce our competitor's work! in seriousness though, there's enough money and trained scientists looking for work... moreover i think there's a whole personality type that is prevalent in science that would be very happy to replicate and reproduce results...
  • tsbischof 5y ago
    I would be glad to have funding explicitly set towards appropriate and open documentation of the actual protocols, equipment, and materials, plus a combined effort to enhance these reporting requirements by the journal. There has to be a balance between the one-off interesting new findings and the large-scale work, but clinical trials definitely fall under the latter category.
• bmitc 5y ago
  That sounds like spending money on trying to replicate results that can’t be replicated doesn’t it? Isn’t that what this study was doing? Doesn’t it make more sense to revisit how the existing funding is spent? I have very little insight into this area of research, so I’d just have to guess everyone’s in a hurry and reaching for results or published papers.
  • a-dub 5y ago
    > Doesn’t it make more sense to revisit how the existing funding is spent? i don't know what you'd find in a financial audit that a scientific audit wouldn't tell you, but a passing financial audit won't tell you if a scientific finding actually sticks. therefore, seems like a better deal to audit the results, no?
    • Eelongate 5y ago
      > "i don't know what you'd find in a financial audit that a scientific audit wouldn't tell you" Financial audits would uncover financial fraud. > "seems like a better deal to audit the results, no?" Better to audit both.
      • a-dub 5y ago
        i suppose. i suspect most financial fraud would be caught by the accountants and grant administrators at respective institutions. accounting and best practices around it have been around for a long time. this discussion is about ensuring scientific velocity by increasing the rigor required to declare success and add new knowledge to the commons. unless there's large scale fraud going on, i honestly don't think bringing in additional auditors to look at books that are already professionally kept is going to benefit anyone other than said auditors.
        • Eelongate 5y ago
          > i suspect most financial fraud would be caught by the accountants and grant administrators at respective institutions. If it was only their money on the line, that would probably suffice. But when researchers (and by extension the institutions they work at) are receiving government grants, I think there need to be independent audits.
        • a-dub 5y ago
          > If it was only their money on the line, that would probably suffice. But when researchers (and by extension the institutions they work at) are receiving government grants, I think there need to be independent audits. these are all nonprofit institutions, some are government run, i'm pretty sure they're required to engage independent professional auditing firms just as a condition of their tax exempt status, let alone acceptance of federal grant dollars.
• Eelongate 5y ago
  It won't change until a track record of producing studies that fail to replicate hurts the career of an academic more than not publishing anything at all. As long as publish-or-perish provides a stronger incentive than the negative repercussions of publishing nonsense, this will continue.
  • hermitdev 5y ago
    I don't know if one necessarily wants to punish for failure to replicate, but I do think that falsified data/records should lead to a death sentence of a scientific career or result in criminal charges if necessary (e.g. for defrauding the associated institutions/grantor). I guess sort of like what Elizabeth Holmes is facing currently.
• refurb 5y ago
  This is a self-resolving problem. - breakthrough study is published - because it’s breakthrough, it attracts attention and other labs look at it (we often tried to repeat breakthrough work in our lab) - if it can’t be replicated, interest just peters out OR it drives someone else to “fix” it. If it’s so important people need to know it gets published. - if it can be replicated further research continues
  • xiphias2 5y ago
    The problem is with step 3: incredible amount of energy goes into building upon a bad study because the fact that it's not retracted.
    • refurb 5y ago
      95% of science is doing stuff that doesn’t work. Replication - independent replication - is not a wasted effort.