3 ms·

I don't entirely understand why the pressure to publish at all costs is the issue here. The article says: > This particular plot is a statistical smoking gun,

I don't entirely understand why the pressure to publish at all costs is the issue here. The article says: > This particular plot is a statistical smoking gun, and suggests that the positive results from the original studies (black dots) were probably the result of p-hacking. They were chance findings, selectively published because they were positive. From that, it seems to me like the problem isn't what the researchers published, it's what they didn't publish. I'm a programmer, not a researcher, but in college chemistry we had marble notebooks that we wrote everything down in. The notebooks made it obvious if a page was torn out, everything had to be written in pen, and if we crossed something out, we had to cross it out in such a way that it was still legible. If we didn't include data in our results, we were expected to explain why. The most valuable thing I learned in those classes was the importance and practice of auditability. I doubt that my classes were unique in this respect and I wonder if this might be the solution to the problem. I've long been a proponent of people simply publishing literally all their data now that internet hosting is so cheap, and only publishing syntheses of the data in journals, giving anyone the opportunity to verify the author's conclusions. Psychology is different due to the need for confidentiality for the people being studied, but the full data should be available to at least the peer reviewers before a study can be published. EDIT: To add to this, elsewhere on this thread there's a lot of discussion about people not publishing negative results because they're not interesting. But reality doesn't have a bias towards results that are interesting, only humans do. I don't know how to solve this problem, but at least making publication of your non-interesting results a condition of publishing your interesting results would be a start.

danieltillett 11y ago
Not publishing negative results is a symptom, not a cause, of the current extreme competition in science. Fix the competition issue and we will fix problems like the non-publication of negative results.
- iamsohungry 11y ago
  I'm not sure I see the causal relationship here. Why does increased competition mean that people are not publishing? I think the problem is that the metrics which academics are competing to maximize are poor. If your metric is "How much interesting science has this person published?" you're going to get very different results from "How much accurate science has this person published?" Reducing competition doesn't change which question is being asked, it just reduces the importance of any metric at all. It seems likely to me that removing competition would give no incentive to publish anything measurable at all; after all, why publish if your career isn't connected to it? Obviously some would publish for intrinsic reasons, and it's possible that stress levels due to competition are leading to shoddy work, but none of this amounts to an argument that less competition would lead to people publishing negative results.

danieltillett 11y ago

Not publishing negative results is a symptom, not a cause, of the current extreme competition in science. Fix the competition issue and we will fix problems like the non-publication of negative results.

iamsohungry 11y ago
I'm not sure I see the causal relationship here. Why does increased competition mean that people are not publishing? I think the problem is that the metrics which academics are competing to maximize are poor. If your metric is "How much interesting science has this person published?" you're going to get very different results from "How much accurate science has this person published?" Reducing competition doesn't change which question is being asked, it just reduces the importance of any metric at all. It seems likely to me that removing competition would give no incentive to publish anything measurable at all; after all, why publish if your career isn't connected to it? Obviously some would publish for intrinsic reasons, and it's possible that stress levels due to competition are leading to shoddy work, but none of this amounts to an argument that less competition would lead to people publishing negative results.