4 ms·

As a serious question, do papers typically get published if they simply disprove a promising idea? As an example, my friend is doing honours research right now

As a serious question, do papers typically get published if they simply disprove a promising idea? As an example, my friend is doing honours research right now, and her supervisor is very emphatic that she should do an experiment that will be 'successful' so she can publish it. My question is, doesn't performing an 'unsuccessful' experiment progress the state of the art as much as proving something? If a reasonable researcher might have an idea, and I prove it won't work, why wouldn't that be shared within the community as a negative result?

elemeno 14y ago
Medical Journals tend not to be interested in publishing 'negative' result papers. I can see why to an extent - they're not of particular value or interest to the vast majority of the journal's readers. They are however exceptionally valuable to anyone else running trials or doing research into similar drugs in the future, so there should be, IMO, somewhere where those papers can be made publically accesible.
- archgoon 14y ago
  "Journal of Plausible, but Ultimately Unworkable Hypotheses" Problem is, you don't want a high publication count in such a hypothetical journal. That's the other side of publication bias.
  - alanctgardner2 14y ago
    But we have computers! This would be final step of proposing a project; check that a negative, peer-reviewed version hasn't been published. Putting them in separate journals actually solves a lot of the complaints in this thread
    - tedunangst 14y ago
      You as a researcher do not want your name to appear in such a journal. Especially not lots of times.
    - archgoon 14y ago
      Sorry for the confusion, that wasn't my point. The issue isn't that there would be a ton of research to wade through, the issue is that scientists whose research didn't pan now have a track record of failure. I can easily imagine a academic position hiring conversation going like "This guy has a higher failure to success ratio than average, and his successful projects aren't above average." EDIT: But perhaps I'm being too cynical (must be the New Year). Such a journal has been constructed. http://www.jnrbm.com/ http://www.jnrbm.com/ With luck, it will be wildly (un)successful :).
eric_bullington 14y ago
No, and this problem even has a name: publication bias. It's a known issue in most modern peer-reviewed research. There are now multiple efforts underway to counteract its effects, such as a US regulation called FDAAA 801 and its resulting data registry clinicaltrials.gov.
munin 14y ago
yes. most likely, what the supervisor means is that the research should have some kind of story and conclusion. an 'unsuccessful' piece of research is one that does not say anything, i.e. "well we tried to replicate this other paper and you know what, we did" or "I had an idea and I thought really hard about it for about a year, but nothing really concrete ever presented itself in either theory or experimentation". a lot of people are kind of grumpy about null hypotheses not being published, for example papers of the form "we tried X and weren't able to make it work, here is how we tried". that is an 'unsuccessful' result (maybe you just suck), but some people would like to be able to read through stories of failure to see if there is a way to be successful. the journal of the null hypothesis (http://www.jasnh.com/ http://www.jasnh.com/) is an effort to do something like this, but it's generally the punchline of jokes...
carbocation 14y ago
There is something of a sea change going on right now (perhaps it's still too early to use that term, but the tide is surely shifting). Let's distinguish two types of "negative" studies: one is a study whose result simply failed to be positive. "We performed exome sequencing in a kindred but failed to identify a common mutation in all affected members of the family." There are so many possible error modes, including erroneous fundamental assumptions about where this family's defect lies, that this is possibly uninteresting to virtually anyone. The second type is a study that was done in a way that was powered to detect a positive result, had it been there. "We performed a randomized controlled trial in 1,000 patients with hemochromatosis to determine whether iron chelation was non-inferior to bloodletting for the treatment of hemochromatosis. We found that iron chelation failed to achieve non-inferiority." In this example (which is made up), this is a negative result that teaches us something immediately, and which could change practice (or could tell everyone that practice is not about to change despite rumors of a potentially new therapy). Along these lines, to prevent these valuable yet "negative" trials from going unpublished, high-end medical journals such as the New England Journal of Medicine are now requiring that all trials enter into a registry.

elemeno 14y ago

Medical Journals tend not to be interested in publishing 'negative' result papers. I can see why to an extent - they're not of particular value or interest to the vast majority of the journal's readers. They are however exceptionally valuable to anyone else running trials or doing research into similar drugs in the future, so there should be, IMO, somewhere where those papers can be made publically accesible.

archgoon 14y ago
"Journal of Plausible, but Ultimately Unworkable Hypotheses" Problem is, you don't want a high publication count in such a hypothetical journal. That's the other side of publication bias.
- alanctgardner2 14y ago
  But we have computers! This would be final step of proposing a project; check that a negative, peer-reviewed version hasn't been published. Putting them in separate journals actually solves a lot of the complaints in this thread
  - tedunangst 14y ago
    You as a researcher do not want your name to appear in such a journal. Especially not lots of times.
  - archgoon 14y ago
    Sorry for the confusion, that wasn't my point. The issue isn't that there would be a ton of research to wade through, the issue is that scientists whose research didn't pan now have a track record of failure. I can easily imagine a academic position hiring conversation going like "This guy has a higher failure to success ratio than average, and his successful projects aren't above average." EDIT: But perhaps I'm being too cynical (must be the New Year). Such a journal has been constructed. http://www.jnrbm.com/ http://www.jnrbm.com/ With luck, it will be wildly (un)successful :).

eric_bullington 14y ago

No, and this problem even has a name: publication bias. It's a known issue in most modern peer-reviewed research. There are now multiple efforts underway to counteract its effects, such as a US regulation called FDAAA 801 and its resulting data registry clinicaltrials.gov.

munin 14y ago

yes. most likely, what the supervisor means is that the research should have some kind of story and conclusion. an 'unsuccessful' piece of research is one that does not say anything, i.e. "well we tried to replicate this other paper and you know what, we did" or "I had an idea and I thought really hard about it for about a year, but nothing really concrete ever presented itself in either theory or experimentation". a lot of people are kind of grumpy about null hypotheses not being published, for example papers of the form "we tried X and weren't able to make it work, here is how we tried". that is an 'unsuccessful' result (maybe you just suck), but some people would like to be able to read through stories of failure to see if there is a way to be successful. the journal of the null hypothesis (http://www.jasnh.com/ http://www.jasnh.com/) is an effort to do something like this, but it's generally the punchline of jokes...

carbocation 14y ago

There is something of a sea change going on right now (perhaps it's still too early to use that term, but the tide is surely shifting). Let's distinguish two types of "negative" studies: one is a study whose result simply failed to be positive. "We performed exome sequencing in a kindred but failed to identify a common mutation in all affected members of the family." There are so many possible error modes, including erroneous fundamental assumptions about where this family's defect lies, that this is possibly uninteresting to virtually anyone. The second type is a study that was done in a way that was powered to detect a positive result, had it been there. "We performed a randomized controlled trial in 1,000 patients with hemochromatosis to determine whether iron chelation was non-inferior to bloodletting for the treatment of hemochromatosis. We found that iron chelation failed to achieve non-inferiority." In this example (which is made up), this is a negative result that teaches us something immediately, and which could change practice (or could tell everyone that practice is not about to change despite rumors of a potentially new therapy). Along these lines, to prevent these valuable yet "negative" trials from going unpublished, high-end medical journals such as the New England Journal of Medicine are now requiring that all trials enter into a registry.