4 ms·

I think this article kind of misses the mark? There is a big gulf between reporting other explanations and tests for the phenomena you're studying and divulging

I think this article kind of misses the mark? There is a big gulf between reporting other explanations and tests for the phenomena you're studying and divulging information that you believe is dangerous. A journal of bad results would be incredible. You could have one for novel or interesting failures as the article posits, and you could have one that catalogued and categorized more mundane failures - if its not an interesting failure you probably don't need to publish all the particulars, just make your findings available for inspection. I think that kind of information condensed down could be very useful.

Fomite 5y ago
"Would anyone really care to subscribe to, let alone publish in, the American Journal of Discarded Hypotheses, the Annals of Failed Experiments, or PLOS Dumpster?" I have often half-joked about the Journal of Cautionary Tales, and would genuinely find "This particular study went sideways, and here's why..." quite valuable. And also, allowing researchers to still get "credit" for contributing to the knowledge of the doing of science.
- fxtentacle 5y ago
  Yes, but it needs a much more catchy name. Many people subscribe to "weekly videos of random stuff exploding" on YouTube.
- vkou 5y ago
  > "This particular study went sideways, and here's why..." You often can't tell if it went sideways because your hypothesis was shite, or your experiment was. And when you can guess that your experiment was, you often don't know why.
  - jononor 5y ago
    Sure, but that is a great reason to publish it. It allows other in the community to pick up on where things left off. Maybe someone else has the missing pieces to the puzzle.
  - Fomite 5y ago
    But you often can. All journals are selective. This would select for "We know why, and there's practical knowledge to be gained from the why."
- samatman 5y ago
  We've talked about founding the Journal of the Null Hypothesis, which publishes well-designed experiments which reveal a particular hypothesis to be wrong or insignificant.
- h2odragon 5y ago
  There's always RISKS digest: https://catless.ncl.ac.uk/Risks/ https://catless.ncl.ac.uk/Risks/
XorNot 5y ago
The problem with a journal of bad results is that it would only be useful if you kept meticulous notes while getting bad results. Which functionally turns them into good results which can be published normally, since you now have enough content to write something like "<Phenonmenon> not found under <conditions>" which is basically how you call out the original authors. The reality of when you can't reproduce something is usually some combination of the authors did not report something they considered minor, but which turned out to be important, and/or that you're just bad at it, but in both cases your actual research probably wasn't trying to study that phenomenon - you wanted to use it for something else. EDIT: I will say, what would be great would be if live-streaming your experiments became a thing (so really just recording them) at least in chemistry (where I worked). There would be immense value in being able to pull the recordings of someone getting the result they claim so you can see their whole technique, setup, lab and process - because that's where the important details creep in.
- cycomanic 5y ago
  The issue is that there already is so much being published that it is nearly impossible to keep up with in many fields. Any solution that includes more publications just misses the mark and is detrimental IMO. We will only get to better science if we move to a situation were publishing less is acceptable, because the only way we can detect dishonesty is if scientists actually have enough time to follow up on weird results.

Fomite 5y ago

"Would anyone really care to subscribe to, let alone publish in, the American Journal of Discarded Hypotheses, the Annals of Failed Experiments, or PLOS Dumpster?" I have often half-joked about the Journal of Cautionary Tales, and would genuinely find "This particular study went sideways, and here's why..." quite valuable. And also, allowing researchers to still get "credit" for contributing to the knowledge of the doing of science.

fxtentacle 5y ago
Yes, but it needs a much more catchy name. Many people subscribe to "weekly videos of random stuff exploding" on YouTube.
vkou 5y ago
> "This particular study went sideways, and here's why..." You often can't tell if it went sideways because your hypothesis was shite, or your experiment was. And when you can guess that your experiment was, you often don't know why.
- jononor 5y ago
  Sure, but that is a great reason to publish it. It allows other in the community to pick up on where things left off. Maybe someone else has the missing pieces to the puzzle.
- Fomite 5y ago
  But you often can. All journals are selective. This would select for "We know why, and there's practical knowledge to be gained from the why."
samatman 5y ago
We've talked about founding the Journal of the Null Hypothesis, which publishes well-designed experiments which reveal a particular hypothesis to be wrong or insignificant.
h2odragon 5y ago
There's always RISKS digest: https://catless.ncl.ac.uk/Risks/ https://catless.ncl.ac.uk/Risks/

XorNot 5y ago

The problem with a journal of bad results is that it would only be useful if you kept meticulous notes while getting bad results. Which functionally turns them into good results which can be published normally, since you now have enough content to write something like "<Phenonmenon> not found under <conditions>" which is basically how you call out the original authors. The reality of when you can't reproduce something is usually some combination of the authors did not report something they considered minor, but which turned out to be important, and/or that you're just bad at it, but in both cases your actual research probably wasn't trying to study that phenomenon - you wanted to use it for something else. EDIT: I will say, what would be great would be if live-streaming your experiments became a thing (so really just recording them) at least in chemistry (where I worked). There would be immense value in being able to pull the recordings of someone getting the result they claim so you can see their whole technique, setup, lab and process - because that's where the important details creep in.

cycomanic 5y ago
The issue is that there already is so much being published that it is nearly impossible to keep up with in many fields. Any solution that includes more publications just misses the mark and is detrimental IMO. We will only get to better science if we move to a situation were publishing less is acceptable, because the only way we can detect dishonesty is if scientists actually have enough time to follow up on weird results.