3 ms·

What if to each publication of a non-null result, academics are given the opportunity to publish their nulls as well, if only as a appendix or better, a counter

What if to each publication of a non-null result, academics are given the opportunity to publish their nulls as well, if only as a appendix or better, a counterpublication to their main conclusions? I don't buy the argument that papers need be of max-n length, now that documents and journals can be easily stored and distributed. I would love something like Living Papers [0][1] to take off, where the null an non-non results could be compared interactively on similar footing. [0]: https://github.com/uwdata/living-papers https://github.com/uwdata/living-papers [1]: https://idl.uw.edu/living-papers-template/ https://idl.uw.edu/living-papers-template/

bluGill 2y ago
A null result may be a dead end and so there is no related paper worth publishing it in. A null result should be published right away in a searchable place, but probably isn't worth a lot of effort in general. I tried X, it didn't work, here is the raw data.
- dleeftink 2y ago
  That's my thought exactly--not a related paper but simply providing additional room for discussing the less shiny bits of the same experiment. Even if the whole thing is a null, the setup, instruments, dependencies and what methods worked/didn't work is worth describing by itself.
  - acchow 2y ago
    All of that - the setup, the instruments, dependencies, methods - should be pre-submitted to the journal before the experimental results arrive. The journal should be the one that uses the data from the experiment and runs your pre-submitted program over the data to produce a result. Papers need to be published backwards.
    - bluGill 2y ago
      Right now you don't even know who will publish you paper until all that is done. Your experiment might be try some promising molecule/drug in a petri dish, and see what happens, if the results are amazing you will get in a different journal than if the results are something happens but the control molecule/drug is better.
      - acchow 2y ago
        Right, I'm saying journals should be picking which to publish before the results arrive. They should even publish it before the results arrive. And then publish again after.
    - bumby 2y ago
      I agree that in an idealized way, this would be much better. But what do you do about going through all this process and ending up with a bad reviewer?* In those cases, how would you handle re-submitting to a different journal without looking like you're creating those artifacts after-the-fact to suit your outcome? Would the pre-submittals need to be handled by some third party? * the current process still has a lot of luck in terms of getting assigned referees. Sometimes you just plain get a bad reviewer who just can't be bothered to read the submission carefully and is quick to reject it. I would hate to see a system that only allows for a single shot at publication
      - naasking 2y ago
        The review happens at the experimental design stage, not the publishing stage. Very little actual work has been done at that stage. After such a review, the journal agrees to publish your results regardless of outcome.

bluGill 2y ago

A null result may be a dead end and so there is no related paper worth publishing it in. A null result should be published right away in a searchable place, but probably isn't worth a lot of effort in general. I tried X, it didn't work, here is the raw data.

dleeftink 2y ago
That's my thought exactly--not a related paper but simply providing additional room for discussing the less shiny bits of the same experiment. Even if the whole thing is a null, the setup, instruments, dependencies and what methods worked/didn't work is worth describing by itself.
- acchow 2y ago
  All of that - the setup, the instruments, dependencies, methods - should be pre-submitted to the journal before the experimental results arrive. The journal should be the one that uses the data from the experiment and runs your pre-submitted program over the data to produce a result. Papers need to be published backwards.
  - bluGill 2y ago
    Right now you don't even know who will publish you paper until all that is done. Your experiment might be try some promising molecule/drug in a petri dish, and see what happens, if the results are amazing you will get in a different journal than if the results are something happens but the control molecule/drug is better.
    - acchow 2y ago
      Right, I'm saying journals should be picking which to publish before the results arrive. They should even publish it before the results arrive. And then publish again after.
  - bumby 2y ago
    I agree that in an idealized way, this would be much better. But what do you do about going through all this process and ending up with a bad reviewer?* In those cases, how would you handle re-submitting to a different journal without looking like you're creating those artifacts after-the-fact to suit your outcome? Would the pre-submittals need to be handled by some third party? * the current process still has a lot of luck in terms of getting assigned referees. Sometimes you just plain get a bad reviewer who just can't be bothered to read the submission carefully and is quick to reject it. I would hate to see a system that only allows for a single shot at publication
    - naasking 2y ago
      The review happens at the experimental design stage, not the publishing stage. Very little actual work has been done at that stage. After such a review, the journal agrees to publish your results regardless of outcome.