6 ms·

I’m not sure “refutation” is the right word. > “We still believe that there is value in exploring how physiological reactions and conscious experience shape po

I’m not sure “refutation” is the right word. > “We still believe that there is value in exploring how physiological reactions and conscious experience shape political attitudes and behavior, but after further consideration, we have concluded that any such relationships are more complicated than we (and the researchers on the Science paper) presumed.“ That sounds more like a clarification or a letter to the editor. I don’t disagree that nuance is important, but I assume Nature believes they can get more nuance out of visiting new studies than reinterpreting old ones.

closed 7y ago
They failed to replicate the study over multiple attempts, 1 with four times the sample size. Using the time reversal heuristic, if we flip the order, so the replication came before the original, how would you feel about Science publishing a paper with a positive result, when multiple studies--with larger samples and the same materials--had not obtained the result? If you think the original should still be published in Science in that case, why? Since the original is published and highly visible, it's dangerous that Science won't correct the highly publicized, unlikely-true result.
- pmiller2 7y ago
  > ...publishing a paper with a positive result, when multiple studies--with larger samples and the same materials--had not obtained the result? That happens all the time. Papers with null results tend to either sit in drawers unpublished, or never get written at all. Nobody knows about the multiple studies because nobody will publish them.
  - roca 7y ago
    And that is a huge problem. Are you arguing that because null results are rarely published, we should publish them even less?
    - pmiller2 7y ago
      Where are you possibly getting that from?
      - Kim_Bruning 7y ago
        We agree that currently it happens all the time. Some number of people (including eg. Roca) say that this is a problem - a "replication crisis" if you will. Those people say that perhaps papers with null results should be published. Perhaps -they say- publication of null results could be (one possible) (part of) the solution. Your reply sounded like you don't completely agree? Is that true, or did Roca read you wrong? Could you clarify?
        pmiller2 7y ago
        I agree that null results should be published. I was making the point that some otherwise reasonable people disagree, for career or other reasons. If I were a researcher, I probably would not attempt to get a null result published; I’d work on something else that actually had a chance of going into my tenure file.

closed 7y ago

They failed to replicate the study over multiple attempts, 1 with four times the sample size. Using the time reversal heuristic, if we flip the order, so the replication came before the original, how would you feel about Science publishing a paper with a positive result, when multiple studies--with larger samples and the same materials--had not obtained the result? If you think the original should still be published in Science in that case, why? Since the original is published and highly visible, it's dangerous that Science won't correct the highly publicized, unlikely-true result.

pmiller2 7y ago
> ...publishing a paper with a positive result, when multiple studies--with larger samples and the same materials--had not obtained the result? That happens all the time. Papers with null results tend to either sit in drawers unpublished, or never get written at all. Nobody knows about the multiple studies because nobody will publish them.
- roca 7y ago
  And that is a huge problem. Are you arguing that because null results are rarely published, we should publish them even less?
  - pmiller2 7y ago
    Where are you possibly getting that from?
    - Kim_Bruning 7y ago
      We agree that currently it happens all the time. Some number of people (including eg. Roca) say that this is a problem - a "replication crisis" if you will. Those people say that perhaps papers with null results should be published. Perhaps -they say- publication of null results could be (one possible) (part of) the solution. Your reply sounded like you don't completely agree? Is that true, or did Roca read you wrong? Could you clarify?
      - pmiller2 7y ago
        I agree that null results should be published. I was making the point that some otherwise reasonable people disagree, for career or other reasons. If I were a researcher, I probably would not attempt to get a null result published; I’d work on something else that actually had a chance of going into my tenure file.