4 ms·

Isn't that always the case?

pessimizer 4y ago
I think some people careen from fully trusting one thing to fully trusting the opposite thing. If you're not one of those people, you'll never understand dismissing things on the basis of "you should be critical of this, because not everything that people say is true." I do feel like that, even though being critical is something we should always do, that in cases where 1) the only reason you started paying attention to something was an intuitive hunch that it could matter, and 2) the only reason you started treating that hunch as established science is because you did experiments that had significant results, then 3) later you found that significance could be entirely accounted for by the file-drawer effect, you need to adjust your expectation that there actually is an effect to lower than your expectation was at step 1). It isn't that the theory hasn't been tested (although you can argue it hasn't been tested for ingeniously enough yet), it's that it has been tested and no effect has been shown. If you allow the existence of interest in a theory (represented in amount of ink spilled and number of experiments done) to raise your expectation that the theory is true, despite experimental indications to the contrary, you're not really doing science, you're just throwing good money after bad, probably motivated by a desire to protect the researchers and institutions that are heavily committed to the truth of the theory and/or the desire to protect other theories that depend on the one that hasn't shown results.
- civilized 4y ago
  There's a lot of junk behavioral science out there, but things like "people often go with a default or recommended option so they can move on with their day" seem so obvious to me that I become suspicious of this debunking for debunking too much.
  - pessimizer 4y ago
    That's just refusing to be convinced by evidence, though. It's good to have hunches, but it's good to let them go after you've done the experiments. Come up with a new hunch and a new experiment that shows why the expected effects weren't seen, and you're right back in there.
    - civilized 4y ago
      "Refusing to be convinced by evidence" is a simplistic false dichotomy. The evidence is interesting but I have several reasons not to immediately take it as definitive. Are you really certain that a big debunking in PNAS, surfing a wave of other celebrated debunkings, should be taken as definitive, when a good deal of the research being debunked was published to similar fanfare in PNAS back when a different kind of research was fashionable? I take neither the original research nor the debunking as particularly credible. Without technical expertise, I'm left to educated guess. It's just my guess.
      - pessimizer 4y ago
        > I have several reasons not to immediately take it as definitive. The one that you expressed is that it "seems so obvious to you that you become suspicious." I'm just taking you at your word.
        civilized 4y ago
        The idea that we should always be convinced by evidence regardless of context is a vast overgeneralization, impossible ("the evidence" overall rarely points only one way, even if the latest chunk of new evidence does), and in contradiction with Bayesian epistemology.
        pessimizer 4y ago
        My problem isn't that I think people should be credulous of everything, it's that I don't think "it's just obvious" is a proper counter to experiments that show nothing. If the effect is so obvious, it should be obvious how to design an experiment that would show it. I don't even know what you're defending here other than believing your first impulse above any subsequent evidence. Nobody is preventing anyone from proving an effect, in fact they poured money into the attempt.

I think some people careen from fully trusting one thing to fully trusting the opposite thing. If you're not one of those people, you'll never understand dismissing things on the basis of "you should be critical of this, because not everything that people say is true." I do feel like that, even though being critical is something we should always do, that in cases where 1) the only reason you started paying attention to something was an intuitive hunch that it could matter, and 2) the only reason you started treating that hunch as established science is because you did experiments that had significant results, then 3) later you found that significance could be entirely accounted for by the file-drawer effect, you need to adjust your expectation that there actually is an effect to lower than your expectation was at step 1). It isn't that the theory hasn't been tested (although you can argue it hasn't been tested for ingeniously enough yet), it's that it has been tested and no effect has been shown. If you allow the existence of interest in a theory (represented in amount of ink spilled and number of experiments done) to raise your expectation that the theory is true, despite experimental indications to the contrary, you're not really doing science, you're just throwing good money after bad, probably motivated by a desire to protect the researchers and institutions that are heavily committed to the truth of the theory and/or the desire to protect other theories that depend on the one that hasn't shown results.

civilized 4y ago
There's a lot of junk behavioral science out there, but things like "people often go with a default or recommended option so they can move on with their day" seem so obvious to me that I become suspicious of this debunking for debunking too much.
- pessimizer 4y ago
  That's just refusing to be convinced by evidence, though. It's good to have hunches, but it's good to let them go after you've done the experiments. Come up with a new hunch and a new experiment that shows why the expected effects weren't seen, and you're right back in there.
  - civilized 4y ago
    "Refusing to be convinced by evidence" is a simplistic false dichotomy. The evidence is interesting but I have several reasons not to immediately take it as definitive. Are you really certain that a big debunking in PNAS, surfing a wave of other celebrated debunkings, should be taken as definitive, when a good deal of the research being debunked was published to similar fanfare in PNAS back when a different kind of research was fashionable? I take neither the original research nor the debunking as particularly credible. Without technical expertise, I'm left to educated guess. It's just my guess.
    - pessimizer 4y ago
      > I have several reasons not to immediately take it as definitive. The one that you expressed is that it "seems so obvious to you that you become suspicious." I'm just taking you at your word.
      - civilized 4y ago
        The idea that we should always be convinced by evidence regardless of context is a vast overgeneralization, impossible ("the evidence" overall rarely points only one way, even if the latest chunk of new evidence does), and in contradiction with Bayesian epistemology.
        pessimizer 4y ago
        My problem isn't that I think people should be credulous of everything, it's that I don't think "it's just obvious" is a proper counter to experiments that show nothing. If the effect is so obvious, it should be obvious how to design an experiment that would show it. I don't even know what you're defending here other than believing your first impulse above any subsequent evidence. Nobody is preventing anyone from proving an effect, in fact they poured money into the attempt.