5 ms·

This article, which boils down to "if the data seems unlikely from what we know beforehand, throw it the hell out and never look at it again", is the exact oppo

This article, which boils down to "if the data seems unlikely from what we know beforehand, throw it the hell out and never look at it again", is the exact opposite of what scientists are taught, and what they are taught - not this article - is the correct approach.

claytonjy 8y ago
This is a very poor and incorrect reading of the article here. The point is not that the data were unlikely and should be thrown out; it's that if the effect size reported was accurate, that would affect so many other things so strongly that the current state of civilization refutes the estimated effect size from the study. It's not that the data was poor, but that it and the analysis was incomplete and ill considered. In psychology in particular, large effects will seep into everyday life well beyond the phenomena being studied, which gives a very useful check on astronomical effects like those originally reported on judges and time of day.
- MrPrinter 8y ago
  GP says, and I concur, that the scientific method would not to be just say "this must be impossible, let's ignore it", but try to disprove the study (maybe by trying and failing to replicate the study, or designing better studies).
  - claytonjy 8y ago
    I think we all agree with that position; the issue is the author of TFA did not make that position, so GGP was arguing against a strawman. The original study was disproved, and TFA includes multiple such links near the top.
    - Retric 8y ago
      The study was not disproved, the conclusion was. The critical part of science is the actual data and methodology. If you do X you will see Y is what constrains new theories. Conclusions on the other hand have very little weight. PS: Sorry pet peeve.
  - mrfredward 8y ago
    Not "this must be impossible, let's ignore it," but "this must be impossible, what did we do wrong?" If your conclusion is impossible, look for the mistake before publishing or trying to replicate it.
- TuGuQuKu 8y ago
  The article, your comment and the argument they make, that this effect -- if it exists -- should be highly visible across all strata of society, are taking a ton of assumptions for granted. There are a number of plausible scenarios in which this argument falls apart completely: a) The act of ruling over criminal proceedings places one in a very unique position of power over complete strangers' lives which only judges routinely experience. What subtle effects on empathy, vindictiveness, leniency do minor fluctuations in mood have that are exacerbated by the unique conditions of this profession? b) Judges are members of an extremely privileged professional class and universally come from extremely privileged backgrounds. The mechanism of the hedonic treadmill might explain how working class folks cope with regular day-to-day stresses with almost no impact to their mood where being peckish at midday is likely the harshest reality of life that a financially secure judge making $100 an hour has to deal with and may disproportionately sour their temper to a comparable degree that someone making minimum wage worries about paying their rent. Etc etc. There is no compelling proof of this argument.

claytonjy 8y ago

This is a very poor and incorrect reading of the article here. The point is not that the data were unlikely and should be thrown out; it's that if the effect size reported was accurate, that would affect so many other things so strongly that the current state of civilization refutes the estimated effect size from the study. It's not that the data was poor, but that it and the analysis was incomplete and ill considered. In psychology in particular, large effects will seep into everyday life well beyond the phenomena being studied, which gives a very useful check on astronomical effects like those originally reported on judges and time of day.

MrPrinter 8y ago
GP says, and I concur, that the scientific method would not to be just say "this must be impossible, let's ignore it", but try to disprove the study (maybe by trying and failing to replicate the study, or designing better studies).
- claytonjy 8y ago
  I think we all agree with that position; the issue is the author of TFA did not make that position, so GGP was arguing against a strawman. The original study was disproved, and TFA includes multiple such links near the top.
  - Retric 8y ago
    The study was not disproved, the conclusion was. The critical part of science is the actual data and methodology. If you do X you will see Y is what constrains new theories. Conclusions on the other hand have very little weight. PS: Sorry pet peeve.
- mrfredward 8y ago
  Not "this must be impossible, let's ignore it," but "this must be impossible, what did we do wrong?" If your conclusion is impossible, look for the mistake before publishing or trying to replicate it.
TuGuQuKu 8y ago
The article, your comment and the argument they make, that this effect -- if it exists -- should be highly visible across all strata of society, are taking a ton of assumptions for granted. There are a number of plausible scenarios in which this argument falls apart completely: a) The act of ruling over criminal proceedings places one in a very unique position of power over complete strangers' lives which only judges routinely experience. What subtle effects on empathy, vindictiveness, leniency do minor fluctuations in mood have that are exacerbated by the unique conditions of this profession? b) Judges are members of an extremely privileged professional class and universally come from extremely privileged backgrounds. The mechanism of the hedonic treadmill might explain how working class folks cope with regular day-to-day stresses with almost no impact to their mood where being peckish at midday is likely the harshest reality of life that a financially secure judge making $100 an hour has to deal with and may disproportionately sour their temper to a comparable degree that someone making minimum wage worries about paying their rent. Etc etc. There is no compelling proof of this argument.