4 ms·

> But I want to take a different approach in this blog. I think we should dismiss this finding, simply because it is impossible. That is truly begging the ques

> But I want to take a different approach in this blog. I think we should dismiss this finding, simply because it is impossible. That is truly begging the question. He goes into details about what the impact to society would be if we're that affected by hunger/fatigue, but the validity of the finding still remains. > It is up to authors to interpret the effect size in their study, and to show the mechanism through which an effect that is impossibly large, becomes plausible. Without such an explanation, the finding should simply be dismissed. Again, horrible circular logic here. Simply disregarding things because they're "impossible" is the antithesis of science. The authors of the paper gave a hypothesis that explains the (undisputed) data. You should not simply dismiss this by just saying "it's not true because it can't possibly be true". For instance, try explaining the germ theory of disease to somebody before Anton van Leeuwenhoek invented the microsope.

nerdponx 9y ago
Based on this data, the difference between the height of 21-year-old men and women in The Netherlands is approximately 13 centimeters. That is a Cohen’s d of 2. That’s the effect size in the hungry judges study. If hunger had an effect on our mental resources of this magnitude, our society would fall into minor chaos every day at 11:45 a.m. Or at the very least, our society would have organized itself around this incredibly strong effect of mental depletion. Just like manufacturers take size differences between men and women into account when producing items such as golf clubs or watches, we would stop teaching in the time before lunch, doctors would not schedule surgery, and driving before lunch would be illegal. If a psychological effect is this big, we don’t need to discover it and publish it in a scientific journal—you would already know it exists. Sort of how the “after lunch dip” is a strong and replicable finding that you can feel yourself (and that, as it happens, is directly in conflict with the finding that judges perform better immediately after lunch—surprisingly, the authors don’t discuss the after lunch dip). It is impossible because we have very strong evidence against it.
- ballenf 9y ago
  The flaw is that the leap from decision in difficult cases to dangerous driving or other behavior are wild guesses. He sets up straw men versions of what the impact of the evidence should be, then rebuts them. The reality is that the same psychological state that makes a judge less lenient might also make us better drivers, not worse.
  - nerdponx 9y ago
    You're missing the point. The point is that the effect sizes in question are large enough that people should notice them in their day to day lives, because the proposed mechanism of action should not only apply to judges, but should apply to everyone everywhere. But we don't see this big obvious affect in other places.
- jimmywanger 9y ago
  > It is impossible because we have very strong evidence against it. We do not have evidence against it. Everything he has written is the _lack_ of things that he thinks should logically happen, rather than things that are happening that would disprove the hunger hypothesis. Evidence would some other experiential data, such as the number of birdies in golf remaining the same before and after lunch.
  - coldtea 9y ago
    >Everything he has written is the _lack_ of things that he thinks should logically happen, rather than things that are happening that would disprove the hunger hypothesis. Actually arguing on the "lack of things that should logically happen" is exactly the same as arguing on "things that are happening". In other words, what he says is the same whether he says "It can't be happen because X would be happen too and we don't see X", or he says "It can't happen because we see Y" (where Y is not X).
    - jimmywanger 9y ago
      > Actually arguing on the "lack of things that should logically happen" is exactly the same as arguing on "things that are happening". Reasoning from first principles is a huge downfall the higher up the science stack you go. The only time you really reason from first principles is in math. When you get to psychology and sociology, you need try fit observed data to hypotheses. The author has suggested interesting follow up experiments, but has given no backing for the disproval of the hypothesis other than "I don't believe that can possibly be true." That's not science, or even an informed opinion on science. That's just religion.
      - coldtea 9y ago
        >Reasoning from first principles is a huge downfall the higher up the science stack you go. The only time you really reason from first principles is in math. When you get to psychology and sociology, you need try fit observed data to hypotheses. Observed reality/experience -- which is what the author invokes -- is not "first principles" though.
- 24gttghh 9y ago
  I'd like to postulate that it may be that such chaos actually exists now, it is just spread throughout the day as not everyone takes breakfast/elevensies/lunch/dinner at exactly the same time as their peers, and metabolisms vary from person to person.
mfoy_ 9y ago
I agree that simply disregarding things because they're impossible is not great science, but in this case this isn't great science from the start, because we're simply measuring two of many, many variables. We're only looking at judgement outcomes, and time of day. The thing is, this result is about the story... about the narrative the findings tell. The original findings told the story of "hungry judges are grumpy judges", but that was conflating correlation with causation. There are other variables which were NOT controlled for (i.e. not great science) that tell a DIFFERENT story. This article is about the author pointing out that there are multiple ways to tell this story, and this particular way is about as impossible as it gets, statistically speaking.
- jimmywanger 9y ago
  > This article is about the author pointing out that there are multiple ways to tell this story, and this particular way is about as impossible as it gets, statistically speaking. The author doesn't cite any statistical examples for his refutation of the study. He simply says that the effect is too big, gives some examples of how big the effect is in other situations, and cites absolutely zero statistics to support his refutation. If there was a study where they compared relative conviction rates of judges who had a 10 minute snack recess every two hours, if that rate was the same, that would be some evidence. In fact, it seems as though the conviction rate goes back to normal every time that the judge takes a break, which might suggest that taking a break is the triggering thing, not the food. Would be interesting to see what would happens if the judge were allowed breaks, but not allowed to eat or drink during the breaks.
  - mfoy_ 9y ago
    As a layman, it felt like his article was addressed at professional psychologists, not laymen like myself (or, presumably, yourself). It read like it was assumed that you'd realize how massively overblown such a result should be interpreted. It would be like if you wrote a couple tests and your tooling said you had 100% code coverage, you be like "Wait, that just can't be right. I was just writing tests for one function..." Or imagine you go to download a 100GB file from some server and the download completes in 5 seconds, you'd be like "Wait, that.... just can't be right. I don't have a SSD, allocating the disk space alone should take longer than that..." Of course, that just my assumption, and my impression.
sp332 9y ago
The beginning of the article has several links giving reasons why the correlation could be explained by other effects. This article explains why the correlation can't be explained by the given reason.
- jimmywanger 9y ago
  > This article explains why the correlation can't be explained by the given reason. The correlation exists, and the authors of the paper suggest a possible hypothesis that would explain it. This article explains nothing - just goes into flights of fancy describing possible outcomes based on the author's logic. If there were actually studies that show leniency vs. hunger that contradict original findings, that would explain why the correlation should not be explained by the given reason. In fact, the referenced study is a perfect example of a good science paper, and the scientific process in general. "Hey, we have a big anomaly here, here's a possible explanation. This explanation also suggests other further papers/studies that could explain the problem."
  - sp332 9y ago
    Well it's probably not a bad paper. And the follow-up from the authors seems pretty solid too. http://m.pnas.org/content/108/42/E834.full http://m.pnas.org/content/108/42/E834.full But that doesn't mean it's right. And yeah the linked article isn't going to get published in PNAS or anything but it's an interesting thought experiment.
phsource 9y ago
I completely agree. The article's author suggests that the study's result is too good (to be true), and casts that as the sole reason to reject the study. I think that such a surprising result definitely deserves scrutiny, but barring problems with the method or stats used, it's merely surprising, not "impossible". Within the paper itself, the authors clearly considered this: A key aspect for interpreting the association between the ordinal position of a case and parole decisions is whether an unobserved factor determines case order in such a way that yields the pattern of results we obtain. They then interview judges and lawyers and examine the process to rule these out! Which is rare enough for a quantitative study like this (where they often publish the conclusions and just say "it could be X"). Unless the author can cite a more specific grievance, I'll trust the data.
- JackFr 9y ago
  > I think that such a surprising result definitely deserves scrutiny But I think the point of the OP is that the effect is so large, it would be impossible for it to be a surprise to anyone.
EGreg 9y ago
Look at Bem's studies. They highlight this phenomenon clearly. Even meta studies can't prove things. http://daniellakens.blogspot.com/2015/04/why-meta-analysis-of-90-precognition.html?m=1 http://daniellakens.blogspot.com/2015/04/why-meta-analysis-o...

nerdponx 9y ago

Based on this data, the difference between the height of 21-year-old men and women in The Netherlands is approximately 13 centimeters. That is a Cohen’s d of 2. That’s the effect size in the hungry judges study. If hunger had an effect on our mental resources of this magnitude, our society would fall into minor chaos every day at 11:45 a.m. Or at the very least, our society would have organized itself around this incredibly strong effect of mental depletion. Just like manufacturers take size differences between men and women into account when producing items such as golf clubs or watches, we would stop teaching in the time before lunch, doctors would not schedule surgery, and driving before lunch would be illegal. If a psychological effect is this big, we don’t need to discover it and publish it in a scientific journal—you would already know it exists. Sort of how the “after lunch dip” is a strong and replicable finding that you can feel yourself (and that, as it happens, is directly in conflict with the finding that judges perform better immediately after lunch—surprisingly, the authors don’t discuss the after lunch dip). It is impossible because we have very strong evidence against it.

ballenf 9y ago
The flaw is that the leap from decision in difficult cases to dangerous driving or other behavior are wild guesses. He sets up straw men versions of what the impact of the evidence should be, then rebuts them. The reality is that the same psychological state that makes a judge less lenient might also make us better drivers, not worse.
- nerdponx 9y ago
  You're missing the point. The point is that the effect sizes in question are large enough that people should notice them in their day to day lives, because the proposed mechanism of action should not only apply to judges, but should apply to everyone everywhere. But we don't see this big obvious affect in other places.
jimmywanger 9y ago
> It is impossible because we have very strong evidence against it. We do not have evidence against it. Everything he has written is the _lack_ of things that he thinks should logically happen, rather than things that are happening that would disprove the hunger hypothesis. Evidence would some other experiential data, such as the number of birdies in golf remaining the same before and after lunch.
- coldtea 9y ago
  >Everything he has written is the _lack_ of things that he thinks should logically happen, rather than things that are happening that would disprove the hunger hypothesis. Actually arguing on the "lack of things that should logically happen" is exactly the same as arguing on "things that are happening". In other words, what he says is the same whether he says "It can't be happen because X would be happen too and we don't see X", or he says "It can't happen because we see Y" (where Y is not X).
  - jimmywanger 9y ago
    > Actually arguing on the "lack of things that should logically happen" is exactly the same as arguing on "things that are happening". Reasoning from first principles is a huge downfall the higher up the science stack you go. The only time you really reason from first principles is in math. When you get to psychology and sociology, you need try fit observed data to hypotheses. The author has suggested interesting follow up experiments, but has given no backing for the disproval of the hypothesis other than "I don't believe that can possibly be true." That's not science, or even an informed opinion on science. That's just religion.
    - coldtea 9y ago
      >Reasoning from first principles is a huge downfall the higher up the science stack you go. The only time you really reason from first principles is in math. When you get to psychology and sociology, you need try fit observed data to hypotheses. Observed reality/experience -- which is what the author invokes -- is not "first principles" though.
24gttghh 9y ago
I'd like to postulate that it may be that such chaos actually exists now, it is just spread throughout the day as not everyone takes breakfast/elevensies/lunch/dinner at exactly the same time as their peers, and metabolisms vary from person to person.

mfoy_ 9y ago

I agree that simply disregarding things because they're impossible is not great science, but in this case this isn't great science from the start, because we're simply measuring two of many, many variables. We're only looking at judgement outcomes, and time of day. The thing is, this result is about the story... about the narrative the findings tell. The original findings told the story of "hungry judges are grumpy judges", but that was conflating correlation with causation. There are other variables which were NOT controlled for (i.e. not great science) that tell a DIFFERENT story. This article is about the author pointing out that there are multiple ways to tell this story, and this particular way is about as impossible as it gets, statistically speaking.

jimmywanger 9y ago
> This article is about the author pointing out that there are multiple ways to tell this story, and this particular way is about as impossible as it gets, statistically speaking. The author doesn't cite any statistical examples for his refutation of the study. He simply says that the effect is too big, gives some examples of how big the effect is in other situations, and cites absolutely zero statistics to support his refutation. If there was a study where they compared relative conviction rates of judges who had a 10 minute snack recess every two hours, if that rate was the same, that would be some evidence. In fact, it seems as though the conviction rate goes back to normal every time that the judge takes a break, which might suggest that taking a break is the triggering thing, not the food. Would be interesting to see what would happens if the judge were allowed breaks, but not allowed to eat or drink during the breaks.
- mfoy_ 9y ago
  As a layman, it felt like his article was addressed at professional psychologists, not laymen like myself (or, presumably, yourself). It read like it was assumed that you'd realize how massively overblown such a result should be interpreted. It would be like if you wrote a couple tests and your tooling said you had 100% code coverage, you be like "Wait, that just can't be right. I was just writing tests for one function..." Or imagine you go to download a 100GB file from some server and the download completes in 5 seconds, you'd be like "Wait, that.... just can't be right. I don't have a SSD, allocating the disk space alone should take longer than that..." Of course, that just my assumption, and my impression.

sp332 9y ago

The beginning of the article has several links giving reasons why the correlation could be explained by other effects. This article explains why the correlation can't be explained by the given reason.

jimmywanger 9y ago
> This article explains why the correlation can't be explained by the given reason. The correlation exists, and the authors of the paper suggest a possible hypothesis that would explain it. This article explains nothing - just goes into flights of fancy describing possible outcomes based on the author's logic. If there were actually studies that show leniency vs. hunger that contradict original findings, that would explain why the correlation should not be explained by the given reason. In fact, the referenced study is a perfect example of a good science paper, and the scientific process in general. "Hey, we have a big anomaly here, here's a possible explanation. This explanation also suggests other further papers/studies that could explain the problem."
- sp332 9y ago
  Well it's probably not a bad paper. And the follow-up from the authors seems pretty solid too. http://m.pnas.org/content/108/42/E834.full http://m.pnas.org/content/108/42/E834.full But that doesn't mean it's right. And yeah the linked article isn't going to get published in PNAS or anything but it's an interesting thought experiment.

phsource 9y ago

I completely agree. The article's author suggests that the study's result is too good (to be true), and casts that as the sole reason to reject the study. I think that such a surprising result definitely deserves scrutiny, but barring problems with the method or stats used, it's merely surprising, not "impossible". Within the paper itself, the authors clearly considered this: A key aspect for interpreting the association between the ordinal position of a case and parole decisions is whether an unobserved factor determines case order in such a way that yields the pattern of results we obtain. They then interview judges and lawyers and examine the process to rule these out! Which is rare enough for a quantitative study like this (where they often publish the conclusions and just say "it could be X"). Unless the author can cite a more specific grievance, I'll trust the data.

JackFr 9y ago
> I think that such a surprising result definitely deserves scrutiny But I think the point of the OP is that the effect is so large, it would be impossible for it to be a surprise to anyone.

EGreg 9y ago

Look at Bem's studies. They highlight this phenomenon clearly. Even meta studies can't prove things. http://daniellakens.blogspot.com/2015/04/why-meta-analysis-of-90-precognition.html?m=1 http://daniellakens.blogspot.com/2015/04/why-meta-analysis-o...