3 ms·

Coffee consumption on health is probably one of the easiest things to study, because it's so easy to collect big data on it and perform robust statistics. My po

by _of 11y ago

Coffee consumption on health is probably one of the easiest things to study, because it's so easy to collect big data on it and perform robust statistics. My point is that coffee consumption is so common and widespread among different populations and genetic backgrounds that we can get very big sample sizes, and sure there can be confounding factors, but with statistics we can adjust and control for that. Then it's a whole other thing to explain the biochemistry behind something like coffee consumption and health, that will require other data and experiments. There is a clear publication bias in science toward positive results, however, if a large, well-designed study running over many years come up with a negative result - this is actually not hard to publish at all.

stdbrouw 11y ago
> there can be confounding factors, but with statistics we can adjust and control for that The extent to which you can adjust for a confounder depends not just on your sample size but also on on whether you were able to measure that confounder without bias. Big samples don't reduce bias. Numerous examples exist of people who are dead certain that they've accounted for every possible confounder and that they've measured those confounders in exactly the right way... except they didn't. My favorite example of this is the benefits of modest alcohol consumption. The group of people who don't drink include those who abstain from alcohol for medical reasons (former alcoholics or medication that doesn't mix well with alcohol) and so of course non-drinkers are going to be less healthy on average than someone who likes a drink every once in a while. It sounds obvious when I state it like this and you would figure every questionnaire and analysis would take it into account, but study after study after study investigating the effect of alcohol on heart health over a timespan of half a century failed to take this possibility into account. Cf. http://www.annalsofepidemiology.org/article/S1047-2797(07)00007-5/abstract http://www.annalsofepidemiology.org/article/S1047-2797(07)00... (There are similar concerns about e.g. the risks associated with red meat and the benefits associated with breastfeeding, two other scientific findings that have become conventional wisdom but that in fact we are still uncertain about.)
- JoeAltmaier 11y ago
  Feynman has a diatribe about this, regarding psychological studies and rats in mazes.

by _of 11y ago

stdbrouw 11y ago

> there can be confounding factors, but with statistics we can adjust and control for that The extent to which you can adjust for a confounder depends not just on your sample size but also on on whether you were able to measure that confounder without bias. Big samples don't reduce bias. Numerous examples exist of people who are dead certain that they've accounted for every possible confounder and that they've measured those confounders in exactly the right way... except they didn't. My favorite example of this is the benefits of modest alcohol consumption. The group of people who don't drink include those who abstain from alcohol for medical reasons (former alcoholics or medication that doesn't mix well with alcohol) and so of course non-drinkers are going to be less healthy on average than someone who likes a drink every once in a while. It sounds obvious when I state it like this and you would figure every questionnaire and analysis would take it into account, but study after study after study investigating the effect of alcohol on heart health over a timespan of half a century failed to take this possibility into account. Cf. http://www.annalsofepidemiology.org/article/S1047-2797(07)00007-5/abstract http://www.annalsofepidemiology.org/article/S1047-2797(07)00... (There are similar concerns about e.g. the risks associated with red meat and the benefits associated with breastfeeding, two other scientific findings that have become conventional wisdom but that in fact we are still uncertain about.)

JoeAltmaier 11y ago
Feynman has a diatribe about this, regarding psychological studies and rats in mazes.