3 ms·

Based on seeing a lot of exercise literature, back when I took an interest: * Small sample sizes * Self-reporting and diary studies * Many uncontrolled varia

Based on seeing a lot of exercise literature, back when I took an interest: * Small sample sizes * Self-reporting and diary studies * Many uncontrolled variables * Inconsistent interventions (not isocaloric, not volume-equivalent or tonnage-equivalent etc) * Impossibility of performing blind trials to control for placebo

astazangasta 10y ago
The whole notion of 'controlled variables' is a ridiculous exercise. At some point we have to cop to the fact that biological interaction is too complex for our shitty parameterization. We're carving mountains with a tack hammer. Maybe there is a better math out there that can manage, but I haven't seen it yet.
- jacques_chester 10y ago
  Two things. First, "it's hard and we can't control everything" isn't the same as "it's impossible to control anything". Second, large sample sizes are preferred exactly to help even out the variables that are uncontrolled or, especially, unknown.
  - astazangasta 10y ago
    I'm not saying "it's impossible to control anything", I am complaining about the idea that the world can be represented as a set of variables, that everything can be quantified and broken down into axes, and this parameterization is a useful or helpful way to model the vast interacting complexity of a biological system. If you have ever tried to do PCA on a gene expression dataset, you know my pain. The exercise is futile; you are attempting to produce a description of the world based on a presumption that is wholly inadequate.
    - jacques_chester 10y ago
      My point was that you're introducing the nirvana fallacy: rejecting all attempts because they cannot satisfy a perfect outcome. That it's impossible to perfectly model reality is not news in any scientific field. In any case, I was talking about grossly observable variables such as "does this person exercise at all?", or "do they smoke?" or "do they drink?". You'd be surprised how many studies don't bother to check simple factors like these.
- throw_away_777 10y ago
  Great points. Often times studies would be much better served collecting more data and looking at interesting correlations than trying and failing to "control" for all the hidden variables and winding up with biased sample sizes of 20 or less.
  - stdbrouw 10y ago
    A confounder remains a confounder even when you've got a bazillion datapoints. Bigger sample sizes increase precision, they don't decrease bias.
    - throw_away_777 10y ago
      A confounder can remain a confounder even when you think you have controlled for all variables. Clearly social scientists are not properly analyzing their data, why should I trust them to be properly controlling for counfounding variables? I'd rather just see studies with enough data to measure the strength of a correlation than studies which are completely unreliable because of bad statistics or not enough data, as explained in the article. For example, you could do a large study where you measured the correlation of wine drinking with obesity. This study would at least tell me interesting information, even though I wouldn't know exactly if there is a causative relationship. But if the correlation was high and I wanted to lose weight I would stop drinking wine and see what happens. Maybe there is a confounding variable, maybe there isn't, I'd take that chance if I wanted to lose weight. This is much better than the current state of food and exercise studies where none of the information is reliable because of cherry picking stats and low sample sizes. As detailed in the article, we don't know anything about food and exercise from studies besides the basic physics that to lose weight you need to use more calories than you take in.
      - jschwartzi 10y ago
        The problem is that a study of the correlation of wine drinking with obesity would find an inverse correlation, as people who drink wine are more likely to be affluent and thus less likely to overeat. I think that when we get down to it the obesity epidemic in the US is a consequence of people not having enough time or money to buy unprocessed foods and cook them, and not a consequence of their lack of control.
aantix 10y ago
Gary Taube's NuSI initiative tried to control for some of these factors with metabolic chambers, having subjects in a controlled environment with exact meals, etc. Still was very hard since the subjects actually burned more calories when let out of their room. On a slightly offtopic note; the first results of NuSI have not been published but in this video, one of the researchers discusses the findings which may (or may not) come as a surprise: _No Metabolic Advantage for Ketosis Found_ https://www.youtube.com/watch?v=MiUyjMjuLl0 https://www.youtube.com/watch?v=MiUyjMjuLl0
- jacques_chester 10y ago
  NuSi might not have published a metabolic ward study, but a recent one didn't come up with any surprises either: http://www.ncbi.nlm.nih.gov/pubmed/27385608 http://www.ncbi.nlm.nih.gov/pubmed/27385608
  - hippich 10y ago
    I would argue that 4 weeks is not enough to get rid of significant insulin resistance (most sources tells that in 4 weeks is about average timeframe for body to get adapted to keto regime.) I would love to see study on keto diet and energy expenditure by body for time frames of 1 year or more.
    - jacques_chester 10y ago
      Sure, but as with the problem of small sample sizes, the root cause is usually that it is hard to get funding for large, long studies.

astazangasta 10y ago

The whole notion of 'controlled variables' is a ridiculous exercise. At some point we have to cop to the fact that biological interaction is too complex for our shitty parameterization. We're carving mountains with a tack hammer. Maybe there is a better math out there that can manage, but I haven't seen it yet.

jacques_chester 10y ago
Two things. First, "it's hard and we can't control everything" isn't the same as "it's impossible to control anything". Second, large sample sizes are preferred exactly to help even out the variables that are uncontrolled or, especially, unknown.
- astazangasta 10y ago
  I'm not saying "it's impossible to control anything", I am complaining about the idea that the world can be represented as a set of variables, that everything can be quantified and broken down into axes, and this parameterization is a useful or helpful way to model the vast interacting complexity of a biological system. If you have ever tried to do PCA on a gene expression dataset, you know my pain. The exercise is futile; you are attempting to produce a description of the world based on a presumption that is wholly inadequate.
  - jacques_chester 10y ago
    My point was that you're introducing the nirvana fallacy: rejecting all attempts because they cannot satisfy a perfect outcome. That it's impossible to perfectly model reality is not news in any scientific field. In any case, I was talking about grossly observable variables such as "does this person exercise at all?", or "do they smoke?" or "do they drink?". You'd be surprised how many studies don't bother to check simple factors like these.
throw_away_777 10y ago
Great points. Often times studies would be much better served collecting more data and looking at interesting correlations than trying and failing to "control" for all the hidden variables and winding up with biased sample sizes of 20 or less.
- stdbrouw 10y ago
  A confounder remains a confounder even when you've got a bazillion datapoints. Bigger sample sizes increase precision, they don't decrease bias.
  - throw_away_777 10y ago
    A confounder can remain a confounder even when you think you have controlled for all variables. Clearly social scientists are not properly analyzing their data, why should I trust them to be properly controlling for counfounding variables? I'd rather just see studies with enough data to measure the strength of a correlation than studies which are completely unreliable because of bad statistics or not enough data, as explained in the article. For example, you could do a large study where you measured the correlation of wine drinking with obesity. This study would at least tell me interesting information, even though I wouldn't know exactly if there is a causative relationship. But if the correlation was high and I wanted to lose weight I would stop drinking wine and see what happens. Maybe there is a confounding variable, maybe there isn't, I'd take that chance if I wanted to lose weight. This is much better than the current state of food and exercise studies where none of the information is reliable because of cherry picking stats and low sample sizes. As detailed in the article, we don't know anything about food and exercise from studies besides the basic physics that to lose weight you need to use more calories than you take in.
    - jschwartzi 10y ago
      The problem is that a study of the correlation of wine drinking with obesity would find an inverse correlation, as people who drink wine are more likely to be affluent and thus less likely to overeat. I think that when we get down to it the obesity epidemic in the US is a consequence of people not having enough time or money to buy unprocessed foods and cook them, and not a consequence of their lack of control.

aantix 10y ago

Gary Taube's NuSI initiative tried to control for some of these factors with metabolic chambers, having subjects in a controlled environment with exact meals, etc. Still was very hard since the subjects actually burned more calories when let out of their room. On a slightly offtopic note; the first results of NuSI have not been published but in this video, one of the researchers discusses the findings which may (or may not) come as a surprise: _No Metabolic Advantage for Ketosis Found_ https://www.youtube.com/watch?v=MiUyjMjuLl0 https://www.youtube.com/watch?v=MiUyjMjuLl0

jacques_chester 10y ago
NuSi might not have published a metabolic ward study, but a recent one didn't come up with any surprises either: http://www.ncbi.nlm.nih.gov/pubmed/27385608 http://www.ncbi.nlm.nih.gov/pubmed/27385608
- hippich 10y ago
  I would argue that 4 weeks is not enough to get rid of significant insulin resistance (most sources tells that in 4 weeks is about average timeframe for body to get adapted to keto regime.) I would love to see study on keto diet and energy expenditure by body for time frames of 1 year or more.
  - jacques_chester 10y ago
    Sure, but as with the problem of small sample sizes, the root cause is usually that it is hard to get funding for large, long studies.