5 ms·

The most statistically astute comment posted here so far is the top-level comment by micro_cam https://news.ycombinator.com/item?id=5493772 https://news.ycombi

The most statistically astute comment posted here so far is the top-level comment by micro_cam https://news.ycombinator.com/item?id=5493772 https://news.ycombinator.com/item?id=5493772 pointing out that the study design here doesn't involve random assignment to strength training of any kind, but rather just observation of the study population over time. Sure, it's a good idea to be stronger rather than weaker. Moreover, it is plausible that exercises that tend to develop strength (as measured in the study) have health benefits above and beyond merely developing strength. There is certainly no reason not to exercise based on this finding. But there is also not a strong reason to predict a longer rather than shorter life from your personal strength measurement, even though the study did the usual kind of regression analysis to control for other independent variables. Simply put, this was not a treatment-control study design, http://norvig.com/experiment-design.html http://norvig.com/experiment-design.html http://escholarship.org/uc/item/6hb3k0nz http://escholarship.org/uc/item/6hb3k0nz so no inference of causation is supported here. The authors were careful to write the word "association" (which is honest), and the authors were careful to investigate all-cause mortality in this study population (which is thoughtful), but we don't know yet how much you or I can improve individual lifespan by doing strength-building exercises. AFTER EDIT: The comment by micro_cam, which deserves your upvote for getting me started on my comment, is especially astute because it mentions that this was not a genetically sensitive study design. To answer the question posed in one of the replies this comment received, that will eventually be an issue worth looking at, which sorts of "endophenotypes" gain the most benefit from what sort of exercise. But we are nowhere near that level of precision of investigation yet. The statement about limitations of the study at the end of the submitted article mentions more issues. Picking up on something I learned from the late Richard Feynman's comments on the Challenger explosion investigation, I would like to see a scatterplot of these data displayed over the calculated regression line, to see how much uncertainty still surrounds their model. As it is, the confidence intervals around the death rates for different categories of strength overlap considerably, so there are some strong people with the same mortality risk as some of the weaker people.

SoftwareMaven 14y ago
This is the problem with all cohort studies. Since there aren't strong controls, you can't determine causality. Fundamentally, this is why we know get consistently conflicting advice about nutrition: we take cohort studies as law instead of using them to form hypotheses that are tested with well controlled experiments.
ellyagg 14y ago
Can you list some things that weren't controlled for that you think would lead to both increased one rep max and longer life expectancy? As Edward Tufte says, "Correlation is not causation but it sure is a hint." That increased strength improves life expectancy should be the null hypothesis. At least at the one end, extreme frailty is a common eye test for poor health. Since this isn't a scientific journal here, but a discussion, I think most of us would agree that stronger people often have more resources to ward off disease, cope with temporary reductions in nutrient bioavailability longer, can defend themselves from threats more easily, and can avoid more dangerous falls by maintaining their center of balance. And that's not a comprehensive list. Personally, I'm satisfied that increased strength reduces all cause mortality and I believe the link is strong enough to be actionable now. I'm an evangelist for it among my older friends and relatives. One way strength reduces deaths is by preventing common injuries, e.g., falling and breaking a hip. A recent study showed that a simple graded test of sitting on the ground to standing could differentiate mortality risk sharply. While we can't measure how much strength and how many years it adds, not least because that's just a vague question, the link here was almost an order of magnitude from the lowest grade to the highest grade for one repetition of standing from the ground. http://www.cbsnews.com/8301-204_162-57559095/sitting-and-standing-test-may-predict-mortality/ http://www.cbsnews.com/8301-204_162-57559095/sitting-and-sta... http://cpr.sagepub.com/content/early/2012/12/10/2047487312471759.full http://cpr.sagepub.com/content/early/2012/12/10/204748731247...
- micro_cam 14y ago
  The big issue is sub populations. I work on similar large studies including genomic data and it is quite frequent to see associations of the form "central americans have a lower rate of disease x" and "central americans have a higher rait of factor y." This produces a statistical association but, especially with genomic factors, factor y can just be something that randomly identifies the group. Similarly in this data set you might find a group of people who have higher strength because of genomic factors that make it easier for them to build strength from the same workouts as "normall" people. These people might also come from a subpopulation (farm famlies, people of Slavic heritage who knows) that has lower rates of cancer for genomic or cultural reasons. Even a small number of such people can make the association look significant though it will disappear once you remove them. It could also be something like a certain food or diet making it both easier to build strength and preventing cancer.
bjterry 14y ago
From a Bayesian perspective, you should have had a high prior that lifting weights will cause an in increase your longevity because of several plausible mechanisms, and this study should slightly increase your posterior probability for that causation. Limiting your understanding of the causations of things only to those things which have had interventionist studies performed on them will have a very deleterious effect on your life, with high probability. No one has performed an interventionist study on basically anything in macroeconomics, politics, relationships, English literature, history or just about anything else, so I hope that you apply the same understanding to all of those fields as you do to the field of health and simply throw your hands up in the air when asked if "cheating causes divorces" or "yelling racial epithets causes a lack of re-election." And please don't After Edit me, I find it patronizing.
- micro_cam 14y ago
  While this is an application of Bayes' theorem it isn't what a statistician would consider a good Bayesian analysis. Nate Silver's book and xkcd have confused this issue but a bayesian does not generally make strong assumptions about a prior distribution. A Bayesian can make strong assumptions about the hierarchical structure of probability distributions linking the paramaters/priors to the observations. Frequently she will then use uninformative prior distributions (like a really broad normall) on the paramaters of interest to avoid confirmation bias and combat overfitting when doing inference. The debate between modern frequentists (like Larry Wassermann whose blog I linked) and bayesians (like Andrew Gelman) has more to do with weather analysis done with an assumed hierarchical model is a good idea. Everyone recognizes that bayes' theorem is valid and priors are a usable tool at this point.
  - bjterry 14y ago
    I understand that in the context of professional applications of statistics, such as when writing research papers, professional statisticians often use uninformative priors. I'm not talking about professional statistics, and I'm not talking about the supposed debate between "frequentists" and "Bayesians." I'm talking about how an individual lives their life, sample size: one. The Earth doesn't have enough resources to answer every question of cause and effect that will come up in your life. For the vast majority of these questions, you will use a three pound mass of mostly fat to determine causation. Teaching that mass of fat that credible mechanisms of action and pre-existing information provides prior context for evaluating newly received information helps it not to discard valuable insight. If you provided me research that showed, weakly, that using racial epithets did not affect a politicians chances of re-election (in relevant countries and scenarios, etc.), I would not discard my existing prior that it does, and I would be right to do so. But I would keep it in mind for the next round of evidence in the event I could be wrong. In this case, no professional statistican is going to perform a Bayesian analysis to tell you the probability that making yourself stronger will cause a decrease in your all-cause mortality. No one is going to aggregate that information and tell YOU what YOU need to do, in this case as in so many others. But there is enough information that you should nonetheless, and this study, like it or not, does add to that information, not detract from it. P.S.: I disagree with using uninformative priors except when they are warranted. This is how alternative medicine masquerades as "Evidence-based Medicine."
ignostic 14y ago
Criticism regarding association, while valid, is largely irrelevant. You need to understand that a clinical trial on this matter is nearly impossible. 1) Imagine the cost of holding a clinical trial over 20 years. 2) Imagine the difficulty of trying to get people to adhere to a workout routine for 20 years! 3) Given what we already know, it's probably unethical to tell a control group not to exercise or build significant muscle strength. 4) Finally, even if you work out all the issues above, imagine trying to get funding for a hugely expensive study trying to answer the question of whether strength training lowers these risks when it's already widely regarded as healthy. The researchers are very aware of the limitations, so they went to great lengths to exclude the most likely confounding variables. > there are also confounding factors like genetics and selection bias To be more accurate, these are potential confounding variables. It's very possible that the association would hold up if we could adjust for these variables, too. Causation here is not proven, that's true, but taken together with everything else we know, it does strengthen the case. E.g. we can independently explain the mechanism whereby the heart is strengthened by resistance training. I'm not an expert in either cancer or heart disease, but I would love to hear whether we can explain the mechanism of action already.
- micro_cam 14y ago
  First, I believe that strength training is good for you and do a ton of it in the form of rock climbing. As a scientist I dislike the trend of observational studies getting massive press (see associations between Wine, Coffee, Chocolate etc and health/lifespan) because, as you say, we are doing (and should be funding more) studies on some of the underlying mechanisms but it is difficult to summarize them in a punchy headline. There are have also been some really elegantly designes observational studies that use comparisons between family members, twins or features of the population to reduce bias in an ethical way.
  - ignostic 14y ago
    You'll see no argument from me there. Understanding underlying mechanisms requires a degree of systemic knowledge far beyond that needed for an observational study, which is one reason we see far fewer of them. The other reason is funding. We are just beginning to understand how the brain works, and for one of the most promising and important areas of research it's remarkably poorly funded.
micro_cam 14y ago
"which sorts...gain the most benefit from what sort of exercise" is really key. An active area of research for me is analyzing (multivariate non linear) regression models to characterize sub populations in which an observed effect is strong or weak for large genomic studies. Correlations, p-values and univariate linear models are frustratingly insufficient in many cases and effects seen across the entire population may lead to recommendations that are bad for a large numer of people. (I'm not saying everyone shouldn't go step away from the keyboard and move some iron...just talking stats.)

SoftwareMaven 14y ago

This is the problem with all cohort studies. Since there aren't strong controls, you can't determine causality. Fundamentally, this is why we know get consistently conflicting advice about nutrition: we take cohort studies as law instead of using them to form hypotheses that are tested with well controlled experiments.

ellyagg 14y ago

Can you list some things that weren't controlled for that you think would lead to both increased one rep max and longer life expectancy? As Edward Tufte says, "Correlation is not causation but it sure is a hint." That increased strength improves life expectancy should be the null hypothesis. At least at the one end, extreme frailty is a common eye test for poor health. Since this isn't a scientific journal here, but a discussion, I think most of us would agree that stronger people often have more resources to ward off disease, cope with temporary reductions in nutrient bioavailability longer, can defend themselves from threats more easily, and can avoid more dangerous falls by maintaining their center of balance. And that's not a comprehensive list. Personally, I'm satisfied that increased strength reduces all cause mortality and I believe the link is strong enough to be actionable now. I'm an evangelist for it among my older friends and relatives. One way strength reduces deaths is by preventing common injuries, e.g., falling and breaking a hip. A recent study showed that a simple graded test of sitting on the ground to standing could differentiate mortality risk sharply. While we can't measure how much strength and how many years it adds, not least because that's just a vague question, the link here was almost an order of magnitude from the lowest grade to the highest grade for one repetition of standing from the ground. http://www.cbsnews.com/8301-204_162-57559095/sitting-and-standing-test-may-predict-mortality/ http://www.cbsnews.com/8301-204_162-57559095/sitting-and-sta... http://cpr.sagepub.com/content/early/2012/12/10/2047487312471759.full http://cpr.sagepub.com/content/early/2012/12/10/204748731247...

micro_cam 14y ago
The big issue is sub populations. I work on similar large studies including genomic data and it is quite frequent to see associations of the form "central americans have a lower rate of disease x" and "central americans have a higher rait of factor y." This produces a statistical association but, especially with genomic factors, factor y can just be something that randomly identifies the group. Similarly in this data set you might find a group of people who have higher strength because of genomic factors that make it easier for them to build strength from the same workouts as "normall" people. These people might also come from a subpopulation (farm famlies, people of Slavic heritage who knows) that has lower rates of cancer for genomic or cultural reasons. Even a small number of such people can make the association look significant though it will disappear once you remove them. It could also be something like a certain food or diet making it both easier to build strength and preventing cancer.

bjterry 14y ago

From a Bayesian perspective, you should have had a high prior that lifting weights will cause an in increase your longevity because of several plausible mechanisms, and this study should slightly increase your posterior probability for that causation. Limiting your understanding of the causations of things only to those things which have had interventionist studies performed on them will have a very deleterious effect on your life, with high probability. No one has performed an interventionist study on basically anything in macroeconomics, politics, relationships, English literature, history or just about anything else, so I hope that you apply the same understanding to all of those fields as you do to the field of health and simply throw your hands up in the air when asked if "cheating causes divorces" or "yelling racial epithets causes a lack of re-election." And please don't After Edit me, I find it patronizing.

micro_cam 14y ago
While this is an application of Bayes' theorem it isn't what a statistician would consider a good Bayesian analysis. Nate Silver's book and xkcd have confused this issue but a bayesian does not generally make strong assumptions about a prior distribution. A Bayesian can make strong assumptions about the hierarchical structure of probability distributions linking the paramaters/priors to the observations. Frequently she will then use uninformative prior distributions (like a really broad normall) on the paramaters of interest to avoid confirmation bias and combat overfitting when doing inference. The debate between modern frequentists (like Larry Wassermann whose blog I linked) and bayesians (like Andrew Gelman) has more to do with weather analysis done with an assumed hierarchical model is a good idea. Everyone recognizes that bayes' theorem is valid and priors are a usable tool at this point.
- bjterry 14y ago
  I understand that in the context of professional applications of statistics, such as when writing research papers, professional statisticians often use uninformative priors. I'm not talking about professional statistics, and I'm not talking about the supposed debate between "frequentists" and "Bayesians." I'm talking about how an individual lives their life, sample size: one. The Earth doesn't have enough resources to answer every question of cause and effect that will come up in your life. For the vast majority of these questions, you will use a three pound mass of mostly fat to determine causation. Teaching that mass of fat that credible mechanisms of action and pre-existing information provides prior context for evaluating newly received information helps it not to discard valuable insight. If you provided me research that showed, weakly, that using racial epithets did not affect a politicians chances of re-election (in relevant countries and scenarios, etc.), I would not discard my existing prior that it does, and I would be right to do so. But I would keep it in mind for the next round of evidence in the event I could be wrong. In this case, no professional statistican is going to perform a Bayesian analysis to tell you the probability that making yourself stronger will cause a decrease in your all-cause mortality. No one is going to aggregate that information and tell YOU what YOU need to do, in this case as in so many others. But there is enough information that you should nonetheless, and this study, like it or not, does add to that information, not detract from it. P.S.: I disagree with using uninformative priors except when they are warranted. This is how alternative medicine masquerades as "Evidence-based Medicine."

ignostic 14y ago

Criticism regarding association, while valid, is largely irrelevant. You need to understand that a clinical trial on this matter is nearly impossible. 1) Imagine the cost of holding a clinical trial over 20 years. 2) Imagine the difficulty of trying to get people to adhere to a workout routine for 20 years! 3) Given what we already know, it's probably unethical to tell a control group not to exercise or build significant muscle strength. 4) Finally, even if you work out all the issues above, imagine trying to get funding for a hugely expensive study trying to answer the question of whether strength training lowers these risks when it's already widely regarded as healthy. The researchers are very aware of the limitations, so they went to great lengths to exclude the most likely confounding variables. > there are also confounding factors like genetics and selection bias To be more accurate, these are potential confounding variables. It's very possible that the association would hold up if we could adjust for these variables, too. Causation here is not proven, that's true, but taken together with everything else we know, it does strengthen the case. E.g. we can independently explain the mechanism whereby the heart is strengthened by resistance training. I'm not an expert in either cancer or heart disease, but I would love to hear whether we can explain the mechanism of action already.

micro_cam 14y ago
First, I believe that strength training is good for you and do a ton of it in the form of rock climbing. As a scientist I dislike the trend of observational studies getting massive press (see associations between Wine, Coffee, Chocolate etc and health/lifespan) because, as you say, we are doing (and should be funding more) studies on some of the underlying mechanisms but it is difficult to summarize them in a punchy headline. There are have also been some really elegantly designes observational studies that use comparisons between family members, twins or features of the population to reduce bias in an ethical way.
- ignostic 14y ago
  You'll see no argument from me there. Understanding underlying mechanisms requires a degree of systemic knowledge far beyond that needed for an observational study, which is one reason we see far fewer of them. The other reason is funding. We are just beginning to understand how the brain works, and for one of the most promising and important areas of research it's remarkably poorly funded.

micro_cam 14y ago

"which sorts...gain the most benefit from what sort of exercise" is really key. An active area of research for me is analyzing (multivariate non linear) regression models to characterize sub populations in which an observed effect is strong or weak for large genomic studies. Correlations, p-values and univariate linear models are frustratingly insufficient in many cases and effects seen across the entire population may lead to recommendations that are bad for a large numer of people. (I'm not saying everyone shouldn't go step away from the keyboard and move some iron...just talking stats.)