5 ms·

Here's a rebuttal article as well, to bring nuance to the conversation: My rejection of the National Marriage Project’s “Before ‘I Do'”: https://familyinequali

Here's a rebuttal article as well, to bring nuance to the conversation: My rejection of the National Marriage Project’s “Before ‘I Do'”: https://familyinequality.wordpress.com/2014/08/27/my-rejection-of-the-national-marriage-projects-before-i-do/ https://familyinequality.wordpress.com/2014/08/27/my-rejecti... I'd be interested if there has been follow up research since.

dudeinjapan 3y ago
Their strongest argument seems to be “correlation doesn’t equal causality”. But it’s somewhat of a moot point. Suppose I do a study that finds lower household wealth is correlated with time spent gambling in Vegas. Do gambling losses cause the lower wealth? Or is gambling more prevalent among lower income earners? If my goal is to choose a wealth advisor, maybe it doesn’t matter.
- Nevermark 3y ago
  That isn’t a good comparison. Vegas, regardless of 1000 other variables you might introduce, is a statistically solid system. You are likely to lose money, with the standard of deviation going down the more you gamble. Low income, education, whatever, don’t change the strong link of gambling to losses. And this is most definitely a causal effect. Verifiable by mathematics, any study you want to make trying testing counter vailing variables, the teleology that develops as economic systems adapt to create that causality to flourish, and the designs of gambling systems. So of course the other variables don’t matter to your financial advisor. — An exception is games where skill can tip odds in your favor, however slight. Such as poker. A financial advisor isn’t going to second guess a poker player with a long standing career of positive outcomes.
  - logicchains 3y ago
    A better example is drinking. Say a study clearly shows that low social economic status people drink twice as much alcohol as other people. This doesn't show that being low social economic status causes people to drink more. But regardless of that, if you wanted to date people who drank less alcohol, on average by dating only people of non-low SES you'd be dating people who drank half as much compared to if you only dated people of low SES. I.e. if A is a reliable predictor of B, it doesn't matter whether A caused B or both A and B are caused by some other factor; A is still a reliable predictor regardless.
    - n9 3y ago
      agree -- all this reminds me of a study that indicated that people that used LSD even one time were like 5x as likely to suffer from depression later in life. I've always thought that it was very interesting that the conclusion drawn was that LSD made you depressed vs. asking questions about the correlation. it is funny, really how obvious the angle is in these kinds of analysis. There's no way you could even start to draw conclusions from these kinds of data sets unless you already knew what you were going to report -- there's less than no information in the numbers being used... the only rational analysis would result in nothing but questions and absolutely no conclusions.
      - dudeinjapan 3y ago
        They should have done a study on what % of people who tried LSD realized that we are all one consciousness experiencing itself subjectively, there's no such thing as death, life is only a dream, and we're the imagination of ourselves. (RIP Bill Hicks)
scythe 3y ago
Some criticisms of the rebuttal are good. Some are not: >The method for arriving at these “adjusted” (for race, education, etc) percentages is not given. There is a pretty standard way of doing this: you stratify the sample, then re-weight the subcategories according to their distribution in the population. For example, if your sample had 10% Black people and 15% Hispanic, you would re-weight to 12% and 18%. There are valid criticisms, and then there is objecting for the sake of objecting. Meanwhile, the original study has its own share of eye-spinners: >Many in Generation YOLO (you only live once) believe that what happens while you’re young won’t affect your future. Generation "YOLO" drinks less as teenagers than any generation before them since Prohibition. Generation "YOLO" has less sex in high school. >There are two other related premarital risk factors for low marital quality in our research sample: Having lived with someone other than a future spouse and/or having been married previously. The effect shown in the figure is very small. As a rule of thumb, the smaller an effect is, the more likely it can be caused by hidden variables. Smoking causing a 1600% increase in lung cancer rates was probably not hidden variables. A 20% increase in the frequency of "high-quality" marriages could be caused by almost anything. Following the same logic, there are a few inferences drawn in the study that are likely to be robust: among college-educated adults (but not non-college adults), those who had a child together before marriage were fifteen times less likely to report high marriage quality (Figure 5). Even if we object to the way data were collected and binned, it's hard to put this rabbit back in the hat: it probably means something, even if it isn't causal. The major weakness of the project is its grandiosity: its assertion that it has identified all sorts of lifestyle factors that can probably lead to a high-quality marriage, and that it can somehow serve as important life advice on these grounds. This drives an excessive optimism about conclusions that, mostly, should be checked again several times before being accepted.

dudeinjapan 3y ago

Their strongest argument seems to be “correlation doesn’t equal causality”. But it’s somewhat of a moot point. Suppose I do a study that finds lower household wealth is correlated with time spent gambling in Vegas. Do gambling losses cause the lower wealth? Or is gambling more prevalent among lower income earners? If my goal is to choose a wealth advisor, maybe it doesn’t matter.

Nevermark 3y ago
That isn’t a good comparison. Vegas, regardless of 1000 other variables you might introduce, is a statistically solid system. You are likely to lose money, with the standard of deviation going down the more you gamble. Low income, education, whatever, don’t change the strong link of gambling to losses. And this is most definitely a causal effect. Verifiable by mathematics, any study you want to make trying testing counter vailing variables, the teleology that develops as economic systems adapt to create that causality to flourish, and the designs of gambling systems. So of course the other variables don’t matter to your financial advisor. — An exception is games where skill can tip odds in your favor, however slight. Such as poker. A financial advisor isn’t going to second guess a poker player with a long standing career of positive outcomes.
- logicchains 3y ago
  A better example is drinking. Say a study clearly shows that low social economic status people drink twice as much alcohol as other people. This doesn't show that being low social economic status causes people to drink more. But regardless of that, if you wanted to date people who drank less alcohol, on average by dating only people of non-low SES you'd be dating people who drank half as much compared to if you only dated people of low SES. I.e. if A is a reliable predictor of B, it doesn't matter whether A caused B or both A and B are caused by some other factor; A is still a reliable predictor regardless.
  - n9 3y ago
    agree -- all this reminds me of a study that indicated that people that used LSD even one time were like 5x as likely to suffer from depression later in life. I've always thought that it was very interesting that the conclusion drawn was that LSD made you depressed vs. asking questions about the correlation. it is funny, really how obvious the angle is in these kinds of analysis. There's no way you could even start to draw conclusions from these kinds of data sets unless you already knew what you were going to report -- there's less than no information in the numbers being used... the only rational analysis would result in nothing but questions and absolutely no conclusions.
    - dudeinjapan 3y ago
      They should have done a study on what % of people who tried LSD realized that we are all one consciousness experiencing itself subjectively, there's no such thing as death, life is only a dream, and we're the imagination of ourselves. (RIP Bill Hicks)

scythe 3y ago

Some criticisms of the rebuttal are good. Some are not: >The method for arriving at these “adjusted” (for race, education, etc) percentages is not given. There is a pretty standard way of doing this: you stratify the sample, then re-weight the subcategories according to their distribution in the population. For example, if your sample had 10% Black people and 15% Hispanic, you would re-weight to 12% and 18%. There are valid criticisms, and then there is objecting for the sake of objecting. Meanwhile, the original study has its own share of eye-spinners: >Many in Generation YOLO (you only live once) believe that what happens while you’re young won’t affect your future. Generation "YOLO" drinks less as teenagers than any generation before them since Prohibition. Generation "YOLO" has less sex in high school. >There are two other related premarital risk factors for low marital quality in our research sample: Having lived with someone other than a future spouse and/or having been married previously. The effect shown in the figure is very small. As a rule of thumb, the smaller an effect is, the more likely it can be caused by hidden variables. Smoking causing a 1600% increase in lung cancer rates was probably not hidden variables. A 20% increase in the frequency of "high-quality" marriages could be caused by almost anything. Following the same logic, there are a few inferences drawn in the study that are likely to be robust: among college-educated adults (but not non-college adults), those who had a child together before marriage were fifteen times less likely to report high marriage quality (Figure 5). Even if we object to the way data were collected and binned, it's hard to put this rabbit back in the hat: it probably means something, even if it isn't causal. The major weakness of the project is its grandiosity: its assertion that it has identified all sorts of lifestyle factors that can probably lead to a high-quality marriage, and that it can somehow serve as important life advice on these grounds. This drives an excessive optimism about conclusions that, mostly, should be checked again several times before being accepted.