3 ms·

Right - but there is also a relationship between IQ and educational attainment. So controlling for it means that you're measuring something fundamentally differ

Right - but there is also a relationship between IQ and educational attainment. So controlling for it means that you're measuring something fundamentally different. That's fine, but you can't just control for things without having a valid theoretical construct to justify it. (Well, OK, you can, and people do, and it's great for getting p<0.05 if you don't care about getting meaningful answers about the underlying questions.)

georgehemingto1 7y ago
it's a triviality to say that you can control for educational attainment (and opportunities for education as well as class) when accounting for differences in IQ: it's hard to do in practice because social science is complicated and requires ACTUALLY DOING THE SCIENCE so gwern's glib comment is just wrong here clearly we're talking about the impact on IQ of whatever factors we make up to describe IQ, and obviously, and provenly, education (and therefore the ability to get education) is in play here, which gwern's glib rebuttal ignores
georgehemingto1 7y ago
honestly it's disappointing when i see people with training in stats make these comments: I'm a fucking poet if I'm anything at all and yet I find myself drawn into these ridiculous discussions about stats (in which I have no formal training) where I feel like I'm one of the only people in the room who knows how to divy up numbers
georgehemingto1 7y ago
also, to clarify things a bit: in my first comment i got a bit mixed up (fuck yeah lifelong learning disability) trying to make my point about lifelong attainment and the IQ analogy and made it sound like the paper in question was about IQ, which is incorrect. The point stands however: controlling for educational background is a thing you can do in such a study as according to the factors I pointed to, and gwern's glib comment is wrong
- gwern 7y ago
  No, your comment was glib. You suggest controlling for variables which are going to create overcontrolling and the sociologist's fallacy, and you know perfectly well that family background is not independent of IQ and attenuates the relationship and your suggestion creates only an uninterpretable mishmash of a marginal effect which includes some causal effects but quite arbitrarily excludes others which run through controlled variables. I did not choose my example of education/IQ/income casually, but picked one which is very common in the literature which people commit because they think they are being rigorous when in fact they're only fucking up their own results and making them less meaningful. One's intelligence is not independent of one's family or genetic background! This matters to testing and things which can change! (Because it gives you residual confounding, if nothing else.) By the way, the proper way to do what you may or may not want to do (because your comments are so glib it's unclear to me what DAG you have in mind or what causal effect you want) is sibling controls, not family background controls, and you will be unhappy as to the results when they have been done.
  - georgehemingto1 7y ago
    This is a total nothingburger, a long paragraph attacking a strawman of an argument you haven't even solicited the details of
    - gwern 7y ago
      The burden is on the person saying that 'such-and-such a thing should be done' to explain themselves. You failed to do that, and you have still failed to do that, and you spend your comments complaining about being criticized, so your comments remain nothingburgers.
  - georgehemingto1 7y ago
    I mean jesus, of course family background is not independent of IQ, but assessments of family background can obviously be informative in understanding how we get from there to here. To say that, quote, "One's intelligence is not independent of one's family or genetic background" is a flaccid truism. Nonetheless particular comparisons within assessments of family background can obviously in principle be informative even if they are hard to do. Your suggestion - with this fringe talk about "the sociologist's fallacy - is to eliminate any explanation from any study which does not assign success to invariant psychological metrics.

georgehemingto1 7y ago

it's a triviality to say that you can control for educational attainment (and opportunities for education as well as class) when accounting for differences in IQ: it's hard to do in practice because social science is complicated and requires ACTUALLY DOING THE SCIENCE so gwern's glib comment is just wrong here clearly we're talking about the impact on IQ of whatever factors we make up to describe IQ, and obviously, and provenly, education (and therefore the ability to get education) is in play here, which gwern's glib rebuttal ignores

georgehemingto1 7y ago

honestly it's disappointing when i see people with training in stats make these comments: I'm a fucking poet if I'm anything at all and yet I find myself drawn into these ridiculous discussions about stats (in which I have no formal training) where I feel like I'm one of the only people in the room who knows how to divy up numbers

georgehemingto1 7y ago

also, to clarify things a bit: in my first comment i got a bit mixed up (fuck yeah lifelong learning disability) trying to make my point about lifelong attainment and the IQ analogy and made it sound like the paper in question was about IQ, which is incorrect. The point stands however: controlling for educational background is a thing you can do in such a study as according to the factors I pointed to, and gwern's glib comment is wrong

gwern 7y ago
No, your comment was glib. You suggest controlling for variables which are going to create overcontrolling and the sociologist's fallacy, and you know perfectly well that family background is not independent of IQ and attenuates the relationship and your suggestion creates only an uninterpretable mishmash of a marginal effect which includes some causal effects but quite arbitrarily excludes others which run through controlled variables. I did not choose my example of education/IQ/income casually, but picked one which is very common in the literature which people commit because they think they are being rigorous when in fact they're only fucking up their own results and making them less meaningful. One's intelligence is not independent of one's family or genetic background! This matters to testing and things which can change! (Because it gives you residual confounding, if nothing else.) By the way, the proper way to do what you may or may not want to do (because your comments are so glib it's unclear to me what DAG you have in mind or what causal effect you want) is sibling controls, not family background controls, and you will be unhappy as to the results when they have been done.
- georgehemingto1 7y ago
  This is a total nothingburger, a long paragraph attacking a strawman of an argument you haven't even solicited the details of
  - gwern 7y ago
    The burden is on the person saying that 'such-and-such a thing should be done' to explain themselves. You failed to do that, and you have still failed to do that, and you spend your comments complaining about being criticized, so your comments remain nothingburgers.
- georgehemingto1 7y ago
  I mean jesus, of course family background is not independent of IQ, but assessments of family background can obviously be informative in understanding how we get from there to here. To say that, quote, "One's intelligence is not independent of one's family or genetic background" is a flaccid truism. Nonetheless particular comparisons within assessments of family background can obviously in principle be informative even if they are hard to do. Your suggestion - with this fringe talk about "the sociologist's fallacy - is to eliminate any explanation from any study which does not assign success to invariant psychological metrics.