4 ms·

> There are generally 3 conclusions you can draw from a correlation like this: A directly causes B, B directly causes A, or something else more complex is at pl

> There are generally 3 conclusions you can draw from a correlation like this: A directly causes B, B directly causes A, or something else more complex is at play. ... Hopefully that explains. It's essentially a correlation vs. causation issue. Note that for predictor it does not really matter correlation vs. causation. That matters for intervention. If you have feature A that in the population always causes features B and C, and no other causes them, then presence of B is perfect predictor of C, but intervention that changes B (and not A) does not affect C. On the other hand if feature D causes feature E in 90% cases, and no other causes it, then D is predictor of E with 10% false positives, but intervention on D affects E. It is true that 'perfect causality' predictors, where the set of accounted factors are only causes of a predicted feature, have advantage that they work the same for any subset (or any change) of population. While predictors that ignore some causal factors (like the predictor from the first example that ignores common causal factor) may have vasly different probabilities for subset (or change) of population (when distribution of ignored factors change). But in practice most real-world causal networks are super complex and many real-world tests ignore many causal factors. So it is kind of isolated demand for rigor [1]. [1] https://www.lesswrong.com/posts/fzeoYhKoYPR3tDYFT/beware-isolated-demands-for-rigor https://www.lesswrong.com/posts/fzeoYhKoYPR3tDYFT/beware-iso...

joshuamorton 6y ago
> Note that for predictor it does not really matter correlation vs. causation. That matters for intervention. Yes, but you don't build a model without intending some intervention, so this distinction is irrelevant in the context of applied ML, although it is a true statistical fact. > So it is kind of isolated demand for rigor [1]. I don't see how. The isolated demand for rigor "fallacy" that Scott Alexander talks about is when you ask for rigor to be applied in ways that are bad faith. Let me rephrase my concern: We have lots of evidence that real-world causal networks are super complex and many real-world tests ignore many causal factors. Similarly, we have no a-priori reason to believe that facial structure is correlated with {sexual orientation, innate level of intelligence, innate criminality}. And in fact we have strong reasons to believe that most of the way that those things present is due to cultural influence. So if someone shows up with a groundbreaking study that shows that they can predict some innate attribute based on facial structure, it's likely that they're actually seeing cultural biases (or cultural correlations) and not innate factors (or correlations with genotypic things). In other words, our priors should be that any model in this space is simply discriminating based on stereotypes, not revealing some innate way to "predict" these attributes. The model isn't a predictor but a recognizer, and while semantic, that's a very important distinction.

joshuamorton 6y ago

> Note that for predictor it does not really matter correlation vs. causation. That matters for intervention. Yes, but you don't build a model without intending some intervention, so this distinction is irrelevant in the context of applied ML, although it is a true statistical fact. > So it is kind of isolated demand for rigor [1]. I don't see how. The isolated demand for rigor "fallacy" that Scott Alexander talks about is when you ask for rigor to be applied in ways that are bad faith. Let me rephrase my concern: We have lots of evidence that real-world causal networks are super complex and many real-world tests ignore many causal factors. Similarly, we have no a-priori reason to believe that facial structure is correlated with {sexual orientation, innate level of intelligence, innate criminality}. And in fact we have strong reasons to believe that most of the way that those things present is due to cultural influence. So if someone shows up with a groundbreaking study that shows that they can predict some innate attribute based on facial structure, it's likely that they're actually seeing cultural biases (or cultural correlations) and not innate factors (or correlations with genotypic things). In other words, our priors should be that any model in this space is simply discriminating based on stereotypes, not revealing some innate way to "predict" these attributes. The model isn't a predictor but a recognizer, and while semantic, that's a very important distinction.