3 ms·

If we're being charitable we can say that the core thrust of Taleb's point remains unchallenged, while the details he quantified are either wrong or lacking in

If we're being charitable we can say that the core thrust of Taleb's point remains unchallenged, while the details he quantified are either wrong or lacking in sufficient rigor to be refuted. The author makes good points, but realistically it's a fool's errand trying to disprove tweets using a litany of R^2 incantations. There are important caveats to what the author's done here: 1. Taleb's real point is that IQ is mostly useful for avoiding (extreme) incompetence rather than selecting for competence. Taleb constructed his argument using statistics which are either incorrect or underspecified, which invites the kind of analysis the author made. But that also means the author isn't substantially engaging with what Taleb means, because Taleb is abrasive and doesn't care to be specific. 2. You should be skeptical of any statistical analysis which leads with Pearson's r (or Spearman's rho, or whatever correlative measure) or the associated R^2. It is a top line measure which is useful but incomplete. You may miss out on the nonlinear relationship or nonperiodic relationship entirely, either of which could hold most of the variance. There are also many ways in which the variance of two things can be associated without there being a predictive relationship. Simply stated: as a statistician would tell you, if all we needed was R^2 then there would be no papers, only abstracts. A little bit of statistics can be a dangerous thing. I'm not defending Taleb per se; rather I'm saying that both sides are arguing past each other and we probably shouldn't draw a conclusion from either.

DougBTX 6y ago
> If we're being charitable we can say that the core thrust of Taleb's point remains unchallenged, while the details he quantified are either wrong or lacking in sufficient rigor to be refuted. It seems we could put this a little stronger, as the article claims to refute all the quantifiable evidence for the core point. As far as I can tell, it does so.
candiodari 6y ago
Euhm, the jsmp post certainly seems more rigorous than Taleb, not less. I also don't understand how your first point works. It contradicts itself. Look, either statistics (because this is a problem with statistics, not with IQ specifically) can predict individuals or not. So either IQ is useful in classifying both success and failure. Or it can't be used for either. Generally using statistical values for predicting individual success or failure is, first, very unreliable in the individual case, and second, not considered "just", for good reason. If you statistically show a group of 10 people has a 90% chance of being criminal, it is considered impossibly hard to pick out the innocent from among them using statistics. Therefore the correct way to respond to statistics is to group people by IQ bands, not to give specific individuals advantages (which, granted, is often done). Of course, this will magnify differences over time, which you may not consider socially just. Your second argument essentially states that "measures that don't take time into account ... don't take time into account". It is wrong insofar that IQ usually does take time (time since birth) into account, but nothing else. That's not a coincidence, that's because they tried other factors and they didn't matter, but obviously a tautology is always right. The point is wrong because it's useless, not because it's untrue. There are many other factors that could be evaluated against IQ, and there has been a ton of research that did so, and never found much correlation. And frankly, it boils down to this: "being good at X" correlates across X. That's to a large extent what IQ is. Everybody has seen that incompetence extends beyond any individual skill an individual might or might not have. Research shows that competence does too. It makes sense to some extent: having exceptional skill at one particular thing allows you to advance quicker when it comes to other skills. When those skills are related this is considered "trivially true" (e.g. learning piano to learn violin quicker), but what's harder to see is that this is in reality almost universally true, and this is in fact what our school system turns out to be based upon. Skills you learn in school aren't generally meant to be generally useful, but to allow you to learn "real" skills that can't be effectively learned in school much faster.
- fractionalhare 6y ago
  > So either IQ is useful in classifying both success and failure. No, statistical relationships are not symmetric in general. You shouldn't expect equal absolute predictive power for events two sigma from the mean in opposite directions if the random variable doesn't have a normal distribution. IQ is normally distributed, but many obvious and conventional measures of success are clearly not. > Your second argument essentially states that "measures that don't take time into account ... don't take time into account". This isn't what I'm intending my argument to "essentially state". My second argument is just that drawing conclusions from "top line" statistics like R^2 is dangerous and potentially misleading.
  - runarberg 6y ago
    Do note also that IQ is normally distributed only by design. There is no convinsing evidence that I have seen for any cognitive ability being normally distributed, let alone general intelligence—however they define it. Usually cognitive abilities have a step skew to the right. And even IQ fails the sniff test for normal distributions as you can imagine that the difference between 70 and 100 is far greater then the difference between 100 and 130.
- sokoloff 6y ago
  What does statistics predicting individuals mean? It is well-studied, well-established, and hopefully uncontroversial that adult men are several inches taller on average than adult women. I can use that information to predict a wide variety of things with very high confidence, but I can’t use it to tell you how tall any individual human will become. That doesn’t make it wrong nor useless.
  - candiodari 6y ago
    Exactly that: statistics may tell you in any group of adults, 60% of the men will be taller than the women. It doesn’t tell you anything about which ones will be.

DougBTX 6y ago

> If we're being charitable we can say that the core thrust of Taleb's point remains unchallenged, while the details he quantified are either wrong or lacking in sufficient rigor to be refuted. It seems we could put this a little stronger, as the article claims to refute all the quantifiable evidence for the core point. As far as I can tell, it does so.

candiodari 6y ago

Euhm, the jsmp post certainly seems more rigorous than Taleb, not less. I also don't understand how your first point works. It contradicts itself. Look, either statistics (because this is a problem with statistics, not with IQ specifically) can predict individuals or not. So either IQ is useful in classifying both success and failure. Or it can't be used for either. Generally using statistical values for predicting individual success or failure is, first, very unreliable in the individual case, and second, not considered "just", for good reason. If you statistically show a group of 10 people has a 90% chance of being criminal, it is considered impossibly hard to pick out the innocent from among them using statistics. Therefore the correct way to respond to statistics is to group people by IQ bands, not to give specific individuals advantages (which, granted, is often done). Of course, this will magnify differences over time, which you may not consider socially just. Your second argument essentially states that "measures that don't take time into account ... don't take time into account". It is wrong insofar that IQ usually does take time (time since birth) into account, but nothing else. That's not a coincidence, that's because they tried other factors and they didn't matter, but obviously a tautology is always right. The point is wrong because it's useless, not because it's untrue. There are many other factors that could be evaluated against IQ, and there has been a ton of research that did so, and never found much correlation. And frankly, it boils down to this: "being good at X" correlates across X. That's to a large extent what IQ is. Everybody has seen that incompetence extends beyond any individual skill an individual might or might not have. Research shows that competence does too. It makes sense to some extent: having exceptional skill at one particular thing allows you to advance quicker when it comes to other skills. When those skills are related this is considered "trivially true" (e.g. learning piano to learn violin quicker), but what's harder to see is that this is in reality almost universally true, and this is in fact what our school system turns out to be based upon. Skills you learn in school aren't generally meant to be generally useful, but to allow you to learn "real" skills that can't be effectively learned in school much faster.

fractionalhare 6y ago
> So either IQ is useful in classifying both success and failure. No, statistical relationships are not symmetric in general. You shouldn't expect equal absolute predictive power for events two sigma from the mean in opposite directions if the random variable doesn't have a normal distribution. IQ is normally distributed, but many obvious and conventional measures of success are clearly not. > Your second argument essentially states that "measures that don't take time into account ... don't take time into account". This isn't what I'm intending my argument to "essentially state". My second argument is just that drawing conclusions from "top line" statistics like R^2 is dangerous and potentially misleading.
- runarberg 6y ago
  Do note also that IQ is normally distributed only by design. There is no convinsing evidence that I have seen for any cognitive ability being normally distributed, let alone general intelligence—however they define it. Usually cognitive abilities have a step skew to the right. And even IQ fails the sniff test for normal distributions as you can imagine that the difference between 70 and 100 is far greater then the difference between 100 and 130.
sokoloff 6y ago
What does statistics predicting individuals mean? It is well-studied, well-established, and hopefully uncontroversial that adult men are several inches taller on average than adult women. I can use that information to predict a wide variety of things with very high confidence, but I can’t use it to tell you how tall any individual human will become. That doesn’t make it wrong nor useless.
- candiodari 6y ago
  Exactly that: statistics may tell you in any group of adults, 60% of the men will be taller than the women. It doesn’t tell you anything about which ones will be.