6 ms·

The whole IQ is only one measure argument really misses the main lesson from psychometrics. It’s entirely well known that IQ test taking is a specific skill. Ho

by User23 3y ago

The whole IQ is only one measure argument really misses the main lesson from psychometrics. It’s entirely well known that IQ test taking is a specific skill. However it’s also known that there is a g factor that predicts about 50% of performance on basically all tasks with a cognitive component. The other 50% is indeed skill specific. So someone with a slightly above average g factor could still be a world class composer or quarterback. Contrariwise someone with an extremely high g might be very good at many different tasks, but not world class at any. So when one talks about measuring intelligence in a general sense, one ought to be talking about g.

tptacek 3y ago
On the importance or supposed profundity of the g factor, here's Cosma Shalizi: http://bactra.org/weblog/523.html http://bactra.org/weblog/523.html (If you like, skip to "Correlations explain g, not the other way around").
- User23 3y ago
  Interesting read. I’m not inclined to critique it much beyond observing that the author clearly set out with an objective in mind and argued well to that end. My brief response is that the fact that we can find similar patterns in noise doesn’t actually show that the discovery of patterns indicates the data is noise. And of course intelligence is a real and observable thing. Some people really are considerably smarter than others. They have better recall, learn faster, and are able to reach correct conclusions either more quickly or with less information than less intelligent persons. Reducing that difference to a single factor almost certainly is a simplification, but just as we can usefully talk about height without getting caught up on difference in leg, torso, and neck length, we can talk about intelligence similarly.
  - tptacek 3y ago
    The article addresses this: for instance, if you were to create a "sound mind and body" score that included "hot or not" results, and the results of the Presidential fitness test, you would inevitably come up with a score that was even more predictive of life outcomes than IQ or g.
    - User23 3y ago
      Physical attractiveness correlates with many good things like fertility and genetic health. Not to mention beautiful people benefit from the halo effect. The causality there though is likely that the relative superiority of attractive persons causes the halo effect to exist. There’s a pretty obvious set of mechanisms that offer causal reasons why attractiveness leads to better outcomes. As for the presidential fitness exam, well that’s true but quite facile. Much like how people who have taken the Wonderlic test are considerably more likely than the general populace to be or have been in the NFL. In fact, I wouldn’t be surprised if taking aptitude tests at all regardless of score predicts above average outcomes because of a selection effect.
      - tptacek 3y ago
        The point is that you can drag a lot of unrelated things together and get an increasingly predictive metric, despite the underlying factors being unrelated, which simply means that "being predictive" is not by itself dispositive of some underlying mechanism. It doesn't mean prediction isn't real! The problem is people going from "prediction" to "inferred ground truth" --- which is exactly what happens in the "g" discussions.
        User23 3y ago
        > which simply means that "being predictive" is not by itself dispositive of some underlying mechanism. Right. I don’t see how any statistical analysis can tell you if a mechanism exists or what it is. Quite the opposite. The better you understand the mechanism of the process that generates your data points the higher quality your statistical analysis will be. And in the case of intelligence we know there must be an actual real underlying factor because some people are observably smarter than others.
        tptacek 3y ago
        I don't think anybody is arguing that people aren't observably smarter than other people; the question is about the metrics we're using, which, I mean, an equally vivid (and equally non-rigorous) argument would be: if you really buy into the IQ/g story, go hang out with a bunch of Mensa people and see how long you hold onto that belief.
rramadass 3y ago
The entire field of Psychometrics is suspect and is a pseudoscience. It came into existence in the early days when ideas from Social Darwinism, Racism and Eugenics were mainstream in the Western world. People ascribed an importance factor to traits which they considered successful in their culture/society/environment and then started making universal and general inferences from it without regard to its validity. These were then codified as tests, published and propagated to the detriment of all. Today we know better; Humans are highly adaptable and everything only exists in the context of a specific environmental/cultural/social context. Genetics vs. Epigenetics, Nature vs. Nurture, Nutrition, Economic strata etc. are all inputs when it comes to our "fitness" for a particular task. We play "roles" which are context-dependent from which you cannot generalize to the "whole person". Psychometrics has been called a "Pathological Science" - https://en.wikipedia.org/wiki/Pathological_science https://en.wikipedia.org/wiki/Pathological_science This article named The Myth of Measurement brings a bunch of research together to show why - https://paulenglert.com/the-myth-of-measurement/ https://paulenglert.com/the-myth-of-measurement/
- concordDance 3y ago
  If the field makes testable predictions that prove true then it cannot be a pseudo science. And indeed, one can predict with accuracy better than chance how well someone will perform in a software developer job (for example) from an IQ test. This same test may do less well in predicting the ability to hunt an antelope across the serengeti, but that still wouldn't make psychometrics pseudoscience. Also, if there wasn't a g factor that would be fairly weird, as there are numerous little slightly deleterious mutations one could have that can minorly affect neuronal functioning. You'd genuinely expect there to be a decent correlation here.
  - rramadass 3y ago
    When the characteristics that a test is supposed to measure are completely ill-defined (there is no concept of a "Measurable Intelligence" independent of a context, leaving aside biological retardation) they are a measure of nothing definitive and any correlations/predictions are subject to bias and/or randomness. Here is Nassim Taleb; 1) IQ is largely a pseudoscientific swindle - https://medium.com/incerto/iq-is-largely-a-pseudoscientific-swindle-f131c101ba39 https://medium.com/incerto/iq-is-largely-a-pseudoscientific-... 2) Fooled by Correlation: Common Misinterpretations in Social "Science" - https://www.academia.edu/39797871/Common_Misapplications_and_Misinterpretations_of_Correlation_in_Social_Science_ https://www.academia.edu/39797871/Common_Misapplications_and... 3) Correlation videos - https://nassimtaleb.org/tag/correlation/ https://nassimtaleb.org/tag/correlation/ See also : a) The Pseudoscience of Psychometry and The Bell Curve - https://www.jstor.org/stable/2967209 https://www.jstor.org/stable/2967209 b) Pseudoscience and Mental Ability: The Origins and Fallacies of the IQ Controversy. - https://eric.ed.gov/?id=ED164662 https://eric.ed.gov/?id=ED164662

by User23 3y ago

tptacek 3y ago

On the importance or supposed profundity of the g factor, here's Cosma Shalizi: http://bactra.org/weblog/523.html http://bactra.org/weblog/523.html (If you like, skip to "Correlations explain g, not the other way around").

User23 3y ago
Interesting read. I’m not inclined to critique it much beyond observing that the author clearly set out with an objective in mind and argued well to that end. My brief response is that the fact that we can find similar patterns in noise doesn’t actually show that the discovery of patterns indicates the data is noise. And of course intelligence is a real and observable thing. Some people really are considerably smarter than others. They have better recall, learn faster, and are able to reach correct conclusions either more quickly or with less information than less intelligent persons. Reducing that difference to a single factor almost certainly is a simplification, but just as we can usefully talk about height without getting caught up on difference in leg, torso, and neck length, we can talk about intelligence similarly.
- tptacek 3y ago
  The article addresses this: for instance, if you were to create a "sound mind and body" score that included "hot or not" results, and the results of the Presidential fitness test, you would inevitably come up with a score that was even more predictive of life outcomes than IQ or g.
  - User23 3y ago
    Physical attractiveness correlates with many good things like fertility and genetic health. Not to mention beautiful people benefit from the halo effect. The causality there though is likely that the relative superiority of attractive persons causes the halo effect to exist. There’s a pretty obvious set of mechanisms that offer causal reasons why attractiveness leads to better outcomes. As for the presidential fitness exam, well that’s true but quite facile. Much like how people who have taken the Wonderlic test are considerably more likely than the general populace to be or have been in the NFL. In fact, I wouldn’t be surprised if taking aptitude tests at all regardless of score predicts above average outcomes because of a selection effect.
    - tptacek 3y ago
      The point is that you can drag a lot of unrelated things together and get an increasingly predictive metric, despite the underlying factors being unrelated, which simply means that "being predictive" is not by itself dispositive of some underlying mechanism. It doesn't mean prediction isn't real! The problem is people going from "prediction" to "inferred ground truth" --- which is exactly what happens in the "g" discussions.
      - User23 3y ago
        > which simply means that "being predictive" is not by itself dispositive of some underlying mechanism. Right. I don’t see how any statistical analysis can tell you if a mechanism exists or what it is. Quite the opposite. The better you understand the mechanism of the process that generates your data points the higher quality your statistical analysis will be. And in the case of intelligence we know there must be an actual real underlying factor because some people are observably smarter than others.
        tptacek 3y ago
        I don't think anybody is arguing that people aren't observably smarter than other people; the question is about the metrics we're using, which, I mean, an equally vivid (and equally non-rigorous) argument would be: if you really buy into the IQ/g story, go hang out with a bunch of Mensa people and see how long you hold onto that belief.

rramadass 3y ago

The entire field of Psychometrics is suspect and is a pseudoscience. It came into existence in the early days when ideas from Social Darwinism, Racism and Eugenics were mainstream in the Western world. People ascribed an importance factor to traits which they considered successful in their culture/society/environment and then started making universal and general inferences from it without regard to its validity. These were then codified as tests, published and propagated to the detriment of all. Today we know better; Humans are highly adaptable and everything only exists in the context of a specific environmental/cultural/social context. Genetics vs. Epigenetics, Nature vs. Nurture, Nutrition, Economic strata etc. are all inputs when it comes to our "fitness" for a particular task. We play "roles" which are context-dependent from which you cannot generalize to the "whole person". Psychometrics has been called a "Pathological Science" - https://en.wikipedia.org/wiki/Pathological_science https://en.wikipedia.org/wiki/Pathological_science This article named The Myth of Measurement brings a bunch of research together to show why - https://paulenglert.com/the-myth-of-measurement/ https://paulenglert.com/the-myth-of-measurement/

concordDance 3y ago
If the field makes testable predictions that prove true then it cannot be a pseudo science. And indeed, one can predict with accuracy better than chance how well someone will perform in a software developer job (for example) from an IQ test. This same test may do less well in predicting the ability to hunt an antelope across the serengeti, but that still wouldn't make psychometrics pseudoscience. Also, if there wasn't a g factor that would be fairly weird, as there are numerous little slightly deleterious mutations one could have that can minorly affect neuronal functioning. You'd genuinely expect there to be a decent correlation here.
- rramadass 3y ago
  When the characteristics that a test is supposed to measure are completely ill-defined (there is no concept of a "Measurable Intelligence" independent of a context, leaving aside biological retardation) they are a measure of nothing definitive and any correlations/predictions are subject to bias and/or randomness. Here is Nassim Taleb; 1) IQ is largely a pseudoscientific swindle - https://medium.com/incerto/iq-is-largely-a-pseudoscientific-swindle-f131c101ba39 https://medium.com/incerto/iq-is-largely-a-pseudoscientific-... 2) Fooled by Correlation: Common Misinterpretations in Social "Science" - https://www.academia.edu/39797871/Common_Misapplications_and_Misinterpretations_of_Correlation_in_Social_Science_ https://www.academia.edu/39797871/Common_Misapplications_and... 3) Correlation videos - https://nassimtaleb.org/tag/correlation/ https://nassimtaleb.org/tag/correlation/ See also : a) The Pseudoscience of Psychometry and The Bell Curve - https://www.jstor.org/stable/2967209 https://www.jstor.org/stable/2967209 b) Pseudoscience and Mental Ability: The Origins and Fallacies of the IQ Controversy. - https://eric.ed.gov/?id=ED164662 https://eric.ed.gov/?id=ED164662