3 ms·

It is because we have a more nuanced take on what intelligence is. We don’t think that intelligence is something that you can measure with a pen and paper test.

It is because we have a more nuanced take on what intelligence is. We don’t think that intelligence is something that you can measure with a pen and paper test. Being smart also includes stuff like “reading the room”, being empathetic, knowing when to quit, etc. If there is a general intelligence—which I very much doubt—our current tests don’t even come close to measuring it.

moonchrome 5y ago
>our current tests don’t even come to measuring it. Then why do they correlate with outcomes so well ?
- runarberg 5y ago
  Maybe because they are designed to and because of selection bias. These tests are expected to correlate, if they don’t they are considered a bad test and is thrown into obscurity (but more likely fixed until it does).
  - moonchrome 5y ago
    But if they didn't even come close to measuring it then there would be no correlation ? It's not like these tests are applied retroactively to explain past performances - they offer actual predictive value. Selection bias can account for some of it but there are ways to test if this and from what I've seen it still holds up.
    - runarberg 5y ago
      I’m not disputing that these tests measure something, they obviously do. However what they are measuring is not what most people consider a satisfactory definition of intelligence. In fact most psychometricians simply give up and define intelligence based on what these tests measure (i.e. operation definition). Without justification that is very poor science.
    - plorkyeran 5y ago
      SAT scores would be positively correlated with success even if they were entirely random as long as universities continued to use them as a factor in admissions. It's sort of a corollary of Goodhart's law that it is difficult to assess the quality of a measure while it is being used as a target.
- jcranmer 5y ago
  IIRC, the strongest predictive factor for SAT scores is... parental income. So if what these tests are measuring is not a "general factor of intelligence" but "quality of home environment", you can readily see how that could correlate well with outcomes like "success in later life."
  - moonchrome 5y ago
    SAT mixes in intelligence with education so it's bound to skew more towards people with access, but it's also correlated with IQ, and IQ has significant heritability as well (things like separated twin studies have shown the genetic component quite clearly). So it's probably a double whammy there - good environment and good genetics.
cm2012 5y ago
What we can measure though is useful since it correlates to so many things. Controlling for every factor we can think of, parental income, school quality, etc etc, people who have high IQs when they're young have better outcomes as adults on average - in income, in happiness surveys, lower divorce rate, etc.
- daenz 5y ago
  I think people aren't aware that controlling for every factor is what distills the idea of general intelligence. That even after you account for everything you can think of, there is some quality that allows people to perform better mentally on many different types of tasks.

moonchrome 5y ago

>our current tests don’t even come to measuring it. Then why do they correlate with outcomes so well ?

runarberg 5y ago
Maybe because they are designed to and because of selection bias. These tests are expected to correlate, if they don’t they are considered a bad test and is thrown into obscurity (but more likely fixed until it does).
- moonchrome 5y ago
  But if they didn't even come close to measuring it then there would be no correlation ? It's not like these tests are applied retroactively to explain past performances - they offer actual predictive value. Selection bias can account for some of it but there are ways to test if this and from what I've seen it still holds up.
  - runarberg 5y ago
    I’m not disputing that these tests measure something, they obviously do. However what they are measuring is not what most people consider a satisfactory definition of intelligence. In fact most psychometricians simply give up and define intelligence based on what these tests measure (i.e. operation definition). Without justification that is very poor science.
  - plorkyeran 5y ago
    SAT scores would be positively correlated with success even if they were entirely random as long as universities continued to use them as a factor in admissions. It's sort of a corollary of Goodhart's law that it is difficult to assess the quality of a measure while it is being used as a target.
jcranmer 5y ago
IIRC, the strongest predictive factor for SAT scores is... parental income. So if what these tests are measuring is not a "general factor of intelligence" but "quality of home environment", you can readily see how that could correlate well with outcomes like "success in later life."
- moonchrome 5y ago
  SAT mixes in intelligence with education so it's bound to skew more towards people with access, but it's also correlated with IQ, and IQ has significant heritability as well (things like separated twin studies have shown the genetic component quite clearly). So it's probably a double whammy there - good environment and good genetics.

cm2012 5y ago

What we can measure though is useful since it correlates to so many things. Controlling for every factor we can think of, parental income, school quality, etc etc, people who have high IQs when they're young have better outcomes as adults on average - in income, in happiness surveys, lower divorce rate, etc.

daenz 5y ago
I think people aren't aware that controlling for every factor is what distills the idea of general intelligence. That even after you account for everything you can think of, there is some quality that allows people to perform better mentally on many different types of tasks.