5 ms·

“A high test score tells you only that a person has high ability and is a good match for that particular test at that point in time,” says Matthews. I have nev

“A high test score tells you only that a person has high ability and is a good match for that particular test at that point in time,” says Matthews. I have never understood the intention behind this type of statement (which I've seen many variants of commonly espoused). Let X be the event "got a high test score at a particular test at a certain point in time." Is Matthews saying X = X (an uninteresting tautology)? Or is he claiming that X is uncorrelated or not predictive of other variables of interest (career success, ability on other task) which is clearly empirically false.

dredmorbius 10y ago
My read is that expertise carries with it specificity. If you happen to be very good at some X[3], but a test is only effectively measuring ability in X[5], where 3 & 5 are some arbitrary specialisations of a capability X, then your test score may not be particularly high. This depends on just how specific a skill is, whether or not the particular subject is a polymath or specialist, and much more. To take an example from athletics, Lance Armstrong was a world-leading cyclist (with, it turns out, the advantage of doping). When he entered into marathon events -- for which his light body frame is generally still an advantage -- he placed respectably, but not as one of the very top finishers. A skills or abilities test for Armstrong had to be specific to his ~60 km cycling distance specialisation. There is some crossover of many skills, but the deep specialisation, ingrained structures, algorithmic knowledge, solution sapces, bags-of-tricks, etc., that are associated with being best in class (not merely "very good") can be quite complex.
- blahblah3 10y ago
  That's a great example and I don't disagree with you. The way I think about "intelligence" is not to mull over what it "really means" but to think about lots of common "intellectual" tasks (in which aptitude can be measured) that people are interested in: reading, writing, math, programming, solving puzzles, etc. Or ideally, theoretical "potential" in each of these tasks, since one person may put more or less effort into another field. What's supported by research and anectodal evidence is that if you did principal components analysis on these measured "aptitudes", there would be a principal component explaining a lot of the variation. A good test can try to measure this principal component. But when you're talking about the very highest levels of specific tasks like cycling or swimming, it would be reasonable to expect that amongst the population of already "elite athletes" there would be less of a dominant first principal component
  - dredmorbius 10y ago
    Thanks. There are different definitions of intelligence. One I like (though I don't quite find it comprehensive) is David Krakauer (of Santa Fe Institute), "Intelligence is Search". That is: it's a search across an n-dimensional space for a least-cost function. There's the question of "function for what?", and I'm thinking here something which allows for useful prediction or outmaneuvering an adversary (as with chess). This gives a set of related or subroutine functions which are required, including many on your list: information acquisitionprocessing, and output. An interesting concept I ran across in the past year or so is that of the medieval curriculum, comprised of the trivium and the quadrivium. The first is further broken down into grammar, logic, and rhetoric, or if you prefer, input (and framing), processing, and output. This tickles my brain. Another element I'm familiar with is that there comes a point in any skill at which it's somewhat less the things you get right as what you avoid getting wrong that matters. A champion athlete has strength or stamina, but also optimised physiology (Michael Phelps's long arms and torso, short legs, and huge feet, for example, are swimming gold, QED), and not screwing anything up in the event. Nailing starts, turns, finishes, and technique, not getting in your own way. Further pondering intelligence also brings into the fold what knowledge, wisdom, correctness, and quality are. Interesting questions.

dredmorbius 10y ago

My read is that expertise carries with it specificity. If you happen to be very good at some X[3], but a test is only effectively measuring ability in X[5], where 3 & 5 are some arbitrary specialisations of a capability X, then your test score may not be particularly high. This depends on just how specific a skill is, whether or not the particular subject is a polymath or specialist, and much more. To take an example from athletics, Lance Armstrong was a world-leading cyclist (with, it turns out, the advantage of doping). When he entered into marathon events -- for which his light body frame is generally still an advantage -- he placed respectably, but not as one of the very top finishers. A skills or abilities test for Armstrong had to be specific to his ~60 km cycling distance specialisation. There is some crossover of many skills, but the deep specialisation, ingrained structures, algorithmic knowledge, solution sapces, bags-of-tricks, etc., that are associated with being best in class (not merely "very good") can be quite complex.

blahblah3 10y ago
That's a great example and I don't disagree with you. The way I think about "intelligence" is not to mull over what it "really means" but to think about lots of common "intellectual" tasks (in which aptitude can be measured) that people are interested in: reading, writing, math, programming, solving puzzles, etc. Or ideally, theoretical "potential" in each of these tasks, since one person may put more or less effort into another field. What's supported by research and anectodal evidence is that if you did principal components analysis on these measured "aptitudes", there would be a principal component explaining a lot of the variation. A good test can try to measure this principal component. But when you're talking about the very highest levels of specific tasks like cycling or swimming, it would be reasonable to expect that amongst the population of already "elite athletes" there would be less of a dominant first principal component
- dredmorbius 10y ago
  Thanks. There are different definitions of intelligence. One I like (though I don't quite find it comprehensive) is David Krakauer (of Santa Fe Institute), "Intelligence is Search". That is: it's a search across an n-dimensional space for a least-cost function. There's the question of "function for what?", and I'm thinking here something which allows for useful prediction or outmaneuvering an adversary (as with chess). This gives a set of related or subroutine functions which are required, including many on your list: information acquisitionprocessing, and output. An interesting concept I ran across in the past year or so is that of the medieval curriculum, comprised of the trivium and the quadrivium. The first is further broken down into grammar, logic, and rhetoric, or if you prefer, input (and framing), processing, and output. This tickles my brain. Another element I'm familiar with is that there comes a point in any skill at which it's somewhat less the things you get right as what you avoid getting wrong that matters. A champion athlete has strength or stamina, but also optimised physiology (Michael Phelps's long arms and torso, short legs, and huge feet, for example, are swimming gold, QED), and not screwing anything up in the event. Nailing starts, turns, finishes, and technique, not getting in your own way. Further pondering intelligence also brings into the fold what knowledge, wisdom, correctness, and quality are. Interesting questions.