5 ms·

Does a "simple algebra test" correlate with IQ? Does "programming aptitude" (whatever the hell that is meant to be) correlate with IQ? The headline would then f

by Paul_S 12y ago

Does a "simple algebra test" correlate with IQ? Does "programming aptitude" (whatever the hell that is meant to be) correlate with IQ? The headline would then follow. I know this is a socially unacceptable opinion but I think those correlations exist.

mgirdley 12y ago
Codeup CEO here. We actually did some brief testing with classical IQ tests and saw two problems: 1) IQ tests are really long and expensive, so difficult to implement 2) They didn't correlate with performance as well as algebra.
- Paul_S 12y ago
  I've read your answer about your methodology of gauging "programming aptitude" and I'm worried it might be susceptible to your judges' biases - it being purely subjective. I'll put more faith in your point number 2 if you come up with a more objective measure.
  - mgirdley 12y ago
    Agreed that an objective measurement would be ideal and this is second best. However, we've had a hard time creating or locating an objective measurement of general programming skill. As demonstrated by the industry's non-use of standardized tests for programming, I don't think anyone else has either. If you have a lead on something, would love to see it.
    - strict9 12y ago
      Trying to create an accurate measurement of programming skill is far more difficult than looking for correlation on tests compared with some other subjective measure. This misguided attempt at instrumenting "skill" could prevent highly capable individuals from entering the field who don't have a solid math foundation, for societal or other reasons.
      - sdenton4 12y ago
        OTOH, identifying a decent mathematical thinking background as a predictor of CS potential is a pretty big win at a societal level. Teaching math is relatively cheap; you don't even have to buy computers...
- murbard2 12y ago
  How do you know they didn't correlate with performance as well as algebra if they were too difficult to implement?
  - rhiever 12y ago
    Because they tried them, then decided to not continue trying them because they didn't correlate as well and were difficult and expensive to implement.
    - murbard2 12y ago
      There main test has 60 datapoints... how many tries do you think they made exactly before deciding they didn't correlate as well? Yes, IQ tests are expensive, but if you buy one test, the marginal cost of giving to an extra student is basically 0. I suspect it wasn't tested at all.
      - mzl 12y ago
        A real IQ test takes at least 2 hours of 1-on-1 time with a psycholigist to administer, so in the event that someone actually wants to do real iq testing the marginal cost is quite high.
        murbard2 12y ago
        That's what psychometricians want you too believe. There is no scientific reason it needs to be administered by a human or be so long.
        barry-cotter 12y ago
        The ASVAB, GRE, LSAT, MCAT, and SAT all have correlations with psychologist administered IQ tests of around 0.9. WORDSUM, which is a ten word vocabulary test has a correlation of 0.71[1]. [1]Every time I use the WORDSUM variable from the GSS people will complain that a score on a 10-question vocabulary test is not a good measure of intelligence. The reality is that “good” is too imprecise a term. The correlation between adult IQ and WORDSUM = 0.71. The source for this number is a 1980 paper, The Enduring Effects of Education on Verbal Skills. I’ve reproduced the relevant table… The Enduring Effect of Education on Verbal Skills http://www.jstor.org/discover/10.2307/2112492?uid=3737800&uid=2&uid=4&sid=21104842098621 http://www.jstor.org/discover/10.2307/2112492?uid=3737800&ui...
- fredophile 12y ago
  I remember reading about a similar but different test that had good results. I can't find the source so this is just from what I remember. Before starting a course the students were shown short snippets of code and asked simple questions about it. They found that consistent answers were a predictor of who would do well. The theory was that people who are able to construct a mental model for the problems ended up being good at programming tasks, even if the models they had made didn't match what code actually does. Have you heard of or tried this approach?
  - brians 12y ago
    I think you're remembering some papers from Dehnadi and Bornat, starting with http://www.eis.mdx.ac.uk/research/PhDArea/saeed/paper1.pdf http://www.eis.mdx.ac.uk/research/PhDArea/saeed/paper1.pdf
    - fredophile 12y ago
      That looks like the paper I was thinking of. Thanks for including te link.
      - rebelshrug 12y ago
        http://retractionwatch.com/2014/07/18/the-camel-doesnt-have-two-humps-programming-aptitude-test-canned-for-overzealous-conclusion/ http://retractionwatch.com/2014/07/18/the-camel-doesnt-have-...
joe_the_user 12y ago
Aside from being wrong and indeed often disparaged, this still common attitude has the unfortunate effect of helping to exclude those who can make an excellent contribution to a programming team but who approach programming from a different point of view - for example, I have a friend who's working quite competently in the field and has general block with respect to math but came to programming through music and graphic design.
delluminatus 12y ago
A lot of things "correlate with IQ" to varying degrees. Education level, length of sleep, size of vocabulary, etc. I don't understand why the relationships you suggested would be socially unacceptable. But just because those two things correlate with IQ (which is probably true, so let's just grant them as true even without evidence for the sake of argument), that does not imply that they are sufficiently strongly correlated with each other to be interesting for hiring purposes. To see my point, consider that being red is correlated with being purple, and being blue is correlated with being purple, but being red is not correlated with being blue. (at least not without evidence).
spacehome 12y ago
Not quite. X can correlate with Y, and Y can correlate with Z without X and Z having a correlation. As a simple toy example let X and Z be independently taken from the standard uniform distribution, and let Y = X + Z.
- Russell91 12y ago
  fwiw, correlation actually is transitive if strong enough. This is obvious if corr(x,y) = 1 and corr(y,z) = 1, then corr(x,z) = 1. More generally, you can get this constraint by looking at the minimal value a for which the matrix [[1, corr(x,y), a], [corr(x,y), 1, corr(y, z)], [a, corr(y,z), 1]] remains positive semidefinite to find the minimal correlation of x and z given that of (x,y) and (y,z). I mention this because in this case the correlations may be strong enough so that the minimal a is positive, and because it's important to understand exactly when our intuition about transitivity of correlation breaks down. For example, when x,y and y,z are perfectly correlated, we can see that corr(x,z) must be greater than 0.9 from: http://www.wolframalpha.com/input/?i=%5B%5B1%2C1%2C0.9%5D%2C%5B1%2C1%2C1%5D%2C%5B0.9%2C1%2C1%5D%5D http://www.wolframalpha.com/input/?i=%5B%5B1%2C1%2C0.9%5D%2C...
fsloth 12y ago
I don't think the existence of those correlations depends on what you think. Har har. Suggesting a correlation is a statistical hypothesis that is testable but not really debatable. I don't think such a hypothesis is awkward but I doubt high IQ as an excellent predictor for an individuals performance in tasks that are not IQ tests. As a layman when it comes to psychology I think IQ measures some things but the way the brain works, no real world task a person does is exactly like those IQ tests. A brain is not a CPU and an IQ is not its clock frequency.
jimmaswell 12y ago
>"programming aptitude" (whatever the hell that is meant to be) What's not clear about the term?
- kasey_junk 12y ago
  We as an industry have no objective measure for what is and is not a good program. Therefore, we have no objective measure for who is good at programming.
  - Retra 12y ago
    If you ask me, a good program is one that is documented accurately and thoroughly. So a good programmer is one who cares enough to document well, and has enough skill to do it with precision, speed, and with good judgement. Of course, being a good scientist or mathematician requires the same skills: know who you are talking to, what they already understand, and why your work matters to them. And be obsessive about doing it the right way.
    - kasey_junk 12y ago
      There is virtually no evidence that program documentation leads to "better" programs. In fact, there are many people that will maintain that the need for code documentation is evidence of a "bad" program. Regardless of whether you subscribe to this theory or the opposite, there is no "objective" way to differentiate between these 2 spectrums.
      - deleted 12y ago
        [deleted]
      - Retra 12y ago
        No evidence? No evidence for a _definition_? I am telling you what a good definition of a good program is. Of course there is no evidence that my definition is the same as your (unstated) definition. You can disagree, but this has nothing to do with evidence.

by Paul_S 12y ago

mgirdley 12y ago

Codeup CEO here. We actually did some brief testing with classical IQ tests and saw two problems: 1) IQ tests are really long and expensive, so difficult to implement 2) They didn't correlate with performance as well as algebra.

Paul_S 12y ago
I've read your answer about your methodology of gauging "programming aptitude" and I'm worried it might be susceptible to your judges' biases - it being purely subjective. I'll put more faith in your point number 2 if you come up with a more objective measure.
- mgirdley 12y ago
  Agreed that an objective measurement would be ideal and this is second best. However, we've had a hard time creating or locating an objective measurement of general programming skill. As demonstrated by the industry's non-use of standardized tests for programming, I don't think anyone else has either. If you have a lead on something, would love to see it.
  - strict9 12y ago
    Trying to create an accurate measurement of programming skill is far more difficult than looking for correlation on tests compared with some other subjective measure. This misguided attempt at instrumenting "skill" could prevent highly capable individuals from entering the field who don't have a solid math foundation, for societal or other reasons.
    - sdenton4 12y ago
      OTOH, identifying a decent mathematical thinking background as a predictor of CS potential is a pretty big win at a societal level. Teaching math is relatively cheap; you don't even have to buy computers...
murbard2 12y ago
How do you know they didn't correlate with performance as well as algebra if they were too difficult to implement?
- rhiever 12y ago
  Because they tried them, then decided to not continue trying them because they didn't correlate as well and were difficult and expensive to implement.
  - murbard2 12y ago
    There main test has 60 datapoints... how many tries do you think they made exactly before deciding they didn't correlate as well? Yes, IQ tests are expensive, but if you buy one test, the marginal cost of giving to an extra student is basically 0. I suspect it wasn't tested at all.
    - mzl 12y ago
      A real IQ test takes at least 2 hours of 1-on-1 time with a psycholigist to administer, so in the event that someone actually wants to do real iq testing the marginal cost is quite high.
      - murbard2 12y ago
        That's what psychometricians want you too believe. There is no scientific reason it needs to be administered by a human or be so long.
      - barry-cotter 12y ago
        The ASVAB, GRE, LSAT, MCAT, and SAT all have correlations with psychologist administered IQ tests of around 0.9. WORDSUM, which is a ten word vocabulary test has a correlation of 0.71[1]. [1]Every time I use the WORDSUM variable from the GSS people will complain that a score on a 10-question vocabulary test is not a good measure of intelligence. The reality is that “good” is too imprecise a term. The correlation between adult IQ and WORDSUM = 0.71. The source for this number is a 1980 paper, The Enduring Effects of Education on Verbal Skills. I’ve reproduced the relevant table… The Enduring Effect of Education on Verbal Skills http://www.jstor.org/discover/10.2307/2112492?uid=3737800&uid=2&uid=4&sid=21104842098621 http://www.jstor.org/discover/10.2307/2112492?uid=3737800&ui...
fredophile 12y ago
I remember reading about a similar but different test that had good results. I can't find the source so this is just from what I remember. Before starting a course the students were shown short snippets of code and asked simple questions about it. They found that consistent answers were a predictor of who would do well. The theory was that people who are able to construct a mental model for the problems ended up being good at programming tasks, even if the models they had made didn't match what code actually does. Have you heard of or tried this approach?
- brians 12y ago
  I think you're remembering some papers from Dehnadi and Bornat, starting with http://www.eis.mdx.ac.uk/research/PhDArea/saeed/paper1.pdf http://www.eis.mdx.ac.uk/research/PhDArea/saeed/paper1.pdf
  - fredophile 12y ago
    That looks like the paper I was thinking of. Thanks for including te link.
    - rebelshrug 12y ago
      http://retractionwatch.com/2014/07/18/the-camel-doesnt-have-two-humps-programming-aptitude-test-canned-for-overzealous-conclusion/ http://retractionwatch.com/2014/07/18/the-camel-doesnt-have-...

joe_the_user 12y ago

Aside from being wrong and indeed often disparaged, this still common attitude has the unfortunate effect of helping to exclude those who can make an excellent contribution to a programming team but who approach programming from a different point of view - for example, I have a friend who's working quite competently in the field and has general block with respect to math but came to programming through music and graphic design.

delluminatus 12y ago

A lot of things "correlate with IQ" to varying degrees. Education level, length of sleep, size of vocabulary, etc. I don't understand why the relationships you suggested would be socially unacceptable. But just because those two things correlate with IQ (which is probably true, so let's just grant them as true even without evidence for the sake of argument), that does not imply that they are sufficiently strongly correlated with each other to be interesting for hiring purposes. To see my point, consider that being red is correlated with being purple, and being blue is correlated with being purple, but being red is not correlated with being blue. (at least not without evidence).

spacehome 12y ago

Not quite. X can correlate with Y, and Y can correlate with Z without X and Z having a correlation. As a simple toy example let X and Z be independently taken from the standard uniform distribution, and let Y = X + Z.

Russell91 12y ago
fwiw, correlation actually is transitive if strong enough. This is obvious if corr(x,y) = 1 and corr(y,z) = 1, then corr(x,z) = 1. More generally, you can get this constraint by looking at the minimal value a for which the matrix [[1, corr(x,y), a], [corr(x,y), 1, corr(y, z)], [a, corr(y,z), 1]] remains positive semidefinite to find the minimal correlation of x and z given that of (x,y) and (y,z). I mention this because in this case the correlations may be strong enough so that the minimal a is positive, and because it's important to understand exactly when our intuition about transitivity of correlation breaks down. For example, when x,y and y,z are perfectly correlated, we can see that corr(x,z) must be greater than 0.9 from: http://www.wolframalpha.com/input/?i=%5B%5B1%2C1%2C0.9%5D%2C%5B1%2C1%2C1%5D%2C%5B0.9%2C1%2C1%5D%5D http://www.wolframalpha.com/input/?i=%5B%5B1%2C1%2C0.9%5D%2C...

fsloth 12y ago

I don't think the existence of those correlations depends on what you think. Har har. Suggesting a correlation is a statistical hypothesis that is testable but not really debatable. I don't think such a hypothesis is awkward but I doubt high IQ as an excellent predictor for an individuals performance in tasks that are not IQ tests. As a layman when it comes to psychology I think IQ measures some things but the way the brain works, no real world task a person does is exactly like those IQ tests. A brain is not a CPU and an IQ is not its clock frequency.

jimmaswell 12y ago

>"programming aptitude" (whatever the hell that is meant to be) What's not clear about the term?

kasey_junk 12y ago
We as an industry have no objective measure for what is and is not a good program. Therefore, we have no objective measure for who is good at programming.
- Retra 12y ago
  If you ask me, a good program is one that is documented accurately and thoroughly. So a good programmer is one who cares enough to document well, and has enough skill to do it with precision, speed, and with good judgement. Of course, being a good scientist or mathematician requires the same skills: know who you are talking to, what they already understand, and why your work matters to them. And be obsessive about doing it the right way.
  - kasey_junk 12y ago
    There is virtually no evidence that program documentation leads to "better" programs. In fact, there are many people that will maintain that the need for code documentation is evidence of a "bad" program. Regardless of whether you subscribe to this theory or the opposite, there is no "objective" way to differentiate between these 2 spectrums.
    - deleted 12y ago
      [deleted]
    - Retra 12y ago
      No evidence? No evidence for a _definition_? I am telling you what a good definition of a good program is. Of course there is no evidence that my definition is the same as your (unstated) definition. You can disagree, but this has nothing to do with evidence.