6 ms·

This is interesting data. I liked what Joel Spolsky said a long time ago. Basically that companies should want two types of engineers. 1. You want a few who ar

This is interesting data. I liked what Joel Spolsky said a long time ago. Basically that companies should want two types of engineers. 1. You want a few who are experts in the company's tech stack. 2. The bar for everyone else is just that they're smart and they get things done. I guess the core problem is that we don't have a good objective measure of the latter. (Maybe an IQ test for smarts, if that wasn't political incorrect, but I don't know how you'd show definitively that you "get things done" in an interview.)

tstock55 11y ago
I accepted an offer at IBM a few months ago and had to take an IQ tests of sorts. It was mostly focused around pattern recognition with some algebra problems thrown in. At best it seemed only tangentially related to programming ability.. I'm not sure that attempting to measure whether a candidate is "smart" or not is a good hiring strategy. A good analogy for this type of interviewing is like trying to judge how good someone is at basketball by measuring how tall they are. Is it related? Sure. But it certainly doesn't give you the whole picture and might not be a useful information at all. Watching them play a few games would be a better strategy. Fortunately there a quite a few startups working on this problem. Lytmus is one in particular that I think looks very promising.
marme 11y ago
IQ tests are not only politically incorrect they are illegal in the US. It is illegal discrimination to make hiring decisions based on IQ scores. You can only use a test that shows those with higher scores perform better at the job. A general IQ test can not be shown to do that. Basically you can not ask questions that test skills that would not be used in the job the candidate is applying for. You must be able to defend the metric you use for hiring and that it is not discriminatory. If you give an IQ test that is unrelated to the job and say no one below 100 IQ will be hired you must prove that someone with 101 IQ can do the job but someone with 99 IQ can not properly do the job
- johncolanduoni 11y ago
  > It is illegal discrimination to make hiring decisions based on IQ scores. You can only use a test that shows those with higher scores perform better at the job. There is more scientific evidence that IQ predicts job performance than for most of the more qualitative criterions that employers use in practice. The source of the illegality of using IQ tests is not legislative (intelligence is not a protected class[1]). It comes from the precedent of Griggs v. Duke Power Co., in which the IQ test was being used as a barrier for jobs where IQ was not a good predictor (not programming), and prior to thorough research on the subject. There have been hundreds of studies showing that IQ tests predict job performance (though not super strongly) at the beginning of careers, even though this fades once they have more experience. This[3] meta-analysis approaches these studies very critically, and concludes that many inflate the effect, and that it doesn't validate the IQ test (i.e. there may be non-cognitive reasons for it) but that the correlation still exists. (For what it's worth, I suspect they're right about the non-cognitive part since the correlation has become much stronger over time, particularly around the 1970s. However, when your goal is to test for something and not to explain it, correlation is good enough.) > Basically you can not ask questions that test skills that would not be used in the job the candidate is applying for. ... You must be able to defend the metric you use for hiring and that it is not discriminatory. No, the burden set in Griggs and later elaborated in a modification to the Civil Rights Act is not to prove that a test is not discriminatory at all. Instead for tests that are discriminatory in practice (though not obviously in design, like IQ), you must show that the test is truly indicative of job performance. If this were not the case, you couldn't ask if people went to college, due to the racial disparity in college graduates. Also you can ask things you can't prove are related to job performance if it isn't discriminatory (not that this is really useful). > If you give an IQ test that is unrelated to the job and say no one below 100 IQ will be hired you must prove that someone with 101 IQ can do the job but someone with 99 IQ can not properly do the job Of course you don't have to show that anyone who fails your test will 100% be worse on a case by case basis than someone who passes. If that were the case you couldn't justify pretty much any test other than "applicant is not dead or in a coma." You just need to show that it is clearly less likely. I would find it really surprising if there wasn't some kind of IQ threshold effect for programming (i.e. the closer you are to the middle of the spectrum, the better a predictor of programming performance IQ is). That would make it easy to justify using an IQ test as a predictor, especially in combination with the more general studies. I don't think IQ would be terribly useful to weed out programmers because I suspect there are more related ways to judge applicants that would already weed out those in the region where IQ is useful, but it is not as simple as "IQ testing for a job is illegal." [1]: https://en.wikipedia.org/wiki/Protected_class https://en.wikipedia.org/wiki/Protected_class [2]: https://en.wikipedia.org/wiki/Griggs_v._Duke_Power_Co https://en.wikipedia.org/wiki/Griggs_v._Duke_Power_Co. [3]: http://www.tandfonline.com/doi/full/10.1080/10888691.2014.983635 http://www.tandfonline.com/doi/full/10.1080/10888691.2014.98...
rguzman 11y ago
The best way to show that you get things done is to have a record of things you've done to point to.

tstock55 11y ago

I accepted an offer at IBM a few months ago and had to take an IQ tests of sorts. It was mostly focused around pattern recognition with some algebra problems thrown in. At best it seemed only tangentially related to programming ability.. I'm not sure that attempting to measure whether a candidate is "smart" or not is a good hiring strategy. A good analogy for this type of interviewing is like trying to judge how good someone is at basketball by measuring how tall they are. Is it related? Sure. But it certainly doesn't give you the whole picture and might not be a useful information at all. Watching them play a few games would be a better strategy. Fortunately there a quite a few startups working on this problem. Lytmus is one in particular that I think looks very promising.

marme 11y ago

IQ tests are not only politically incorrect they are illegal in the US. It is illegal discrimination to make hiring decisions based on IQ scores. You can only use a test that shows those with higher scores perform better at the job. A general IQ test can not be shown to do that. Basically you can not ask questions that test skills that would not be used in the job the candidate is applying for. You must be able to defend the metric you use for hiring and that it is not discriminatory. If you give an IQ test that is unrelated to the job and say no one below 100 IQ will be hired you must prove that someone with 101 IQ can do the job but someone with 99 IQ can not properly do the job

johncolanduoni 11y ago
> It is illegal discrimination to make hiring decisions based on IQ scores. You can only use a test that shows those with higher scores perform better at the job. There is more scientific evidence that IQ predicts job performance than for most of the more qualitative criterions that employers use in practice. The source of the illegality of using IQ tests is not legislative (intelligence is not a protected class[1]). It comes from the precedent of Griggs v. Duke Power Co., in which the IQ test was being used as a barrier for jobs where IQ was not a good predictor (not programming), and prior to thorough research on the subject. There have been hundreds of studies showing that IQ tests predict job performance (though not super strongly) at the beginning of careers, even though this fades once they have more experience. This[3] meta-analysis approaches these studies very critically, and concludes that many inflate the effect, and that it doesn't validate the IQ test (i.e. there may be non-cognitive reasons for it) but that the correlation still exists. (For what it's worth, I suspect they're right about the non-cognitive part since the correlation has become much stronger over time, particularly around the 1970s. However, when your goal is to test for something and not to explain it, correlation is good enough.) > Basically you can not ask questions that test skills that would not be used in the job the candidate is applying for. ... You must be able to defend the metric you use for hiring and that it is not discriminatory. No, the burden set in Griggs and later elaborated in a modification to the Civil Rights Act is not to prove that a test is not discriminatory at all. Instead for tests that are discriminatory in practice (though not obviously in design, like IQ), you must show that the test is truly indicative of job performance. If this were not the case, you couldn't ask if people went to college, due to the racial disparity in college graduates. Also you can ask things you can't prove are related to job performance if it isn't discriminatory (not that this is really useful). > If you give an IQ test that is unrelated to the job and say no one below 100 IQ will be hired you must prove that someone with 101 IQ can do the job but someone with 99 IQ can not properly do the job Of course you don't have to show that anyone who fails your test will 100% be worse on a case by case basis than someone who passes. If that were the case you couldn't justify pretty much any test other than "applicant is not dead or in a coma." You just need to show that it is clearly less likely. I would find it really surprising if there wasn't some kind of IQ threshold effect for programming (i.e. the closer you are to the middle of the spectrum, the better a predictor of programming performance IQ is). That would make it easy to justify using an IQ test as a predictor, especially in combination with the more general studies. I don't think IQ would be terribly useful to weed out programmers because I suspect there are more related ways to judge applicants that would already weed out those in the region where IQ is useful, but it is not as simple as "IQ testing for a job is illegal." [1]: https://en.wikipedia.org/wiki/Protected_class https://en.wikipedia.org/wiki/Protected_class [2]: https://en.wikipedia.org/wiki/Griggs_v._Duke_Power_Co https://en.wikipedia.org/wiki/Griggs_v._Duke_Power_Co. [3]: http://www.tandfonline.com/doi/full/10.1080/10888691.2014.983635 http://www.tandfonline.com/doi/full/10.1080/10888691.2014.98...

rguzman 11y ago

The best way to show that you get things done is to have a record of things you've done to point to.