12 ms·

Top Tech Companies: You filter on raw IQ, here is how to do it more effectively

32 points by nadam 13y ago

mynameisme 13y ago
I feel like your article is just a not so subtle ad for your game.
honu 13y ago
While I find the premise of the blog post to be potentially interesting (2nd post since 2011), I wonder if this is much more than a thinly veiled advertisement.
larsberg 13y ago
This post is a mischaracterization of why tech companies as basic coding questions. Or at least Microsoft, where I learned to interview and coached my own team of interviewers over several years. 1) Basic coding questions are a nice filter. If you can't remove an element from a doubly-linked list in C on a whiteboard immediately after graduating, what were you doing the last four/five years? It's like the height signs before an amusement park ride: "you must be at least this competent." These are not intended to be hard problems. You should not be crippled if you "think differently" or "form important theories." Further, dropping one on the floor is not a deal-killer; you just have to get most of them. 2) Do you really understand how a computer works? If I give you some inputs (or you come up with them), can you walk through your program and describe the execution and memory state in some detail? You'd be shocked how many 4.0 GPA CS students from Stanford/etc. cannot. I am willing to teach new graduates how to work at scale. At a top firm, I am not willing to send them back to school to take Operating Systems and Architecture again. 3) Can we have a discussion about your code and the problem without you breaking down completely? More edge cases, expand it to a broader domain, increase perf (MSFT), scale it to web size (Google), etc. None of these are raw IQ measures. If you want to see such measures, look to the finance industry where, for example, my wife had to take the Wonderlic Test during one interview, which is definitely more a measure of "raw sprinting smarts."
- olalonde 13y ago
  > point 1) I remember what a doubly linked list is but I wouldn't remember C well enough to pass the test. Does that make me a bad programmer? > point 2) Depending on the position, this might or might not matter. If you are modeling high level concepts in functional languages for example, does it really matter if you don't know what a logic gate is or how processes are scheduled? > point 3) Agreed. I would say whatever questions you chose, it will only filter out for a certain type of programmer. Your questions are most likely targeted at C/C++ systems programmers.
  - Arelius 13y ago
    > I remember what a doubly linked list is but I wouldn't remember C well enough to pass the test. Does that make me a bad programmer? Unless the position was for a C role, Most people would accept psuedo code, or help you along with enough syntax to get you through.
  - kyllo 13y ago
    larsberg dated himself by saying C; most companies today would want you to do it in your choice of either Java or C++. They would let you use objects, not make you do it with structs.
- ep103 13y ago
  Could I have an example of walk though and describe the memory state? Do you mean just describing what references are in the stack and heap?
EllaMentry 13y ago
Ignoring the not-so-thinly-veiled-advertisement. The premise is wrong. Companies optimize for a lot more than raw IQ (company fit, engineering skills, response to criticism and pressure etc.) Humans can also get better a solving problems like he presents. It's not about memorizing test cases, it is about developing heuristics and noticing patterns - something humans are actually very good at. If you practice you will get better.
- hnriot 13y ago
  Why "response to criticism" - is that prevalent in your corporate culture? Sounds like a great reason to walk out of an interview.
  - EllaMentry 13y ago
    In a professional environment, generally the best way to get started is to simply throw something up on a whiteboard, tear it apart, throw something else, tear it apart...rinse repeat. If I tell someone in an interview that their solution is flawed, I want them to be intrigued, ask questions and find out way OR tell me why I am wrong/mistaken in a professional manner - if they are stubborn, rude, angry or simply fail to respond, they will not be a good fit. In any large culture people will constantly tell you you are going about something the wrong way, it is simply in the nature of large cultures. It is the job of any professional engineer to compile notes, listen and react appropriately to this. Sometimes the other people don't have the whole story, or are simply just wrong...other times they know something you don't and you have to be willing to scrap what you were doing and go back to the drawing board - with a new contact and fresh information.
  - kyllo 13y ago
    A lot of people take criticism of their work as personal attacks. They don't have the ability to separate the two. Those kinds of people are not fun to work with, particularly in a peer review situation. You don't want to hire people who will break down during code review because someone else didn't like the way they coded something.
streptomycin 13y ago
The most effective way to filter on raw IQ would be to... filter on raw IQ. But that's illegal.
- thirsteh 13y ago
  Except for cops.
  - daniel-cussen 13y ago
    And the military.
    - saraid216 13y ago
      I can't say I object to people who probably have access to lethal weaponry being required to have a minimum level of intelligence. But I'm not a gun advocate.
      - thirsteh 13y ago
        Minimum or maximum: http://abcnews.go.com/US/court-oks-barring-high-iqs-cops/story?id=95836 http://abcnews.go.com/US/court-oks-barring-high-iqs-cops/sto...
        saraid216 13y ago
        Yeah, that seems like a problem. Maybe we should make it less likely that police officers who go through the training will leave.
- tsotha 13y ago
  It's not technically illegal, just legally perilous. Some companies do it, but they're companies that require a whole lot of other qualifications - "we only hire from Harvard and Yale" or "applicants must have ten years managing at least 150 employees".
kozikow 13y ago
There are two major types of problems while programming. Firstly how to break down a bigger problem into set of smaller problems. Secondly how to solve small problems quickly and efficiently. Large scale design question represent the former. Type three interview questions test the later. While raw IQ may he correlating with it this is not what big tech companies are looking for IMHO.
rachelbythebay 13y ago
I suggest they might want to filter based on IQ but the math doesn't support them. Pick a cut-off point, below which you will not hire a person. Assume perfect filtering and no mistakes. How big can your company get? Start with 7 billion humans, then reduce to the set of those with at least the target IQ. Are there enough people in the world to make this work? How about after subtracting people who don't do that kind of work or don't speak the right languages? Or they're in the wrong country and won't move. Or they're at some other employer. Keep subtracting from the pool and see who's left. If your company is bigger than the number you get after this filtering, then you aren't really filtering that way. You just think you are.
retr0rocket 13y ago
samatman 13y ago
The game Set would appear at first glimpse to be the same sort of raw-horsepower IQ test. And yet, skilled players are substantially better than unskilled players. They also tend to be formidably intelligent people, but any game may be gamed.
kenster07 13y ago
Raw IQ should only be one of several components to consider. There are many proxies for determining raw IQ, but the proxies themselves should not be the end goal. Remember that writing maintainable code is also about communicating an idea -- knowing how to pick the right word to describe a variable or function -- knowing how to document your code succinctly, are critical and have a significant effect on a company's bottom line in the long-run. Furthermore, you want to hire someone who will not negatively affect the productivity of his coworkers with his attitude. He must be able to work productively in a team setting.
deleted 13y ago
[deleted]
od2m 13y ago
I've always had difficulty interviewing because I'm a #2 on this guys list--the "long distance runner" of thinking. My talent has always been slow, methodical crunching of massive amounts of data-- analysis. There's no way to demonstrate this talent during an interviews. I've gotten feedback from interviews and heard, "It's not that you didn't answer every question correctly, it's that you weren't fast enough." So I essentially can't show my most valuable skill to a potential employer.
edmond_dantes 13y ago
My IQ is 136. I can attest that it is not an indicator of anything. I am a lazy, average programmer. Whatever benefit I gain from having a higher IQ is offset by my inability to focus for more than a minute. I don't understand how people can study and lack many basic skills because I aways seemed to sneak by without them. I am a chronic procrastinator, only the fear of having to write a final in 24 hours made me study/cram by pulling all nighters. I have a CS degree and rarely went to class, having collected every letter grade A+ through F, and survived being on academic probation (my semester with four D's and an A+). Just one data-point and a counter-example as to why IQ is not a good measure of anything. According to Gwern[1]: It is worth noting that studies of human genius frequently say that raw IQ and talent are not helpful past unexceptional levels like 130 IQ; what makes the difference is motivation and the "drive to mastery" 1: http://www.gwern.net/Terrorism%20is%20not%20Effective http://www.gwern.net/Terrorism%20is%20not%20Effective
- MDS100 13y ago
  ADHS?
  - edmond_dantes 13y ago
    I've never been diagnosed. In high school I had an uncharacterized organizational learning disability. Out of desperation I wanted to be medicated and have made the huge step, huge for me, of talking to a doctor who referred me -- we'll see. The stories I've seen on HN have soured my opinion of medicating. I don't want to become addicted or unable to function, less than I already do, without drugs.
dinkumthinkum 13y ago
I struggle to find the point of this article. OK, I'm somewhat being coy and perhaps somewhat dismissing it with a middlebrow but in a real sense I'm not quite sure what is being argued. Is this the developer of "Find the Mafia!"? It's not as if the "Top Tech Companies" are actually measuring IQ, so that seems to belie the point. Sure, you could say they are asking questions that you believe correlate with raw IQ if you want, I might say "citation needed" but the fact is it is not exactly what they are doing. Now, I'm not sure this particular type of puzzle is the perfect solution, or even a candidate solution. Yeah, I'm a little flummoxed. By the way, the simpler explanation for asking question about CS theory and algorithms is that they want people that have really studied them thoroughly. I know, it's a crazy idea, ... but it just might be the answer. :)
tokenadult 13y ago
I read the fine article and all the comments submitted here before seeing some questions that can be answered by reading a FAQ I've posted here on HN before about company hiring procedures. The last full posting of that FAQ was a while ago, https://news.ycombinator.com/item?id=5227923 https://news.ycombinator.com/item?id=5227923 and the tl;dr of the FAQ is "If you are hiring for any kind of job in the United States, prefer a work-sample test as your hiring procedure. If you are hiring in most other parts of the world, use a work-sample test in combination with a general mental ability test." Readers of HN have asked me to put that FAQ somewhere on the Web with a permanent link, and I am revising it for posting to my personal website. If you have suggestions for how to make the FAQ better, I'd be glad to hear them, as I have already been doing further research on company hiring procedures.
deleted 13y ago
[deleted]

32 points by nadam 13y ago

I feel like your article is just a not so subtle ad for your game.

honu 13y ago

While I find the premise of the blog post to be potentially interesting (2nd post since 2011), I wonder if this is much more than a thinly veiled advertisement.

larsberg 13y ago

This post is a mischaracterization of why tech companies as basic coding questions. Or at least Microsoft, where I learned to interview and coached my own team of interviewers over several years. 1) Basic coding questions are a nice filter. If you can't remove an element from a doubly-linked list in C on a whiteboard immediately after graduating, what were you doing the last four/five years? It's like the height signs before an amusement park ride: "you must be at least this competent." These are not intended to be hard problems. You should not be crippled if you "think differently" or "form important theories." Further, dropping one on the floor is not a deal-killer; you just have to get most of them. 2) Do you really understand how a computer works? If I give you some inputs (or you come up with them), can you walk through your program and describe the execution and memory state in some detail? You'd be shocked how many 4.0 GPA CS students from Stanford/etc. cannot. I am willing to teach new graduates how to work at scale. At a top firm, I am not willing to send them back to school to take Operating Systems and Architecture again. 3) Can we have a discussion about your code and the problem without you breaking down completely? More edge cases, expand it to a broader domain, increase perf (MSFT), scale it to web size (Google), etc. None of these are raw IQ measures. If you want to see such measures, look to the finance industry where, for example, my wife had to take the Wonderlic Test during one interview, which is definitely more a measure of "raw sprinting smarts."

olalonde 13y ago
> point 1) I remember what a doubly linked list is but I wouldn't remember C well enough to pass the test. Does that make me a bad programmer? > point 2) Depending on the position, this might or might not matter. If you are modeling high level concepts in functional languages for example, does it really matter if you don't know what a logic gate is or how processes are scheduled? > point 3) Agreed. I would say whatever questions you chose, it will only filter out for a certain type of programmer. Your questions are most likely targeted at C/C++ systems programmers.
- Arelius 13y ago
  > I remember what a doubly linked list is but I wouldn't remember C well enough to pass the test. Does that make me a bad programmer? Unless the position was for a C role, Most people would accept psuedo code, or help you along with enough syntax to get you through.
- kyllo 13y ago
  larsberg dated himself by saying C; most companies today would want you to do it in your choice of either Java or C++. They would let you use objects, not make you do it with structs.
ep103 13y ago
Could I have an example of walk though and describe the memory state? Do you mean just describing what references are in the stack and heap?

EllaMentry 13y ago

Ignoring the not-so-thinly-veiled-advertisement. The premise is wrong. Companies optimize for a lot more than raw IQ (company fit, engineering skills, response to criticism and pressure etc.) Humans can also get better a solving problems like he presents. It's not about memorizing test cases, it is about developing heuristics and noticing patterns - something humans are actually very good at. If you practice you will get better.

hnriot 13y ago
Why "response to criticism" - is that prevalent in your corporate culture? Sounds like a great reason to walk out of an interview.
- EllaMentry 13y ago
  In a professional environment, generally the best way to get started is to simply throw something up on a whiteboard, tear it apart, throw something else, tear it apart...rinse repeat. If I tell someone in an interview that their solution is flawed, I want them to be intrigued, ask questions and find out way OR tell me why I am wrong/mistaken in a professional manner - if they are stubborn, rude, angry or simply fail to respond, they will not be a good fit. In any large culture people will constantly tell you you are going about something the wrong way, it is simply in the nature of large cultures. It is the job of any professional engineer to compile notes, listen and react appropriately to this. Sometimes the other people don't have the whole story, or are simply just wrong...other times they know something you don't and you have to be willing to scrap what you were doing and go back to the drawing board - with a new contact and fresh information.
- kyllo 13y ago
  A lot of people take criticism of their work as personal attacks. They don't have the ability to separate the two. Those kinds of people are not fun to work with, particularly in a peer review situation. You don't want to hire people who will break down during code review because someone else didn't like the way they coded something.

streptomycin 13y ago

The most effective way to filter on raw IQ would be to... filter on raw IQ. But that's illegal.

thirsteh 13y ago
Except for cops.
- daniel-cussen 13y ago
  And the military.
  - saraid216 13y ago
    I can't say I object to people who probably have access to lethal weaponry being required to have a minimum level of intelligence. But I'm not a gun advocate.
    - thirsteh 13y ago
      Minimum or maximum: http://abcnews.go.com/US/court-oks-barring-high-iqs-cops/story?id=95836 http://abcnews.go.com/US/court-oks-barring-high-iqs-cops/sto...
      - saraid216 13y ago
        Yeah, that seems like a problem. Maybe we should make it less likely that police officers who go through the training will leave.
tsotha 13y ago
It's not technically illegal, just legally perilous. Some companies do it, but they're companies that require a whole lot of other qualifications - "we only hire from Harvard and Yale" or "applicants must have ten years managing at least 150 employees".

kozikow 13y ago

There are two major types of problems while programming. Firstly how to break down a bigger problem into set of smaller problems. Secondly how to solve small problems quickly and efficiently. Large scale design question represent the former. Type three interview questions test the later. While raw IQ may he correlating with it this is not what big tech companies are looking for IMHO.

rachelbythebay 13y ago

I suggest they might want to filter based on IQ but the math doesn't support them. Pick a cut-off point, below which you will not hire a person. Assume perfect filtering and no mistakes. How big can your company get? Start with 7 billion humans, then reduce to the set of those with at least the target IQ. Are there enough people in the world to make this work? How about after subtracting people who don't do that kind of work or don't speak the right languages? Or they're in the wrong country and won't move. Or they're at some other employer. Keep subtracting from the pool and see who's left. If your company is bigger than the number you get after this filtering, then you aren't really filtering that way. You just think you are.

retr0rocket 13y ago

samatman 13y ago

The game Set would appear at first glimpse to be the same sort of raw-horsepower IQ test. And yet, skilled players are substantially better than unskilled players. They also tend to be formidably intelligent people, but any game may be gamed.

kenster07 13y ago

Raw IQ should only be one of several components to consider. There are many proxies for determining raw IQ, but the proxies themselves should not be the end goal. Remember that writing maintainable code is also about communicating an idea -- knowing how to pick the right word to describe a variable or function -- knowing how to document your code succinctly, are critical and have a significant effect on a company's bottom line in the long-run. Furthermore, you want to hire someone who will not negatively affect the productivity of his coworkers with his attitude. He must be able to work productively in a team setting.

deleted 13y ago

[deleted]

od2m 13y ago

I've always had difficulty interviewing because I'm a #2 on this guys list--the "long distance runner" of thinking. My talent has always been slow, methodical crunching of massive amounts of data-- analysis. There's no way to demonstrate this talent during an interviews. I've gotten feedback from interviews and heard, "It's not that you didn't answer every question correctly, it's that you weren't fast enough." So I essentially can't show my most valuable skill to a potential employer.

edmond_dantes 13y ago

My IQ is 136. I can attest that it is not an indicator of anything. I am a lazy, average programmer. Whatever benefit I gain from having a higher IQ is offset by my inability to focus for more than a minute. I don't understand how people can study and lack many basic skills because I aways seemed to sneak by without them. I am a chronic procrastinator, only the fear of having to write a final in 24 hours made me study/cram by pulling all nighters. I have a CS degree and rarely went to class, having collected every letter grade A+ through F, and survived being on academic probation (my semester with four D's and an A+). Just one data-point and a counter-example as to why IQ is not a good measure of anything. According to Gwern[1]: It is worth noting that studies of human genius frequently say that raw IQ and talent are not helpful past unexceptional levels like 130 IQ; what makes the difference is motivation and the "drive to mastery" 1: http://www.gwern.net/Terrorism%20is%20not%20Effective http://www.gwern.net/Terrorism%20is%20not%20Effective

MDS100 13y ago
ADHS?
- edmond_dantes 13y ago
  I've never been diagnosed. In high school I had an uncharacterized organizational learning disability. Out of desperation I wanted to be medicated and have made the huge step, huge for me, of talking to a doctor who referred me -- we'll see. The stories I've seen on HN have soured my opinion of medicating. I don't want to become addicted or unable to function, less than I already do, without drugs.

dinkumthinkum 13y ago

I struggle to find the point of this article. OK, I'm somewhat being coy and perhaps somewhat dismissing it with a middlebrow but in a real sense I'm not quite sure what is being argued. Is this the developer of "Find the Mafia!"? It's not as if the "Top Tech Companies" are actually measuring IQ, so that seems to belie the point. Sure, you could say they are asking questions that you believe correlate with raw IQ if you want, I might say "citation needed" but the fact is it is not exactly what they are doing. Now, I'm not sure this particular type of puzzle is the perfect solution, or even a candidate solution. Yeah, I'm a little flummoxed. By the way, the simpler explanation for asking question about CS theory and algorithms is that they want people that have really studied them thoroughly. I know, it's a crazy idea, ... but it just might be the answer. :)

tokenadult 13y ago

I read the fine article and all the comments submitted here before seeing some questions that can be answered by reading a FAQ I've posted here on HN before about company hiring procedures. The last full posting of that FAQ was a while ago, https://news.ycombinator.com/item?id=5227923 https://news.ycombinator.com/item?id=5227923 and the tl;dr of the FAQ is "If you are hiring for any kind of job in the United States, prefer a work-sample test as your hiring procedure. If you are hiring in most other parts of the world, use a work-sample test in combination with a general mental ability test." Readers of HN have asked me to put that FAQ somewhere on the Web with a permanent link, and I am revising it for posting to my personal website. If you have suggestions for how to make the FAQ better, I'd be glad to hear them, as I have already been doing further research on company hiring procedures.

deleted 13y ago

[deleted]