3 ms·

> However, whether or not a candidate answers a question correctly is not the only source of signal during an interview. You can also evaluate their process by,

> However, whether or not a candidate answers a question correctly is not the only source of signal during an interview. You can also evaluate their process by, for example, observing how long it takes them to finish, how clean their code is, and how much they struggle while finding a solution. Our analysis shows that this second source of signal (process) is almost as predictive as the first (correctness). I seem to do OK when an interviewer: - asks me a question that sounds like a real problem, not a contrived one (although on occasion I'll have fun with a contrived puzzle if the interviewer has a sense of humor & makes the process light hearted) - doesn't push me down a path that requires me to implement a "simpler" solution I'd never consider (e.g. asking me a question that clearly wants an O(N) solution & then pushes me to try the O(2^n solution first) - talks like a person with a problem, and not as someone who clearly knows what they want & simply won't say it - doesn't try to "see how I think", because I code as much in my head as I do on a screen, meaning most of the code I throw into a text editor is the latest thought in a stream of random ideas until I get to one that works - doesn't constantly interrupt me - states their actual expectations, such as "I don't expect you to finish, what I'm really looking for is X"

kyralis 8y ago
As a frequent interviewer I will say that you are not alone in that: any interviewer that isn't following the above guidelines is unlikely to get a good signal from anyone. To your final point about expectations: One question that we try to ask ourselves about any particular interview question is "how may bits of information are we getting out of this?" Trivia questions are usually less than 1 - you'll find out if they know some trivia, but you don't honestly really even care. The best questions are those that get you several bits over the course of the interview period, rather than a single yes/no answer at the end. Questions where there is no final end point or where we explicitly do not expect candidates to finish, are often the most effective; we can explore the pathways that are working, dive into side channels that seem interesting, and get data along the way rather than just a checkmark on some individually-useless problem.
- PopeDotNinja 8y ago
  I like the idea of extracting as many useful bits of information as possible!
ninja10 8y ago
So much this^^! Like everyone here is saying, interviewing itself is a skill, and part of that skill is how the interviewer communicates and takes part in the problem (or lack there of ). Every point you mentioned is an issue I've seen as well.
em-bee 8y ago
regarding the simpler solution, i just had that situation today. working on elevatorsaga.com the candidate clearly wanted to solve the full problem correctly (the way a real elevator works) from the get-go. however doing so would have taken him several hours without him knowing if he got any closer to a working solution. i had to push him to try something simpler first just so that he would get to somewhere meaningful within the hour available. so which approach is better depends on the goals. in my case i wanted the candidate to solve the basic problem first: (make every person get to their destination, no matter how long the elevator needs) and then optimize to make it go faster.
- PopeDotNinja 8y ago
  I hear what you're saying, I do. But if the candidate knows what an optimal solution is, and that optimal solution takes several hours, maybe you're asking a question that is too big? For example, asking someone to write a self balancing binary search tree in an interview might be too much. And asking a question that ultimately demands using a self balancing BST might be a bit much. It's like asking a question that really needs an associative array, but then imposing the rule that associative arrays aren't available. How about simply asking the candidate about binary trees, if they've used them, and what they've used them for? Last week an interviewer asked me about if I was familiar with the Egg Dropping problem (https://brilliant.org/wiki/egg-dropping/ https://brilliant.org/wiki/egg-dropping/). I wasn't, but I remember saying something like, "I haven't done this one before, but I'm pretty sure it's going to take O(log n) guesses to find the max floor from which you can safely drop the egg." The interviewer asked me how I'd implement it, and then we got into the weeds when I started asking for what the requirements were. He wouldn't share any information about inputs & outputs, so I just started writing the dumbest thing possible, predictably coded myself into a corner, and burned a potential interview win-win on learning how answer that kind of question for that kind of interviewer.

kyralis 8y ago

As a frequent interviewer I will say that you are not alone in that: any interviewer that isn't following the above guidelines is unlikely to get a good signal from anyone. To your final point about expectations: One question that we try to ask ourselves about any particular interview question is "how may bits of information are we getting out of this?" Trivia questions are usually less than 1 - you'll find out if they know some trivia, but you don't honestly really even care. The best questions are those that get you several bits over the course of the interview period, rather than a single yes/no answer at the end. Questions where there is no final end point or where we explicitly do not expect candidates to finish, are often the most effective; we can explore the pathways that are working, dive into side channels that seem interesting, and get data along the way rather than just a checkmark on some individually-useless problem.

PopeDotNinja 8y ago
I like the idea of extracting as many useful bits of information as possible!

ninja10 8y ago

So much this^^! Like everyone here is saying, interviewing itself is a skill, and part of that skill is how the interviewer communicates and takes part in the problem (or lack there of ). Every point you mentioned is an issue I've seen as well.

em-bee 8y ago

regarding the simpler solution, i just had that situation today. working on elevatorsaga.com the candidate clearly wanted to solve the full problem correctly (the way a real elevator works) from the get-go. however doing so would have taken him several hours without him knowing if he got any closer to a working solution. i had to push him to try something simpler first just so that he would get to somewhere meaningful within the hour available. so which approach is better depends on the goals. in my case i wanted the candidate to solve the basic problem first: (make every person get to their destination, no matter how long the elevator needs) and then optimize to make it go faster.

PopeDotNinja 8y ago
I hear what you're saying, I do. But if the candidate knows what an optimal solution is, and that optimal solution takes several hours, maybe you're asking a question that is too big? For example, asking someone to write a self balancing binary search tree in an interview might be too much. And asking a question that ultimately demands using a self balancing BST might be a bit much. It's like asking a question that really needs an associative array, but then imposing the rule that associative arrays aren't available. How about simply asking the candidate about binary trees, if they've used them, and what they've used them for? Last week an interviewer asked me about if I was familiar with the Egg Dropping problem (https://brilliant.org/wiki/egg-dropping/ https://brilliant.org/wiki/egg-dropping/). I wasn't, but I remember saying something like, "I haven't done this one before, but I'm pretty sure it's going to take O(log n) guesses to find the max floor from which you can safely drop the egg." The interviewer asked me how I'd implement it, and then we got into the weeds when I started asking for what the requirements were. He wouldn't share any information about inputs & outputs, so I just started writing the dumbest thing possible, predictably coded myself into a corner, and burned a potential interview win-win on learning how answer that kind of question for that kind of interviewer.