4 ms·

I like this idea because it allows me to figure out if they have one of the most important skills: being able to say "I don't know" without wasting a lot of tim

I like this idea because it allows me to figure out if they have one of the most important skills: being able to say "I don't know" without wasting a lot of time. IMO anyone else saying you should tell them the solutions might not have answers are dancing around needing to have that skill.

jrochkind1 8y ago
I guess if an applicant has done enough research/prep for technical coding interviews, they probably will come prepared realizing that not all questions may have answers or may be able to be answered without context they aren't expected to have. I know I will after this discussion! I know I would also get very thrown by a question without an answer if I wasn't expecting it, in a way I don't think I would be on the job. (On the job I know too well through experience that not all problems have good solutions, but in an interview question I maybe would have assumed there was an answer they expected me to get, before this exchange). Of course, the end goal of interviewing for an employer is maximizing their chance of filling position with good people, which involves balancing false negatives and false positives -- and in fact a "false positive" (wrong hire) is probably _worse_ than a "false negative" (turning down someone that would have been great). So... filtering out some candidate that might have been good may be a neccessary sacrifice, not a flaw in the process, so long as you're doing even better at successfully filtering out those that wouldn't work out. Whatever works! If I felt someone was intentionally giving me a "trick question" in an interview, I might conclude they weren't someone I wanted to work for anyway, so everybody wins I guess.
- TallGuyShort 8y ago
  It doesn't even have to be a question for which there is no answer. IMO it works best with progressive questions where you can start with something straightforward that just validates basic knowledge they should have, and progresses to get increasingly difficult. I used to ask about the layers in the network stack, and how you might troubleshoot it. I start with something as simple as a firewall blocking a port, or the server not listening on the port you expect. If they do that, great. Anyone who's had to troubleshoot distributed systems should have checked that stuff plenty. Then we try a protocol they might not have used. Most developers have a rough idea of UDP vs TCP, but they might be confidently wrong about some details or try to make stuff up and say "I'm pretty sure...". We go down a layer and try some scenarios where routing is messed up. I don't expect most devs to know about that, maybe someone with more sysadmin experience, but by now it's painfully obvious if they don't know what they're talking about. It happens to be pretty handy on my team, but if at this point someone says, "you know I really don't know a lot about networking", great! I'm glad you told me. Let's skip talking about voltage fluctuations in Cat5 cables and get back to software development. Another good one is designing a data structure where there's a lot of different access patterns you might optimize for. Gives them a chance to demonstrate some simple coding and CS skills before it gets complex, but then there's lots of room to talk about the use cases. Once you're talking about use cases, are they willing to ask about the trade-offs? Priorities of various use-cases? etc.
mjevans 8y ago
Then problem is that getting to "I don't know" can take a LONG time for someone who's actually a good fit. A generalist (jack of all trades, willing to become a master in some that are used often in a local time domain) might actually know a lot of that information. A good way of approaching this might be to provide problem feedback during the mock, stating that there's a given problem reported with the results or some other plausible complication and seeing how things go from there. Also, the not knowing thing. I've hit that a few times myself, however I'll generally exhaust every reasonable option I have to look-up or reverse engineer the solution on my own first. Google / duckduckgo, stack-overflow, asking some other tech friends if they've encountered similar things before; if I worked somewhere with other technical users that I expect might have seen this they'd be just before the out of company friends on that list. By the time I get to something I don't know, and can't find out on my own, it's actually a DIFFICULT or VERY domain specific problem. Generally then I go for adding diagnostic prints or using a debugger to trace the problem and find out where things are going wrong so I at least can narrow down causes/solutions. Even then, it's still taking a __LOT__ to get to I don't know.
- TallGuyShort 8y ago
  Getting to an absolute "I don't know" does indeed take too long, but doesn't really do much for me as an interviewer anyway. My goal in these questions is to get to "I don't know, but I think... and I would verify by...", "I don't know, but one could try..." or "I don't know but I would Google for..." are all excellent places to end up. You're not BS'ing me, but you're not stumped either. You know sensible next steps. We all have to look stuff up regularly or test the unknown. Can be a problem trying to out-generalist a generalist, but if you pick something in your niche (I ask them lots of other questions relating to their background, so this question doesn't have to cover the exact job requirements) and practice the problem with dozens of interviewers, it'll be pretty hard to have someone who can genuinely out-think you in the depth of that problem. And missing out on the "I don't know" is a small price to pay to find a candidate that does that well on the problem!

jrochkind1 8y ago

I guess if an applicant has done enough research/prep for technical coding interviews, they probably will come prepared realizing that not all questions may have answers or may be able to be answered without context they aren't expected to have. I know I will after this discussion! I know I would also get very thrown by a question without an answer if I wasn't expecting it, in a way I don't think I would be on the job. (On the job I know too well through experience that not all problems have good solutions, but in an interview question I maybe would have assumed there was an answer they expected me to get, before this exchange). Of course, the end goal of interviewing for an employer is maximizing their chance of filling position with good people, which involves balancing false negatives and false positives -- and in fact a "false positive" (wrong hire) is probably _worse_ than a "false negative" (turning down someone that would have been great). So... filtering out some candidate that might have been good may be a neccessary sacrifice, not a flaw in the process, so long as you're doing even better at successfully filtering out those that wouldn't work out. Whatever works! If I felt someone was intentionally giving me a "trick question" in an interview, I might conclude they weren't someone I wanted to work for anyway, so everybody wins I guess.

TallGuyShort 8y ago
It doesn't even have to be a question for which there is no answer. IMO it works best with progressive questions where you can start with something straightforward that just validates basic knowledge they should have, and progresses to get increasingly difficult. I used to ask about the layers in the network stack, and how you might troubleshoot it. I start with something as simple as a firewall blocking a port, or the server not listening on the port you expect. If they do that, great. Anyone who's had to troubleshoot distributed systems should have checked that stuff plenty. Then we try a protocol they might not have used. Most developers have a rough idea of UDP vs TCP, but they might be confidently wrong about some details or try to make stuff up and say "I'm pretty sure...". We go down a layer and try some scenarios where routing is messed up. I don't expect most devs to know about that, maybe someone with more sysadmin experience, but by now it's painfully obvious if they don't know what they're talking about. It happens to be pretty handy on my team, but if at this point someone says, "you know I really don't know a lot about networking", great! I'm glad you told me. Let's skip talking about voltage fluctuations in Cat5 cables and get back to software development. Another good one is designing a data structure where there's a lot of different access patterns you might optimize for. Gives them a chance to demonstrate some simple coding and CS skills before it gets complex, but then there's lots of room to talk about the use cases. Once you're talking about use cases, are they willing to ask about the trade-offs? Priorities of various use-cases? etc.

mjevans 8y ago

Then problem is that getting to "I don't know" can take a LONG time for someone who's actually a good fit. A generalist (jack of all trades, willing to become a master in some that are used often in a local time domain) might actually know a lot of that information. A good way of approaching this might be to provide problem feedback during the mock, stating that there's a given problem reported with the results or some other plausible complication and seeing how things go from there. Also, the not knowing thing. I've hit that a few times myself, however I'll generally exhaust every reasonable option I have to look-up or reverse engineer the solution on my own first. Google / duckduckgo, stack-overflow, asking some other tech friends if they've encountered similar things before; if I worked somewhere with other technical users that I expect might have seen this they'd be just before the out of company friends on that list. By the time I get to something I don't know, and can't find out on my own, it's actually a DIFFICULT or VERY domain specific problem. Generally then I go for adding diagnostic prints or using a debugger to trace the problem and find out where things are going wrong so I at least can narrow down causes/solutions. Even then, it's still taking a __LOT__ to get to I don't know.

TallGuyShort 8y ago
Getting to an absolute "I don't know" does indeed take too long, but doesn't really do much for me as an interviewer anyway. My goal in these questions is to get to "I don't know, but I think... and I would verify by...", "I don't know, but one could try..." or "I don't know but I would Google for..." are all excellent places to end up. You're not BS'ing me, but you're not stumped either. You know sensible next steps. We all have to look stuff up regularly or test the unknown. Can be a problem trying to out-generalist a generalist, but if you pick something in your niche (I ask them lots of other questions relating to their background, so this question doesn't have to cover the exact job requirements) and practice the problem with dozens of interviewers, it'll be pretty hard to have someone who can genuinely out-think you in the depth of that problem. And missing out on the "I don't know" is a small price to pay to find a candidate that does that well on the problem!