3 ms·

I completely agree that this is a very simple problem; I'm not at all arguing that it isn't. What I am arguing is that this doesn't make it super-unlikely that

by gjm11 2y ago

I completely agree that this is a very simple problem; I'm not at all arguing that it isn't. What I am arguing is that this doesn't make it super-unlikely that an otherwise competent person screws it up. If it's really a 1-in-10,000 event, isn't it rather surprising that we've got someone right here in this thread to whom it happened? Someone, moreover, not just barely competent but able to do well with a demanding employer, and who has a lot of exposure to this sort of interview question? I think this is good evidence that, in fact, it's not vanishingly unlikely that someone genuinely competent flubs what should be an embarrassingly straightforward coding task in an interview. That doesn't, unfortunately, mean I have a good suggestion for a better way of telling who is and who isn't capable of writing code. Maybe this is the best we can do. If it really is just a matter of "sometimes, at random, people have bad days"[1] then provided it's fairly rare this issue doesn't matter too much. I'm more worried about the possibility that it's more "a smallish fraction of otherwise-good people often have this sort of bad day", because then that smallish fraction of people may be getting completely overlooked, which is bad for them because it may take them ages to get a job and bad for employers because it effectively reduces the pool of good people to hire. [1] In this particular case, the "bad day" seems to have included an obnoxious interviewer, which it's reasonable to hope wouldn't be repeated at another company's interview. But, again, pointing out a problem unfortunately doesn't guarantee having a good answer. But, equally, not having a good answer doesn't mean the problem isn't real. It would be very nice to believe that all the people failing interviews because they fail to solve an easy coding problem are undeserving incompetents who should be bombing out of the interviews, but it looks to me like that ain't so.

ltbarcly3 2y ago
That's fine. Lets say 5% of the time a world class programmer would get this wrong, but 100% of people who can't program get this wrong. Lets say we interview 100 (N) people for a role. (100 is a lot but pick a number yourself and follow along) The probability that a random interviewee is a top programmer is lets say 1/50 (T). So in our pool of 100 we have on average 2. Give everyone this problem, and if they get it right they move to round 2 which is 4 more interviews. Lets say 20 (P) people pass to round #2. We are paying for a total of 100 + 80 = 180 interviews (N + 4P). To give everyone who failed the first round another try would require another 80 (N - P) interviews. So for almost 50% more cost (N-P)/(N + 4P) we gain (T * N * 5%) = 0.1 additional top programmer making it to round 2 on average. Just interviewing another 80 people (same cost) would get us (T * 80 * 95%) == 1.52 additional top programmers to round #2 on average. There's basically no reasonable % you can pick for how often people screw up where it makes sense to do anything but just ignore the possibility.
- gjm11 2y ago
  Sure, but now you're making a different claim from the one you made before. Before, you were saying: anyone who says "muh anxiety" is just trying to cover for the fact that they're incompetent. I suggested that we've got pretty good evidence that interview stress really truly can, and not only-vanishingly-occasionally, make competent people fail to do simple interview-type coding tasks. So now you're advancing a different claim: even if some competent people appear incompetent in some interviews because of the stress of interviewing, you should make interview decisions as if those people are just plain incompetent, because most people who fail to do simple interview-type coding tasks really are incompetent and identifying the few who were just stressed out is difficult. That could very well be true! But it's not what I was arguing against before. ... But I'll argue against it just a bit, even though I mean it when I say it could well be true. 1. There may be much less inefficient ways of catching the competent-but-stressed candidates than "just interview everyone twice"; completely incompetent people probably don't interview exactly the same as competent-but-stressed ones. (I don't know for sure because, of course, when you interview someone and they don't perform well you don't generally get to tell which category they were in. But I bet it would be possible to find out.) 2. If the situation is that some people are particularly susceptible to interview stress (but that this doesn't make them bad at actual programming jobs unless they're in an extra-stressful environment), the "eh, just reject them" strategy might be good for companies that are hiring but punishingly bad for the people affected who just can't get a job because they are bad at interviews. If there were a good way to identify people who are good but fall apart readily under interview-stress, that might be a big deal for those people. (Which someone on the hiring side might not care about, of course; but it's sad when some group of people gets systematically screwed over.) I repeat, as I have said before, that I don't have a solution to this problem nor even good reason to believe that there is one. I'm just pushing back against the blithe assertion that there isn't a real problem here.

by gjm11 2y ago

ltbarcly3 2y ago

That's fine. Lets say 5% of the time a world class programmer would get this wrong, but 100% of people who can't program get this wrong. Lets say we interview 100 (N) people for a role. (100 is a lot but pick a number yourself and follow along) The probability that a random interviewee is a top programmer is lets say 1/50 (T). So in our pool of 100 we have on average 2. Give everyone this problem, and if they get it right they move to round 2 which is 4 more interviews. Lets say 20 (P) people pass to round #2. We are paying for a total of 100 + 80 = 180 interviews (N + 4P). To give everyone who failed the first round another try would require another 80 (N - P) interviews. So for almost 50% more cost (N-P)/(N + 4P) we gain (T * N * 5%) = 0.1 additional top programmer making it to round 2 on average. Just interviewing another 80 people (same cost) would get us (T * 80 * 95%) == 1.52 additional top programmers to round #2 on average. There's basically no reasonable % you can pick for how often people screw up where it makes sense to do anything but just ignore the possibility.

gjm11 2y ago
Sure, but now you're making a different claim from the one you made before. Before, you were saying: anyone who says "muh anxiety" is just trying to cover for the fact that they're incompetent. I suggested that we've got pretty good evidence that interview stress really truly can, and not only-vanishingly-occasionally, make competent people fail to do simple interview-type coding tasks. So now you're advancing a different claim: even if some competent people appear incompetent in some interviews because of the stress of interviewing, you should make interview decisions as if those people are just plain incompetent, because most people who fail to do simple interview-type coding tasks really are incompetent and identifying the few who were just stressed out is difficult. That could very well be true! But it's not what I was arguing against before. ... But I'll argue against it just a bit, even though I mean it when I say it could well be true. 1. There may be much less inefficient ways of catching the competent-but-stressed candidates than "just interview everyone twice"; completely incompetent people probably don't interview exactly the same as competent-but-stressed ones. (I don't know for sure because, of course, when you interview someone and they don't perform well you don't generally get to tell which category they were in. But I bet it would be possible to find out.) 2. If the situation is that some people are particularly susceptible to interview stress (but that this doesn't make them bad at actual programming jobs unless they're in an extra-stressful environment), the "eh, just reject them" strategy might be good for companies that are hiring but punishingly bad for the people affected who just can't get a job because they are bad at interviews. If there were a good way to identify people who are good but fall apart readily under interview-stress, that might be a big deal for those people. (Which someone on the hiring side might not care about, of course; but it's sad when some group of people gets systematically screwed over.) I repeat, as I have said before, that I don't have a solution to this problem nor even good reason to believe that there is one. I'm just pushing back against the blithe assertion that there isn't a real problem here.