6 ms·

I've never understood the hate that WBI get here and elsewhere on the internet. I see them the same way as standardized tests: without them, there's no good way

I've never understood the hate that WBI get here and elsewhere on the internet. I see them the same way as standardized tests: without them, there's no good way to independently evaluate people's abilities in a way that is immune to cheating, dishonesty, or fraud (e.g. stealing someone's code and putting it on your github repo, being a terrible employee at previous jobs but just scraping by long enough to move onto the next company, straight up lying on a resume, etc.). Sure, as the author notes, WBI have a high false negative rate. But if you're actually a good programmer, learning basic CS concepts should be extremely easy, and these types of problems, perhaps with a bit of practice, should be relatively trivial. I personally wouldn't want to work with someone who can't extract a path from a DFS, not out of any sense of elitism but just because such an employee likely would be unable to do many other problems properly.

rimliu 9y ago
> there's no good way to independently evaluate people's > abilities And all WBI let you do is to evaluate someone's ability to do WBI.
- opportune 9y ago
  I'd argue that there is a not-insignificant amount of overlap between whiteboarding and coding. Most coding in practice is not as "algorithmic" (in the sense that the problems are theoretically challenging) but having the knowledge and know-how of how to write optimal/efficient algorithms will definitely be echoed throughout any code you write. In addition, in my experience most WBI are not particularly challenging problems that you have probably never seen before. I mentioned the ability to extract a path from a DFS; I would argue that this is such a basic CS skill that being unable to solve that problem indicates an insufficient amount of knowledge of CS. There are definitely WBI that are poorly matched with real job requirements and knowledge. I think these are often seen at smaller companies trying to copy FB/Google without knowing what they are doing. WBI that involve using important CS constructs such as a binary tree or a directed graph IMO are testing important concepts
- pebers 9y ago
  Sure, but you could make that argument about anything you ask candidates in interviews. At some point you have to ask something that attempts to evaluate their performance in an interview timespan. On the other hand, people's performance in whiteboard interviews may positively correlate with later performance as employees. In which case they let you make a reasonable assumption about later performance without being able to evaluate it directly, which is really what interviews are all about.
- deleted 9y ago
  [deleted]
kowdermeister 9y ago
> Sure, as the author notes, WBI have a high false negative rate. That's why this practice should be abandoned. Give the candidates a small, two dayish homework that reveals their ability to do their would-be daily jobs. If it looks good, call them in and talk about their submission. Heck, you might even ask them do draw some diagrams or detailed explanation of their own solution.
- frandroid 9y ago
  Candidates that have offers from other companies that didn't ask them for homework will probably go for these companies first. We scheduled 8 people for interviews last week and 2 of them dropped out before their scheduled interview times because they took jobs elsewhere. Unless you're a top-tier tech employer, you don't have the luxury of asking candidates to do leg work.
  - akras14 9y ago
    Yes, this 100% imho
  - cholantesh 9y ago
    Many candidates also interpret this as an attempt to get free work out of them, too. I personally don't mind them, so long as they're well-designed.
    - kowdermeister 9y ago
      These assignments are tasks that are no way could be used in production, at better places these are even paid assignments. I've seen more than one ad like that.
  - kowdermeister 9y ago
    Small companies do it here (Europe), not sure about the US. I would happily do the leg work if I really dig the company. If they drop out from the queue, they probably prioritized you lower, not a big loss.
  - 9y ago
deleted 9y ago
[deleted]
Apocryphon 9y ago
Standardized tests, while a great idea on principle, leads to extreme optimizing to simply pass exams with less regard for the material that's meant to be learned. And they can be cheated: http://www.straitstimes.com/world/extreme-measures-to-crack-down-on-exam-cheats-around-the-world http://www.straitstimes.com/world/extreme-measures-to-crack-... https://blogs.voanews.com/student-union/2017/02/27/sat-scores-of-asian-students-cancelled-over-cheating/ https://blogs.voanews.com/student-union/2017/02/27/sat-score... https://www.quora.com/Do-Chinese-people-have-creativity/answer/Michelle-Zhou-28 https://www.quora.com/Do-Chinese-people-have-creativity/answ...
- skylark 9y ago
  The question is not whether the test can be gamed, but whether WBI have sufficient predictive power to be used as an interview tool. I think this is absolutely the case at large companies which need to screen hundreds of thousands of candidates per year, perhaps less so at smaller companies.
hibikir 9y ago
Your assumption in the very first paragraph is that they are a good way to independently evaluate people's abilities. It makes perfect sense to look for a way to evaluate people before we hire them. It's a real problem, and we must do something, but do we have any basis to say that the whiteboard interviews correlate with good performance. Otherwise, we go back to the classic "syllogism" to justify policy proposals 1. This is an important problem, and we must do something. 2. X is something. 3. Therefore, we must do this. I've been developing professionally for over 15 years, and I am quite good at whiteboard interviews: 100% offer rate. That said, Many of the best developers I've worked with are not any good at whiteboard interviews, often because they lack the composure under pressure, or like to take some time to think everything through, which are correlated with scoring badly in those interviews. Therefore, no matter how good their code is, they'll have a lot of trouble getting into a company that does whiteboarding interviewing, or even practical interviews under time pressure. I also have found that, in my list of things on what makes an effective member of a programming team, being able to come up with a DFS algorithm under pressure is nowhere near the top of the list. An attitude towards trying to learn what others bring to the table, instead of discounting them immediately because they aren't good at some specific CS shibboleth, is far closer to the top though. We just test for random algorithm questions for this because it's easy to measure, not because it's really all that helpful, and that's pretty unfortunate IMO.
- smallnamespace 9y ago
  > they lack the composure under pressure Depending on the exact role and project, that can be a severe negative as well (e.g. devops). In that case, putting them through the stress of a WBI is a side benefit.
  - meIias 9y ago
    The stress of trying to land a job is different than the stress of trying to meet a deadline at a job, no? And Im not saying that meeting deadlines is the only stress, Im just saying that being stressed out about how you will provide for yourself is different than the stress of having to perform well at a job
avaer 9y ago
> I see them the same way as standardized tests: without them, there's no good way to independently evaluate people's abilities How about straight up pay them to solve a real problem with you? If they're clearly incompetent, cut it short fast (under an hour). If they're competent, you know that they're competent at the job, not at a whiteboard interview. Yes, I've seen this work at small-medium outfits. > I personally wouldn't want to work with someone who can't extract a path from a DFS What if this person hacks blockchain merkle trees for breakfast, but can't be bothered to regurgitate tangential runtimes of whatever algo you decided to ask about, which they memorized and forgot 10 years ago? I've seen this to be a nontrivial intersection of skills.
- frandroid 9y ago
  We'd turn on a dime and ask them to tell us about their merkle tree hacking on the white board, and the poke at it. The goal is to get a sense of people's skill, and we need to understand that they're not regurgitating something they've memorized ahead of time. Someone we interviewed last week was hoping for fizzbuzz and couldn't answer an equally simple question!
- UncleMeat 9y ago
  There is no chance that I can find a problem that I need to solve that I can get a candidate to work on and finish in a day. Too much custom infrastructure and technical background. Too many problems with IP. Any code I have a candidate write will get thrown away. So then why is a whiteboard interview different than a real problem? They don't have access to a developer environment but that's the only major difference. One should be able to reason about running times, not memorize them. If a candidate cannot explain why their implementation of an algorithm they've never worked with is actually quadratic in disguise, for example, then I'd need to babysit their code if they were hired to make sure the system stays efficient.
- abeyer 9y ago
  > What if this person hacks blockchain merkle trees for breakfast, but can't be bothered to regurgitate tangential runtimes of whatever algo you decided to ask about, which they memorized and forgot 10 years ago? I've seen this to be a nontrivial intersection of skills. But most runtime derivations are pretty basic math and logic. You don't need to memorize a runtime of some algorithm, because if you understand the algorithm and what runtime analysis is about, you can derive it (in an informal hand-wavy way, at least) on the spot on the whiteboard. I wouldn't expect a rigorous derivation of the master theorem during an interview, but being able to reason a bit about worst-case scenarios and the iteration bounds of nested loops or a tree traversal seems like a pretty fundamental skill if you're discussing algorithmic problems.
smallnamespace 9y ago
The cost to an employer of a false positive (hiring someone unqualified) is probably 1-2 orders of magnitude higher than the downside of missing an actually-good coder who can't pass a WBI. Not only does it take a lot of time and bandwidth to fire someone incompetent, but now you have to deal with the damage they've done to your codebase and projects in the meantime. Even if your WBI process rejects 10 good coders for every bad coder it weeds out, you're coming out ahead. As for all these posts from famous or well-known hackers who assert that they don't need to show that they know their stuff -- sure, if you have a portfolio of open source projects that everyone uses, then you have the luxury of not needing to demonstrate your technical competence. How does that help employers evaluate anyone else?
- kthejoker2 9y ago
  This sounds more like a structural problem. 1) All new employees go on 90 day contract-to-hire. So reduce the firing overhead to 0. 2) Similarly, just build some sort of extra ring of QA around new hire code, so all their codebase changes get reviewed by a peer -specifically to assess their long-term employment at the company.- That 10:1 ratio's not set in stone.
caliagent 9y ago
Because nobody gets assigned a task and rushes to a white board with a non-collaborative audience. The WB wire framing etc is always a discussion amongst peers. Sit your prospects at computers spec'd like they would be using in their role with appropriate Internet access. Knowing what to Google is generally the most used function. What is googled varies with each individual. The pressure of dumping pseudo code is a poor metric.
- abeyer 9y ago
  A good WBI should feel like a discussion with a peer. While I think the interviewer has to be careful not to direct/guide things too much, they should be giving input/feedback/questions along the way, much like a coworker on a real design problem might. Dumping pseudocode shouldn't ever be the metric to evaluate a WBI, but just a product of the conversation with the interviewer, which is a far more interesting indicator.
  - caliagent 9y ago
    I agree but have yet to find one that went that way. More companies in the wild, as I alluded to, who feel that's what they should be doing with little to no understanding of how. Same for all of those brainteaser fizzbuzz-esque problems.
    - abeyer 9y ago
      Good ones aren't as common as they should be...bad interviews in general are disturbingly common, though, I don't think tech interviews in general or whiteboard specifically are any different. Brainteasers and fizzbuzz are very different things though: I'm not totally against brain teasers as a prompt for discussion, but they're not my favorite types of questions, and can be hard to judge or of little use if the person either already knows the trick, or just doesn't figure it out. Fizzbuzz on the other hand makes a lot of sense when you're interviewing someone who you don't have any other knowledge of their coding. That type of question absolutely makes a good first pass filter: the problem is easy to explain, the implementation is trivial to do in a few minutes if you are at all qualified. Sure, it's annoying for an experienced person to have to do a fizzbuzz variant for the 200th time, but given that it takes almost no time, and helps the interviewer quickly jump to more advanced stuff or cut off an interview that isn't going to go anywhere, it's a pretty minor inconvenience.

rimliu 9y ago

> there's no good way to independently evaluate people's > abilities And all WBI let you do is to evaluate someone's ability to do WBI.

opportune 9y ago
I'd argue that there is a not-insignificant amount of overlap between whiteboarding and coding. Most coding in practice is not as "algorithmic" (in the sense that the problems are theoretically challenging) but having the knowledge and know-how of how to write optimal/efficient algorithms will definitely be echoed throughout any code you write. In addition, in my experience most WBI are not particularly challenging problems that you have probably never seen before. I mentioned the ability to extract a path from a DFS; I would argue that this is such a basic CS skill that being unable to solve that problem indicates an insufficient amount of knowledge of CS. There are definitely WBI that are poorly matched with real job requirements and knowledge. I think these are often seen at smaller companies trying to copy FB/Google without knowing what they are doing. WBI that involve using important CS constructs such as a binary tree or a directed graph IMO are testing important concepts
pebers 9y ago
Sure, but you could make that argument about anything you ask candidates in interviews. At some point you have to ask something that attempts to evaluate their performance in an interview timespan. On the other hand, people's performance in whiteboard interviews may positively correlate with later performance as employees. In which case they let you make a reasonable assumption about later performance without being able to evaluate it directly, which is really what interviews are all about.
deleted 9y ago
[deleted]

kowdermeister 9y ago

> Sure, as the author notes, WBI have a high false negative rate. That's why this practice should be abandoned. Give the candidates a small, two dayish homework that reveals their ability to do their would-be daily jobs. If it looks good, call them in and talk about their submission. Heck, you might even ask them do draw some diagrams or detailed explanation of their own solution.

frandroid 9y ago
Candidates that have offers from other companies that didn't ask them for homework will probably go for these companies first. We scheduled 8 people for interviews last week and 2 of them dropped out before their scheduled interview times because they took jobs elsewhere. Unless you're a top-tier tech employer, you don't have the luxury of asking candidates to do leg work.
- akras14 9y ago
  Yes, this 100% imho
- cholantesh 9y ago
  Many candidates also interpret this as an attempt to get free work out of them, too. I personally don't mind them, so long as they're well-designed.
  - kowdermeister 9y ago
    These assignments are tasks that are no way could be used in production, at better places these are even paid assignments. I've seen more than one ad like that.
- kowdermeister 9y ago
  Small companies do it here (Europe), not sure about the US. I would happily do the leg work if I really dig the company. If they drop out from the queue, they probably prioritized you lower, not a big loss.
- 9y ago

deleted 9y ago

[deleted]

Apocryphon 9y ago

Standardized tests, while a great idea on principle, leads to extreme optimizing to simply pass exams with less regard for the material that's meant to be learned. And they can be cheated: http://www.straitstimes.com/world/extreme-measures-to-crack-down-on-exam-cheats-around-the-world http://www.straitstimes.com/world/extreme-measures-to-crack-... https://blogs.voanews.com/student-union/2017/02/27/sat-scores-of-asian-students-cancelled-over-cheating/ https://blogs.voanews.com/student-union/2017/02/27/sat-score... https://www.quora.com/Do-Chinese-people-have-creativity/answer/Michelle-Zhou-28 https://www.quora.com/Do-Chinese-people-have-creativity/answ...

skylark 9y ago
The question is not whether the test can be gamed, but whether WBI have sufficient predictive power to be used as an interview tool. I think this is absolutely the case at large companies which need to screen hundreds of thousands of candidates per year, perhaps less so at smaller companies.

hibikir 9y ago

Your assumption in the very first paragraph is that they are a good way to independently evaluate people's abilities. It makes perfect sense to look for a way to evaluate people before we hire them. It's a real problem, and we must do something, but do we have any basis to say that the whiteboard interviews correlate with good performance. Otherwise, we go back to the classic "syllogism" to justify policy proposals 1. This is an important problem, and we must do something. 2. X is something. 3. Therefore, we must do this. I've been developing professionally for over 15 years, and I am quite good at whiteboard interviews: 100% offer rate. That said, Many of the best developers I've worked with are not any good at whiteboard interviews, often because they lack the composure under pressure, or like to take some time to think everything through, which are correlated with scoring badly in those interviews. Therefore, no matter how good their code is, they'll have a lot of trouble getting into a company that does whiteboarding interviewing, or even practical interviews under time pressure. I also have found that, in my list of things on what makes an effective member of a programming team, being able to come up with a DFS algorithm under pressure is nowhere near the top of the list. An attitude towards trying to learn what others bring to the table, instead of discounting them immediately because they aren't good at some specific CS shibboleth, is far closer to the top though. We just test for random algorithm questions for this because it's easy to measure, not because it's really all that helpful, and that's pretty unfortunate IMO.

smallnamespace 9y ago
> they lack the composure under pressure Depending on the exact role and project, that can be a severe negative as well (e.g. devops). In that case, putting them through the stress of a WBI is a side benefit.
- meIias 9y ago
  The stress of trying to land a job is different than the stress of trying to meet a deadline at a job, no? And Im not saying that meeting deadlines is the only stress, Im just saying that being stressed out about how you will provide for yourself is different than the stress of having to perform well at a job

avaer 9y ago

> I see them the same way as standardized tests: without them, there's no good way to independently evaluate people's abilities How about straight up pay them to solve a real problem with you? If they're clearly incompetent, cut it short fast (under an hour). If they're competent, you know that they're competent at the job, not at a whiteboard interview. Yes, I've seen this work at small-medium outfits. > I personally wouldn't want to work with someone who can't extract a path from a DFS What if this person hacks blockchain merkle trees for breakfast, but can't be bothered to regurgitate tangential runtimes of whatever algo you decided to ask about, which they memorized and forgot 10 years ago? I've seen this to be a nontrivial intersection of skills.

frandroid 9y ago
We'd turn on a dime and ask them to tell us about their merkle tree hacking on the white board, and the poke at it. The goal is to get a sense of people's skill, and we need to understand that they're not regurgitating something they've memorized ahead of time. Someone we interviewed last week was hoping for fizzbuzz and couldn't answer an equally simple question!
UncleMeat 9y ago
There is no chance that I can find a problem that I need to solve that I can get a candidate to work on and finish in a day. Too much custom infrastructure and technical background. Too many problems with IP. Any code I have a candidate write will get thrown away. So then why is a whiteboard interview different than a real problem? They don't have access to a developer environment but that's the only major difference. One should be able to reason about running times, not memorize them. If a candidate cannot explain why their implementation of an algorithm they've never worked with is actually quadratic in disguise, for example, then I'd need to babysit their code if they were hired to make sure the system stays efficient.
abeyer 9y ago
> What if this person hacks blockchain merkle trees for breakfast, but can't be bothered to regurgitate tangential runtimes of whatever algo you decided to ask about, which they memorized and forgot 10 years ago? I've seen this to be a nontrivial intersection of skills. But most runtime derivations are pretty basic math and logic. You don't need to memorize a runtime of some algorithm, because if you understand the algorithm and what runtime analysis is about, you can derive it (in an informal hand-wavy way, at least) on the spot on the whiteboard. I wouldn't expect a rigorous derivation of the master theorem during an interview, but being able to reason a bit about worst-case scenarios and the iteration bounds of nested loops or a tree traversal seems like a pretty fundamental skill if you're discussing algorithmic problems.

smallnamespace 9y ago

The cost to an employer of a false positive (hiring someone unqualified) is probably 1-2 orders of magnitude higher than the downside of missing an actually-good coder who can't pass a WBI. Not only does it take a lot of time and bandwidth to fire someone incompetent, but now you have to deal with the damage they've done to your codebase and projects in the meantime. Even if your WBI process rejects 10 good coders for every bad coder it weeds out, you're coming out ahead. As for all these posts from famous or well-known hackers who assert that they don't need to show that they know their stuff -- sure, if you have a portfolio of open source projects that everyone uses, then you have the luxury of not needing to demonstrate your technical competence. How does that help employers evaluate anyone else?

kthejoker2 9y ago
This sounds more like a structural problem. 1) All new employees go on 90 day contract-to-hire. So reduce the firing overhead to 0. 2) Similarly, just build some sort of extra ring of QA around new hire code, so all their codebase changes get reviewed by a peer -specifically to assess their long-term employment at the company.- That 10:1 ratio's not set in stone.

caliagent 9y ago

Because nobody gets assigned a task and rushes to a white board with a non-collaborative audience. The WB wire framing etc is always a discussion amongst peers. Sit your prospects at computers spec'd like they would be using in their role with appropriate Internet access. Knowing what to Google is generally the most used function. What is googled varies with each individual. The pressure of dumping pseudo code is a poor metric.

abeyer 9y ago
A good WBI should feel like a discussion with a peer. While I think the interviewer has to be careful not to direct/guide things too much, they should be giving input/feedback/questions along the way, much like a coworker on a real design problem might. Dumping pseudocode shouldn't ever be the metric to evaluate a WBI, but just a product of the conversation with the interviewer, which is a far more interesting indicator.
- caliagent 9y ago
  I agree but have yet to find one that went that way. More companies in the wild, as I alluded to, who feel that's what they should be doing with little to no understanding of how. Same for all of those brainteaser fizzbuzz-esque problems.
  - abeyer 9y ago
    Good ones aren't as common as they should be...bad interviews in general are disturbingly common, though, I don't think tech interviews in general or whiteboard specifically are any different. Brainteasers and fizzbuzz are very different things though: I'm not totally against brain teasers as a prompt for discussion, but they're not my favorite types of questions, and can be hard to judge or of little use if the person either already knows the trick, or just doesn't figure it out. Fizzbuzz on the other hand makes a lot of sense when you're interviewing someone who you don't have any other knowledge of their coding. That type of question absolutely makes a good first pass filter: the problem is easy to explain, the implementation is trivial to do in a few minutes if you are at all qualified. Sure, it's annoying for an experienced person to have to do a fizzbuzz variant for the 200th time, but given that it takes almost no time, and helps the interviewer quickly jump to more advanced stuff or cut off an interview that isn't going to go anywhere, it's a pretty minor inconvenience.