6 ms·

This is not, nor are similar whiteboard questions, a good interviewing problem. Not because the problem is particularly difficult or uninteresting, but because

by seekely 15y ago

This is not, nor are similar whiteboard questions, a good interviewing problem. Not because the problem is particularly difficult or uninteresting, but because no matter how much the interviewer wants these questions to just be 'signals' of your ability, the candidate's performance at the whiteboard will likely be the determining factor for landing a job. The whiteboard gets too much weight, because it's too hard for an engineer (or likely anybody) to ignore somebody having a bad moment and flailing on something that seems so trivial to the interviewer. And this is why Google's interviewing process and reliance on these problems is terrible. You are put in five 1 hour sessions where you are asked Pascal Triangle esque whiteboard problems. Get a bit intimated? Stressed because of the interview? Miss your morning coffee? Start to panic a little bit? And suddenly you stumble on 1 or 2 of the problems and make an embarrassing impression. At this point, it does not matter how strong your portfolio is, how many recommendations from fellow Googlers you have received, past performances, or your samples of past code (all of which are much stronger signals than a whiteboard problem). You will not be getting the job.

jameshart 15y ago
I hear this complaint a lot, but I kind of don't get it. I just don't know any way to make job interviews fair for people who are having a bad day. Can we just stipulate that any interview technique will fail to recognise the potential of some candidates? False negatives of that sort are kind of inevitable. And from the point of view of a hiring company? a few false negatives aren't the end of the world. A false positive is way more costly.
- seekely 15y ago
  Because a candidate with a mediocre background yet a strong showing on the whiteboard will get the job. The problem isn't not hiring somebody having a bad day. The problem is the over reliance on the whiteboard signal, despite a candidate's history of contradictory evidence (either positive or negative). If you are going to use typical whiteboard problems as a signal, it is really really hard to not heavily weigh the results, especially when sour.
  - spacemanaki 15y ago
    "Because a candidate with a mediocre background yet a strong showing on the whiteboard will get the job." Is there evidence that this happens a lot? Does this happen at Google, where whiteboard programming during interviews is common? "The problem is the over reliance on the whiteboard signal, despite a history of contradictory evidence (either positive or negative). If you are going to use typical whiteboard problems as a signal, it is really really hard to not heavily weigh the results, especially when sour." What history of contradictory evidence? Do you have any suggestions for other signals that have proven effective in your experience? I have very, very little experience as an interviewer and no experience dealing with and evaluating the success of new hires in the long term, so I definitely don't claim to have the answers here.
    - seekely 15y ago
      Sorry, the 'history of contradictory evidence' should have read the 'candidate's history of contradictory evidence' (e.g. having a really strong portfolio and work history to contradict a poor showing on the whiteboard). I try to interview a candidate like he is a friend I haven't caught up with in many years. I want to have as honest and interesting of a conversation as possible, hear all his opinions, and learn all about what he has done and can do. If I feel the conversation was honest, the personalities were compatible, and there is evidence of good work and reliability in the past, I will likely consider the candidate a hire. The problem with this method can be 1) It relies on the candidate having some sort of tangible history (referrals, solid portfolio or resume, open source projects, etc...), so new grads are tougher for example. 2) Having an 'on the level' conversation with somebody who knows they are being interviewed is not always possible. Nor are all interviewers capable of extracting useful information this way.
  - jameshart 15y ago
    Your argument is that using coding tests under interview conditions is bad because a poor interviewer will overweigh it in their assessment. But the same applies to everything that takes place in an interview. Some interviewers will make up their mind on the candidate's handshake, so it's probably not fair to have handshakes in an interview. Others might place too much weight on the vocabulary and language an interviewee uses in answering questions, so we should probably not let them hear the interviewee speak. And asking candidates to write code risks the interviewer placing too much emphasis on the interviewee's ability to demonstrate their coding skills, so we should prohibit that too.
- sontek 15y ago
  We hire based on personality. Before we bring an interviewee in we have looked at their bitbucket, github, portfolio, etc.. By time they get to the interview we know they are good enough to work for us, so we just talk technology with them and gauge based on that. If the person is inexperienced or does not have a very public profile (no opensource projects), then we will drop back and throw them some random problems to solve but never a puzzle, since most people just memorize the solutions to puzzles or learn the solutions after doing enough interviews. We prefer giving them problems like writing a text based guess the number game or tic tac toe. Mindless easy to solve project. They could spit it out in the shortest number of lines, which is what the bad puzzle questions encourage, but we are looking for how well the design/architect code. So the person who creates a good class structure and writes tests for his/her game is the one who gets hired, not the person who prefers shortest lines of code.
  - victorhn 15y ago
    Can you be more specific about what do you mean by personality and how do you measure?
    - sontek 15y ago
      Our interviews are never in the office. They are at a restaurant or bar. We just have drinks and food with them and chat. Talk about technology, them, us, etc... If we can have lunch and drinks with someone and they are talented then they are a good fit for the team. The lunch/bar setting also loosens the interviewee up and they are always less nervous than if we had brought them into the conference room. We don't care how talented you are if we can't stand being around you. If we had to choose between a mediocre programmer who has a lot of passion and fun to be around or a brilliant programmer who is arrogant, we'll take the mediocre guy and just train him up into being a brilliant programmer.
      - orangecat 15y ago
        You're selecting for extroverts at the expense of introverts. Which may be desirable for your company, but I would (somewhat selfishly) not want that to be the common case. we'll take the mediocre guy and just train him up into being a brilliant programmer If you actually have a successful process for doing that, you could be owning many islands.

by seekely 15y ago

jameshart 15y ago

I hear this complaint a lot, but I kind of don't get it. I just don't know any way to make job interviews fair for people who are having a bad day. Can we just stipulate that any interview technique will fail to recognise the potential of some candidates? False negatives of that sort are kind of inevitable. And from the point of view of a hiring company? a few false negatives aren't the end of the world. A false positive is way more costly.

seekely 15y ago
Because a candidate with a mediocre background yet a strong showing on the whiteboard will get the job. The problem isn't not hiring somebody having a bad day. The problem is the over reliance on the whiteboard signal, despite a candidate's history of contradictory evidence (either positive or negative). If you are going to use typical whiteboard problems as a signal, it is really really hard to not heavily weigh the results, especially when sour.
- spacemanaki 15y ago
  "Because a candidate with a mediocre background yet a strong showing on the whiteboard will get the job." Is there evidence that this happens a lot? Does this happen at Google, where whiteboard programming during interviews is common? "The problem is the over reliance on the whiteboard signal, despite a history of contradictory evidence (either positive or negative). If you are going to use typical whiteboard problems as a signal, it is really really hard to not heavily weigh the results, especially when sour." What history of contradictory evidence? Do you have any suggestions for other signals that have proven effective in your experience? I have very, very little experience as an interviewer and no experience dealing with and evaluating the success of new hires in the long term, so I definitely don't claim to have the answers here.
  - seekely 15y ago
    Sorry, the 'history of contradictory evidence' should have read the 'candidate's history of contradictory evidence' (e.g. having a really strong portfolio and work history to contradict a poor showing on the whiteboard). I try to interview a candidate like he is a friend I haven't caught up with in many years. I want to have as honest and interesting of a conversation as possible, hear all his opinions, and learn all about what he has done and can do. If I feel the conversation was honest, the personalities were compatible, and there is evidence of good work and reliability in the past, I will likely consider the candidate a hire. The problem with this method can be 1) It relies on the candidate having some sort of tangible history (referrals, solid portfolio or resume, open source projects, etc...), so new grads are tougher for example. 2) Having an 'on the level' conversation with somebody who knows they are being interviewed is not always possible. Nor are all interviewers capable of extracting useful information this way.
- jameshart 15y ago
  Your argument is that using coding tests under interview conditions is bad because a poor interviewer will overweigh it in their assessment. But the same applies to everything that takes place in an interview. Some interviewers will make up their mind on the candidate's handshake, so it's probably not fair to have handshakes in an interview. Others might place too much weight on the vocabulary and language an interviewee uses in answering questions, so we should probably not let them hear the interviewee speak. And asking candidates to write code risks the interviewer placing too much emphasis on the interviewee's ability to demonstrate their coding skills, so we should prohibit that too.
sontek 15y ago
We hire based on personality. Before we bring an interviewee in we have looked at their bitbucket, github, portfolio, etc.. By time they get to the interview we know they are good enough to work for us, so we just talk technology with them and gauge based on that. If the person is inexperienced or does not have a very public profile (no opensource projects), then we will drop back and throw them some random problems to solve but never a puzzle, since most people just memorize the solutions to puzzles or learn the solutions after doing enough interviews. We prefer giving them problems like writing a text based guess the number game or tic tac toe. Mindless easy to solve project. They could spit it out in the shortest number of lines, which is what the bad puzzle questions encourage, but we are looking for how well the design/architect code. So the person who creates a good class structure and writes tests for his/her game is the one who gets hired, not the person who prefers shortest lines of code.
- victorhn 15y ago
  Can you be more specific about what do you mean by personality and how do you measure?
  - sontek 15y ago
    Our interviews are never in the office. They are at a restaurant or bar. We just have drinks and food with them and chat. Talk about technology, them, us, etc... If we can have lunch and drinks with someone and they are talented then they are a good fit for the team. The lunch/bar setting also loosens the interviewee up and they are always less nervous than if we had brought them into the conference room. We don't care how talented you are if we can't stand being around you. If we had to choose between a mediocre programmer who has a lot of passion and fun to be around or a brilliant programmer who is arrogant, we'll take the mediocre guy and just train him up into being a brilliant programmer.
    - orangecat 15y ago
      You're selecting for extroverts at the expense of introverts. Which may be desirable for your company, but I would (somewhat selfishly) not want that to be the common case. we'll take the mediocre guy and just train him up into being a brilliant programmer If you actually have a successful process for doing that, you could be owning many islands.