9 ms·

Whiteboard Interviews Suck, Get Good at Them Anyway

59 points by akras14 9y ago

Pica_soO 9y ago
Whats funny about WBIs is that if you decide for yourself during the interview that this is not the company for you- you can derail the conversation into unfamiliar territory's and watch the interviewers struggle.
- blueplanet200 9y ago
  Why is that funny? Most interviewers are just doing what their company asked them, to give a whiteboarding problem to a potential candidate and figure out if they're worth hiring based on that. I agree the process sucks and it's a hoop to jump through, but why take it out on the interviewer?
  - dkarapetyan 9y ago
    Because they are complicit. Saying they're just doing their job is not an excuse for propagating exclusionary and low signal interview practices.
    - izacus 9y ago
      "Exclusionary?" Seriously? I didn't know you're entitled to a job.
      - Will_Parker 9y ago
        Notice how it's a whiteboard and not a board of colour.
        dkarapetyan 9y ago
        There are blackboards as well. Color or ethnicity of the board is not the problem.
        GhostVII 9y ago
        But whiteboards are far overrepresented. Definitely an issue.
      - deleted 9y ago
        [deleted]
    - lr4444lr 9y ago
      What part of the interview process doesn't have a purposeful exclusionary element? You're trying to weed out unsuitable candidates.
      - dkarapetyan 9y ago
        Better ways to do that than spend 5 hours in front of a whiteboard. Never gone through a whiteboard interview and then thought to myself that was time well spent, neither on the interviewer nor interviewee side. I've written integer programs, ray tracing simulation software for fields of heliostats, custom orchestration tools and deployment pipelines, etc. and not once has anything said or done during a whiteboard interview been relevant to any of it. The actual job is pretty much orthogonal to whatever the interviewer thinks they are figuring out during a whiteboard interview and as above the stuff I've worked on goes over the heads of 90% of the programmer population so 9/10 they can't and are not qualified to judge things properly but somehow they still think through the magic of the whiteboard they'll divine the candidate's true capabilities. When working on hard problems I go to the library and don't stand in front of a whiteboard to see if I'm smart enough to reinvent some wheel. Chances are someone else has already figured it out and I just need to read some paper and then implement it. If they haven't figured it out then that means the problem is harder than I thought and will either need to rethink my approach entirely or spend my time on something else. Unless of course the job is to stand in front of a whiteboard and wave furiously, in which case you should be using a whiteboard to interview.
    - GhostVII 9y ago
      Doing their job is a perfect excuse for doing a whiteboard problem during an interview. If the company has standards for interviews, you should follow them, it would be pretty arrogant to just do it a different way because you think you know better than the standards.
      - Pica_soO 9y ago
        Thats a missunderstanding on what standards are. Standards are minimal levels to fullfill- everyone is free to surpass them. And the first sentence of any well written standard is- "This rules may not apply universally, use your brain in case of emergencys". I get that its a horrible thing for some to consider that there is creative freedom involved in work and not everything in life follows exact and well defined rules. Anything else, is just holding humanity back. I ve seen standards written by assembly programmers, still alive and walking the earth, torturing hundreds and undead for ever, because what is once set in motion. Really good standards, standardize intentions. "Every Programm shall consists of layers and components. Every component shall be sperated by a interface." A bad standard tells you how the interface has to look like- every time.
  - tutufan 9y ago
    There is kind of a symmetry to it. Get a question that the asker has studied and watch the answerer squirm. Seems like it'd be more fun, and probably more insightful, to take turns... :-)
    - izacus 9y ago
      Also great way to get marked as "that asshole on the interview". People know each other in the industry.
      - Pica_soO 9y ago
        Yes, one only has to proof competency on those topics once- after that to ask - is intentionally embarrassing.
      - dkarapetyan 9y ago
        So then you're advocating for an appeasement ritual? Most interviewers would rather feel smart than talk to another person as an equal. More than a few times I've seen candidates disqualified because they made the interviewer feel inadequate. Who exactly is the asshole in that case?
        izacus 9y ago
        No, I'm not advocating for anything like that. But if you want to deliberately make fun or piss off your interviewer, remember that that may have later consequences on your career. Nothing you do is in vacuum and there's no such thing as "just business" when insulting behaviour is involved. (Obviously, this goes both ways, shitty companies tend to have trouble getting good people to interview.)
- otoburb 9y ago
  >>[...] if you decide for yourself during the interview that this is not the company for you [...] ... then perhaps it's best to let them know and save everybody some time (including your own).
  - deleted 9y ago
    [deleted]
- xyzzyz 9y ago
  I do whiteboard interviews at the whiteboard interview Mecca (Google), and detecting the digressions and putting the candidate back on the right track, is one of the skills the interviewers are explicitly trained for.
  - scott00 9y ago
    My mental model of how candidates with a CS degree do on a whiteboard interview is something like score = (grade in data structures class) + (grade in algorithms class) - k(time since undergrad) + b(time spent studying for interview). I'm curious if you think that's a good model, and if it is, would it be easier to just look at their grades in those classes, or if you actually care about the amount of time spent studying.
    - nissimk 9y ago
      It's not just the time spent studying, it's the right kind of studying. "Teaching to the test" so to speak. I liken it to my driver's ed class in high school. My teacher wasn't teaching us how to drive, he was teaching us how to pass the test. Every single thing he taught us was about how to pass the test. I passed on my first try because I was very well trained. Unfortunately, the whiteboard thing is not as standardized, so it's somewhat harder to teach to the test, but it is not impossible. People waste a lot of time I think when they can memorize a small number of basic algorithms and then learn to map the question given to one of those techniques. But writing the code with a marker on a board definitely takes practice as well. The available training materials are good, but they are not great. A focus on shortcuts would be nice. I remember my verbal SAT improving significantly just from the single tip to read the reading comprehension questions before reading the passage. I suspect there are similar tips that can improve your speed at figuring out the trick that solves these interview problems quickly.
    - xyzzyz 9y ago
      There's a huge variance between different schools, and an average grade from one university may be better than good grade from some different one. Additionally, I don't really ask questions that require some particular knowledge from algorithm classes -- it would be silly to determine whether someone is a good hire based on whether they remember details of, say, Hopcroft-Karp algorithm. On the other hand, many algorithm classes are all about this kind of stuff. So, I generally feel like it makes more sense to try to determine the actual aptitude when you have an opportunity, instead of trying to measure it by possible decades old proxy with huge and unknown variance. If you had good grade and/or spent some time studying for the interview, and really meet the hiring bar, you'll most likely do well in the interviews anyway. There's some noise in the process, and good qualified candidates are sometimes rejected, but from company's perspective, it's better to have some false negatives, than to give another metric that can be easily gamed, and get false positives. Google has enough qualified candidates applying for a job to make it not that big of a deal if you lose some due to the noise.
1_800_UNICORN 9y ago
Any company with an interview process where the key to passing the interview is "Pay more attention in your Algorithms and Data Structures classes", is a company that I'm never taking a job at.
- guelo 9y ago
  Why would I want to hire a candidate that paid thousands of dollars for Algorithms and Data Structures classes but didn't feel the need to pay attention? Don't you think that says something about the candidate?
  - 1_800_UNICORN 9y ago
    I've been out of college for 10 years, writing software professionally... it's so rare to have to calculate the big-O of an algorithm, or use a data structure much fancier than an array or a hash table, or implement my own search algorithm. And if I had to do any of those things, I would look it up and leverage this collaborative thing we've created called the internet. So why should I be expected to regurgitate trivia at a whiteboard to get your job?
    - guelo 9y ago
      It's weird to me seeing statements like this because I don't work on any esoteric or fancy code, just frontend UIs, but I'm frequently making data structure and algorithm decisions.
      - pb8226 9y ago
        Then ask questions about the UI framework the developer has experience in and ask about real problems faced with that framework for a complex UI.
  - rootusrootus 9y ago
    If I'm understanding 1_800_UNICORN's point correctly, it's more along the lines of 'any company that expects an experienced developer to recite academic details on algorithms'. Certainly my current knowledge of algorithms is very practical and I'm certain I've forgotten much of the academic stuff I learned 20 years ago. You remember that which is useful, forget that which is not (situational, of course) and when a company is looking for the latter, they're unrealistic or they want an academic programmer, which I'm not.
rabble 9y ago
If an org does WBI's then it's a good indication that they are not choosing the best applicants. Instead of either getting good at them or trying to get the industry to change, we should embrace this. If they do WBI's then they probably do lots of other things which run counter to best practices. Isn't it so much better to expose to the outside world a culture of cargo culting inability to evaluate techniques for the efficacy. There are so many orgs who do run themselves well. The real tricky party is how to determine that quickly. With WBI's we can get a short cut. Long Live The White Board Interview!
- akras14 9y ago
  How would you hire?
  - samastur 9y ago
    Mostly pair-programming coding session on site or homework that is later discussed and "developed" on site. Candidate picks which option works best for her.
  - tutufan 9y ago
    Conversation, half technical and half to figure out what the person would be like to work with. For example, tell me about the most interesting/difficult/satisfying problem was that you've worked on. Or, tell me what happens when you point your browser at a web site, in as much detail as you can muster. What are your go-to tools to solve problems
- hn_anon_founder 9y ago
  >If an org does WBI's then it's a good indication that they are not choosing the best applicants I don't think the data supports this. Say what you will about Google, but they do WBIs and the average quality of their engineers is very very high. Same with FB. How would you hire?
- rootusrootus 9y ago
  I'd like to agree with you but my current gig involved a WBI and the company itself actually has stellar software development processes and tools. It might just have been that the hiring manager wasn't too comfy doing it any other way, but in any case it did not correlate strongly to the culture inside the company. Perhaps use it as a warning sign, but you might want to talk to your potential co-workers as well and see what they're actually doing and how.
dkarapetyan 9y ago
Much better to go through triplebyte or interviewing.io. Pretty sure the signal from them is much higher than whiteboard interviews.
NumberSix 9y ago
The article makes the following statement: Whiteboard interviews have a very high false negative rate (rejecting people who are good), but they also have a low false positive rate (hiring people who are not good). On what basis is this statement made? No study or source appears to be cited. The specific false negative and false positive rates are not given. Also, what exactly is the definition of "good" used?
- nissimk 9y ago
  It's not possible to compute the false negative rate because once you turn down a candidate you get no more information. I suppose if people apply multiple times and are accepted later and then go on to do well on the job, you can count them as false negative in the first interview. The sample size for that would be small though. And I'm sure the true believers would say that these candidates improved their programming skills between the two interviews, not their interviewing skills. I am quite interested in the false positive rate which can be computed by correlating performance reviews with interview performance.
  - rlanday 9y ago
    > I suppose if people apply multiple times and are accepted later and then go on to do well on the job, you can count them as false negative in the first interview. Isn't it more likely that they were correctly rejected multiple times and just got lucky enough to squeak through one time?
  - zzzcpan 9y ago
    > I am quite interested in the false positive rate which can be computed by correlating performance reviews with interview performance. You would still be excluding all the false negatives making it kind of relative performance of interview survivors. Meaning that you won't have anything to compare your false positive rate to proclaim it as low or as high.
    - nissimk 9y ago
      You could compare it to other interview techniques if you were willing to test them.
opportune 9y ago
I've never understood the hate that WBI get here and elsewhere on the internet. I see them the same way as standardized tests: without them, there's no good way to independently evaluate people's abilities in a way that is immune to cheating, dishonesty, or fraud (e.g. stealing someone's code and putting it on your github repo, being a terrible employee at previous jobs but just scraping by long enough to move onto the next company, straight up lying on a resume, etc.). Sure, as the author notes, WBI have a high false negative rate. But if you're actually a good programmer, learning basic CS concepts should be extremely easy, and these types of problems, perhaps with a bit of practice, should be relatively trivial. I personally wouldn't want to work with someone who can't extract a path from a DFS, not out of any sense of elitism but just because such an employee likely would be unable to do many other problems properly.
- rimliu 9y ago
  > there's no good way to independently evaluate people's > abilities And all WBI let you do is to evaluate someone's ability to do WBI.
  - opportune 9y ago
    I'd argue that there is a not-insignificant amount of overlap between whiteboarding and coding. Most coding in practice is not as "algorithmic" (in the sense that the problems are theoretically challenging) but having the knowledge and know-how of how to write optimal/efficient algorithms will definitely be echoed throughout any code you write. In addition, in my experience most WBI are not particularly challenging problems that you have probably never seen before. I mentioned the ability to extract a path from a DFS; I would argue that this is such a basic CS skill that being unable to solve that problem indicates an insufficient amount of knowledge of CS. There are definitely WBI that are poorly matched with real job requirements and knowledge. I think these are often seen at smaller companies trying to copy FB/Google without knowing what they are doing. WBI that involve using important CS constructs such as a binary tree or a directed graph IMO are testing important concepts
  - pebers 9y ago
    Sure, but you could make that argument about anything you ask candidates in interviews. At some point you have to ask something that attempts to evaluate their performance in an interview timespan. On the other hand, people's performance in whiteboard interviews may positively correlate with later performance as employees. In which case they let you make a reasonable assumption about later performance without being able to evaluate it directly, which is really what interviews are all about.
cholantesh 9y ago
>Whiteboard interviews have a very high false negative rate (rejecting people who are good), but they also have a low false positive rate (hiring people who are not good). That's a pretty bold assertion. Is there data to show this?
adrr 9y ago
Reasoning behind why whiteboard interviews suck is based that doesn't model real job skills. Maybe our org is an outlier but but we heavily rely on whiteboards to plan out architecture(ERDs,flow), problem solving, training/knowledge sharing. Our trello cards sometimes contains images of whiteboarded diagrams. Being able to explain and whiteboard your stuff is a very important skill at organization.
- jay-anderson 9y ago
  Sketching out ideas and preliminary diagrams is very different from programming on a whiteboard. If a whiteboard interview were talking at a higher level about the problems at hand then I don't think it'd be such a problem. However, that doesn't seem to be the case. In whiteboard interviews you're asked to write detailed code on a board.
- rimliu 9y ago
  How many images in your trello contain C code for reversing string?
  - neeleshs 9y ago
    Yep, whiteboard programming - not good . Whiteboard interviews is too generic a name to use for that.
- ng12 9y ago
  I'm fairly convinced much of the hate comes from people who interview at firms who do WBI incorrectly. The question I give is tailored for the following points: You can pick up unfamiliar code/data structures easily You can understand a given problem space without too much explaining You can code the "easy stuff" without thinking about it too much You can distill and discuss the difficult parts of an algorithm All of which are traits I consider key for someone I'll be working with.
jaredcwhite 9y ago
WBIs will go away if the developer community rallies around not doing them. Companies can offer crazy incentives to work there like free massage, organic lunches, and game rooms, but they can't figure out a better way to hire than conducting WBIs? I call BS. They'll figure it out when they're pressured to do so...aka they can't find good talent anymore. Let's put on the pressure.
- _dark_matter_ 9y ago
  There is no benefit for people who are good at WBI's to join you in protest, and the bad people weren't going to pass them anyways, so the companies won't care either. On top of that, anyone giving a WBI got hired with a WBI, so the cycle continues. I am not saying WBIs are good or bad - just stating that this kind of rally won't have the desired effect.
krosaen 9y ago
In a similar vein, I view whiteboard interviews (and required prep) similar to professional exams, like the Bar exam for lawyers. While I think it's unreasonable to expect good professionals to be able to pass such exams without a lot of practice, it's perhaps not as unreasonable to expect good professionals to be able to pass such exams if they study. Nobody likes tests, but I suspect that a high percentage of the good programmers who belly ache about whiteboard interviewers could in fact get good at them if they put in the time. So yes, whiteboard interviews or not optimal and they are unfair, but if you are rejecting them out of hand and refusing to study for them, you are only taking away some opportunities. I say this as someone who hasn't had to do a whiteboard interview for 10+ years. The jury is out as to whether I'll take the pain to study for them again should I do a full job search among tech companies again. Also: note to author, I believe, "They have little co-relation" in your opening paragraph should be "They have little correlation".
lr4444lr 9y ago
I used to believe that I got rejected after many whiteboard interviews, not because I did poorly, but because I didn’t have the same (right) background as the interviewer. I served in the military, I was a Front End Engineer, I didn’t go to a fancy school, I used the wrong language, I’ve used the wrong framework etc. I think the author is hitting on some deep-seated psychological aversion people have to white board interviews. If you can identify that you were turned down based on your failure to do it well, you have to accept that it's your fault - not that you happened not to be good at that tech stack, or didn't have a "personality fit". (I know there are different kinds of white board interviews - I'm assuming the generic pseudo-code algos and data structures type in keeping with the article's generality.) Exposure to that kind of evidence of your inadequacy can be hard to accept. But it's also remediable. Get cracking and practice.
neeleshs 9y ago
I see a lot of naysayers for whiteboard, but pretty much no opinion on what they think is better than whiteboards.
- hibikir 9y ago
  Small project, on your own time, expected to take about 3 hours or so. Automated tests can do the very basics of evaluation. Then, 1 to 2 hours explaining the decisions made, and adding one more feature to the project. It lowers stress levels of the candidate, as they get to a lot of the thinking in advance. It's practical, as it requires more detailed programming skills. It provides a better sample of the candidate, as 5 20 lines of code snippets don't compare with a larger codebase, and makes the candidate explain something they really did themselves. Total time spent by the candidate is probably about the same, but the employers saves engineering time. I've been through that system a couple of times, it was both more enjoyable and gave people a far better picture of my skills than just how well I remember a tree traversal algorithm that I haven't had to write in forever, as data structures often handle their own traversal.
- metaphorm 9y ago
  need context to determine what would be better. giving the same interview for every position you're hiring for is not usually the right thing to do. Google is very unusual in their practice of hiring into a general pool. Most companies hire for more specific openings. In general though, when I interview it has three main sections, tailored for the role. First, is conversational. Talk about the resume, talk about work experience, talk about technology, etc. Second, is a practical example. Sometimes (but not always) this involves solving an algorithmic problem on a whiteboard. Sometimes it involves a guided tour of the codebase and functionality of a relevant project the candidate has worked on. Rarely it might be a take-home coding task. Third, is a more general example usually of some kind of system design problem. A lot of candidates like to write/draw on the whiteboard for this section but it's not strictly necessary nor do I ask them to do so. There's a board in the room if they want to but I don't ask them to use it.
- falcolas 9y ago
  Talk to them. Have a conversation about "how would you solve this (real) problem" - the same kind of conversation you have with a peer when they're stuck on a problem. If both you and they are professional developers, you will be able to get into a nice in-depth discussion of the pros and cons, and go into nice tangental discussions along the way. If they aren't a professional developer, it will show in no time at all; the longest I've let such a conversation go was 10 minutes just to see how long the interviewee would keep trying and snowball me. I've found this is simply much faster and more informative about their experience with subjects that matter in the daily work of a software dev.
cubano 9y ago
What I remember about much of that code I looked at in DS classes way back in the 80's was that, due to very limited memory space, all the variables were like I,J,K and X,Y,Z. You have no idea how much more difficult that made it to follow sorts and tree algorithms. Thankfully, I seemed to have some sort of coding gift that allowed me to write the damn things at the terminal and make them work. I got A's in all my CS classes, but now I guess I am so used to the magics of PHP object handling I've literally stopped caring about that shit. It's hard to see why I would want to again.
metaphorm 9y ago
I think one of the real problems that doesn't get discussed very much is Google-envy. A lot of companies that don't really have problems like Google, and don't really have organizations like Google, and don't really have applicant pools like Google, will yet interview the same way Google does. It really might be the correct way to interview for Google. Certainly they've put a lot of thought into how to manage their hiring process and they still lean heavily on the whiteboard. I'm almost positive, however, that most companies are really nothing like Google and really should not be using the same interviewing techniques. Smaller companies with more quotidian problems would be better served with a more conversational and less time-pressured interview style.
- scrpn 9y ago
  Google explains here their interview observations : https://www.google.ro/amp/s/www.wired.com/2015/04/hire-like-google/amp/ https://www.google.ro/amp/s/www.wired.com/2015/04/hire-like-...
nunez 9y ago
i 1000% agree with this. there is no good way to really know whether a candidate is good or not, but their ability to express their thoughts, in code, on a whiteboard is a really strong indicator that they can think through a problem and (more importantly) communicate a solution as a software engineer. the next best approach --- a take home assignment --- is also good at this (or better, IMO, since you're free to implement that solution however you damn well please and it more closely translates into the work they'll actually be doing), but people complain that this is too invasive of their time and that they are doing free work. which is it? I have to imagine that what most people want is to be asked about their experience, talk through it, and expect that to be enough. which is a very solid first pass filter. it's just not strong enough of a filter to extend an offer with confidence in many situations; it's too easy to bullshit and most of us engineers are damn expensive. (it's not just salary; we want dedicated offices and sit stand desks and super comfy chairs and free food and drinks and games and cadillac health insurance and 401ks and so on and so on) i also never understood why whiteboarding was hard. if you know how to code and know your language passably well, then why can't you write code on something that isn't a computer?
mbrodersen 9y ago
I have hired a lot of developers, working for different companies in different countries. And I have never ever seen a case where a developer did well in the WBI interview, was hired, and then turned out later to not be a good developer. However I have seen plenty of examples of developers (interviewed by somebody else) who didn't do a WBI interview and later was fired because they were not good developers. My point is that if you can't do a WBI, well then perhaps you are simply not as good a developer as you think you are. Developing is about thinking. And WBI style interviews force you to think for yourself without copy-pasting a solution from Google.
GhostVII 9y ago
I think whiteboard questions are great if you make them either pretty easy questions, or more complex questions that you just diagram and discuss. In the first case, you filter out poor coders very easily, and see how the better coders apporach the problem. In the second case, you learn about their thought process when approaching a problem where you can't just modify an algorithm you have already practiced and answer it.
samblr 9y ago
Im a 'competent' programmer who has done few whiteboard interviews in my time and I have failed in most of them. Here is why ? I consider myself a competent programmer since I have developed decent work@work and my peers recognise it too. But how would any interviewer can know about this ? My resume is like any other resume in tech world. They cannot. So I've to be put through WBI. For many years I couldn't have known what was wrong with me. Every time I failed I felt miserable to be honest. Eventually, I have narrowed down the reason. It is not just at interview I fail to problem solve - even when I try doing at home with a white board and 'explanation' I fail. English is not my region language but one would not doubt on my conversation abilities as I manage it ok. It is that I simply fail to explain solving programming puzzles in english since while programming I 'do' think in my regional language and most of the time in 'silent' mode in my head. Now 'silent' mode is again dreaded in interviews - as interviewers they want to know 'how you approach solving a problem'. I think a better solution to WBI would be to make interviewee do programming questions on a laptop (alone) where interview is held - say make it an hour worth of exercise. Have that program run through some time bound tests (like google code jam etc) and may be do a code walk through later with interviewer. This way interviewer is sure interviewee can code. Also, one of things that will definitely help would be to have some sort of evidence that you can code (git, code competitions etc). This gives some sort of confidence to interviewer.
tutufan 9y ago
If you select stringently for WBI ability, you'll get strong WBIers. However, you're also paying the opportunity cost of not selecting for other traits and skills that might matter a lot more. +1 though for "Senior Whiteboard Engineer"--that term is genius!

59 points by akras14 9y ago

Whats funny about WBIs is that if you decide for yourself during the interview that this is not the company for you- you can derail the conversation into unfamiliar territory's and watch the interviewers struggle.

blueplanet200 9y ago
Why is that funny? Most interviewers are just doing what their company asked them, to give a whiteboarding problem to a potential candidate and figure out if they're worth hiring based on that. I agree the process sucks and it's a hoop to jump through, but why take it out on the interviewer?
- dkarapetyan 9y ago
  Because they are complicit. Saying they're just doing their job is not an excuse for propagating exclusionary and low signal interview practices.
  - izacus 9y ago
    "Exclusionary?" Seriously? I didn't know you're entitled to a job.
    - Will_Parker 9y ago
      Notice how it's a whiteboard and not a board of colour.
      - dkarapetyan 9y ago
        There are blackboards as well. Color or ethnicity of the board is not the problem.
        GhostVII 9y ago
        But whiteboards are far overrepresented. Definitely an issue.
    - deleted 9y ago
      [deleted]
  - lr4444lr 9y ago
    What part of the interview process doesn't have a purposeful exclusionary element? You're trying to weed out unsuitable candidates.
    - dkarapetyan 9y ago
      Better ways to do that than spend 5 hours in front of a whiteboard. Never gone through a whiteboard interview and then thought to myself that was time well spent, neither on the interviewer nor interviewee side. I've written integer programs, ray tracing simulation software for fields of heliostats, custom orchestration tools and deployment pipelines, etc. and not once has anything said or done during a whiteboard interview been relevant to any of it. The actual job is pretty much orthogonal to whatever the interviewer thinks they are figuring out during a whiteboard interview and as above the stuff I've worked on goes over the heads of 90% of the programmer population so 9/10 they can't and are not qualified to judge things properly but somehow they still think through the magic of the whiteboard they'll divine the candidate's true capabilities. When working on hard problems I go to the library and don't stand in front of a whiteboard to see if I'm smart enough to reinvent some wheel. Chances are someone else has already figured it out and I just need to read some paper and then implement it. If they haven't figured it out then that means the problem is harder than I thought and will either need to rethink my approach entirely or spend my time on something else. Unless of course the job is to stand in front of a whiteboard and wave furiously, in which case you should be using a whiteboard to interview.
  - GhostVII 9y ago
    Doing their job is a perfect excuse for doing a whiteboard problem during an interview. If the company has standards for interviews, you should follow them, it would be pretty arrogant to just do it a different way because you think you know better than the standards.
    - Pica_soO 9y ago
      Thats a missunderstanding on what standards are. Standards are minimal levels to fullfill- everyone is free to surpass them. And the first sentence of any well written standard is- "This rules may not apply universally, use your brain in case of emergencys". I get that its a horrible thing for some to consider that there is creative freedom involved in work and not everything in life follows exact and well defined rules. Anything else, is just holding humanity back. I ve seen standards written by assembly programmers, still alive and walking the earth, torturing hundreds and undead for ever, because what is once set in motion. Really good standards, standardize intentions. "Every Programm shall consists of layers and components. Every component shall be sperated by a interface." A bad standard tells you how the interface has to look like- every time.
- tutufan 9y ago
  There is kind of a symmetry to it. Get a question that the asker has studied and watch the answerer squirm. Seems like it'd be more fun, and probably more insightful, to take turns... :-)
  - izacus 9y ago
    Also great way to get marked as "that asshole on the interview". People know each other in the industry.
    - Pica_soO 9y ago
      Yes, one only has to proof competency on those topics once- after that to ask - is intentionally embarrassing.
    - dkarapetyan 9y ago
      So then you're advocating for an appeasement ritual? Most interviewers would rather feel smart than talk to another person as an equal. More than a few times I've seen candidates disqualified because they made the interviewer feel inadequate. Who exactly is the asshole in that case?
      - izacus 9y ago
        No, I'm not advocating for anything like that. But if you want to deliberately make fun or piss off your interviewer, remember that that may have later consequences on your career. Nothing you do is in vacuum and there's no such thing as "just business" when insulting behaviour is involved. (Obviously, this goes both ways, shitty companies tend to have trouble getting good people to interview.)
otoburb 9y ago
>>[...] if you decide for yourself during the interview that this is not the company for you [...] ... then perhaps it's best to let them know and save everybody some time (including your own).
- deleted 9y ago
  [deleted]
xyzzyz 9y ago
I do whiteboard interviews at the whiteboard interview Mecca (Google), and detecting the digressions and putting the candidate back on the right track, is one of the skills the interviewers are explicitly trained for.
- scott00 9y ago
  My mental model of how candidates with a CS degree do on a whiteboard interview is something like score = (grade in data structures class) + (grade in algorithms class) - k(time since undergrad) + b(time spent studying for interview). I'm curious if you think that's a good model, and if it is, would it be easier to just look at their grades in those classes, or if you actually care about the amount of time spent studying.
  - nissimk 9y ago
    It's not just the time spent studying, it's the right kind of studying. "Teaching to the test" so to speak. I liken it to my driver's ed class in high school. My teacher wasn't teaching us how to drive, he was teaching us how to pass the test. Every single thing he taught us was about how to pass the test. I passed on my first try because I was very well trained. Unfortunately, the whiteboard thing is not as standardized, so it's somewhat harder to teach to the test, but it is not impossible. People waste a lot of time I think when they can memorize a small number of basic algorithms and then learn to map the question given to one of those techniques. But writing the code with a marker on a board definitely takes practice as well. The available training materials are good, but they are not great. A focus on shortcuts would be nice. I remember my verbal SAT improving significantly just from the single tip to read the reading comprehension questions before reading the passage. I suspect there are similar tips that can improve your speed at figuring out the trick that solves these interview problems quickly.
  - xyzzyz 9y ago
    There's a huge variance between different schools, and an average grade from one university may be better than good grade from some different one. Additionally, I don't really ask questions that require some particular knowledge from algorithm classes -- it would be silly to determine whether someone is a good hire based on whether they remember details of, say, Hopcroft-Karp algorithm. On the other hand, many algorithm classes are all about this kind of stuff. So, I generally feel like it makes more sense to try to determine the actual aptitude when you have an opportunity, instead of trying to measure it by possible decades old proxy with huge and unknown variance. If you had good grade and/or spent some time studying for the interview, and really meet the hiring bar, you'll most likely do well in the interviews anyway. There's some noise in the process, and good qualified candidates are sometimes rejected, but from company's perspective, it's better to have some false negatives, than to give another metric that can be easily gamed, and get false positives. Google has enough qualified candidates applying for a job to make it not that big of a deal if you lose some due to the noise.

1_800_UNICORN 9y ago

Any company with an interview process where the key to passing the interview is "Pay more attention in your Algorithms and Data Structures classes", is a company that I'm never taking a job at.

guelo 9y ago
Why would I want to hire a candidate that paid thousands of dollars for Algorithms and Data Structures classes but didn't feel the need to pay attention? Don't you think that says something about the candidate?
- 1_800_UNICORN 9y ago
  I've been out of college for 10 years, writing software professionally... it's so rare to have to calculate the big-O of an algorithm, or use a data structure much fancier than an array or a hash table, or implement my own search algorithm. And if I had to do any of those things, I would look it up and leverage this collaborative thing we've created called the internet. So why should I be expected to regurgitate trivia at a whiteboard to get your job?
  - guelo 9y ago
    It's weird to me seeing statements like this because I don't work on any esoteric or fancy code, just frontend UIs, but I'm frequently making data structure and algorithm decisions.
    - pb8226 9y ago
      Then ask questions about the UI framework the developer has experience in and ask about real problems faced with that framework for a complex UI.
- rootusrootus 9y ago
  If I'm understanding 1_800_UNICORN's point correctly, it's more along the lines of 'any company that expects an experienced developer to recite academic details on algorithms'. Certainly my current knowledge of algorithms is very practical and I'm certain I've forgotten much of the academic stuff I learned 20 years ago. You remember that which is useful, forget that which is not (situational, of course) and when a company is looking for the latter, they're unrealistic or they want an academic programmer, which I'm not.

rabble 9y ago

If an org does WBI's then it's a good indication that they are not choosing the best applicants. Instead of either getting good at them or trying to get the industry to change, we should embrace this. If they do WBI's then they probably do lots of other things which run counter to best practices. Isn't it so much better to expose to the outside world a culture of cargo culting inability to evaluate techniques for the efficacy. There are so many orgs who do run themselves well. The real tricky party is how to determine that quickly. With WBI's we can get a short cut. Long Live The White Board Interview!

akras14 9y ago
How would you hire?
- samastur 9y ago
  Mostly pair-programming coding session on site or homework that is later discussed and "developed" on site. Candidate picks which option works best for her.
- tutufan 9y ago
  Conversation, half technical and half to figure out what the person would be like to work with. For example, tell me about the most interesting/difficult/satisfying problem was that you've worked on. Or, tell me what happens when you point your browser at a web site, in as much detail as you can muster. What are your go-to tools to solve problems
hn_anon_founder 9y ago
>If an org does WBI's then it's a good indication that they are not choosing the best applicants I don't think the data supports this. Say what you will about Google, but they do WBIs and the average quality of their engineers is very very high. Same with FB. How would you hire?
rootusrootus 9y ago
I'd like to agree with you but my current gig involved a WBI and the company itself actually has stellar software development processes and tools. It might just have been that the hiring manager wasn't too comfy doing it any other way, but in any case it did not correlate strongly to the culture inside the company. Perhaps use it as a warning sign, but you might want to talk to your potential co-workers as well and see what they're actually doing and how.

dkarapetyan 9y ago

Much better to go through triplebyte or interviewing.io. Pretty sure the signal from them is much higher than whiteboard interviews.

NumberSix 9y ago

The article makes the following statement: Whiteboard interviews have a very high false negative rate (rejecting people who are good), but they also have a low false positive rate (hiring people who are not good). On what basis is this statement made? No study or source appears to be cited. The specific false negative and false positive rates are not given. Also, what exactly is the definition of "good" used?

nissimk 9y ago
It's not possible to compute the false negative rate because once you turn down a candidate you get no more information. I suppose if people apply multiple times and are accepted later and then go on to do well on the job, you can count them as false negative in the first interview. The sample size for that would be small though. And I'm sure the true believers would say that these candidates improved their programming skills between the two interviews, not their interviewing skills. I am quite interested in the false positive rate which can be computed by correlating performance reviews with interview performance.
- rlanday 9y ago
  > I suppose if people apply multiple times and are accepted later and then go on to do well on the job, you can count them as false negative in the first interview. Isn't it more likely that they were correctly rejected multiple times and just got lucky enough to squeak through one time?
- zzzcpan 9y ago
  > I am quite interested in the false positive rate which can be computed by correlating performance reviews with interview performance. You would still be excluding all the false negatives making it kind of relative performance of interview survivors. Meaning that you won't have anything to compare your false positive rate to proclaim it as low or as high.
  - nissimk 9y ago
    You could compare it to other interview techniques if you were willing to test them.

opportune 9y ago

I've never understood the hate that WBI get here and elsewhere on the internet. I see them the same way as standardized tests: without them, there's no good way to independently evaluate people's abilities in a way that is immune to cheating, dishonesty, or fraud (e.g. stealing someone's code and putting it on your github repo, being a terrible employee at previous jobs but just scraping by long enough to move onto the next company, straight up lying on a resume, etc.). Sure, as the author notes, WBI have a high false negative rate. But if you're actually a good programmer, learning basic CS concepts should be extremely easy, and these types of problems, perhaps with a bit of practice, should be relatively trivial. I personally wouldn't want to work with someone who can't extract a path from a DFS, not out of any sense of elitism but just because such an employee likely would be unable to do many other problems properly.

rimliu 9y ago
> there's no good way to independently evaluate people's > abilities And all WBI let you do is to evaluate someone's ability to do WBI.
- opportune 9y ago
  I'd argue that there is a not-insignificant amount of overlap between whiteboarding and coding. Most coding in practice is not as "algorithmic" (in the sense that the problems are theoretically challenging) but having the knowledge and know-how of how to write optimal/efficient algorithms will definitely be echoed throughout any code you write. In addition, in my experience most WBI are not particularly challenging problems that you have probably never seen before. I mentioned the ability to extract a path from a DFS; I would argue that this is such a basic CS skill that being unable to solve that problem indicates an insufficient amount of knowledge of CS. There are definitely WBI that are poorly matched with real job requirements and knowledge. I think these are often seen at smaller companies trying to copy FB/Google without knowing what they are doing. WBI that involve using important CS constructs such as a binary tree or a directed graph IMO are testing important concepts
- pebers 9y ago
  Sure, but you could make that argument about anything you ask candidates in interviews. At some point you have to ask something that attempts to evaluate their performance in an interview timespan. On the other hand, people's performance in whiteboard interviews may positively correlate with later performance as employees. In which case they let you make a reasonable assumption about later performance without being able to evaluate it directly, which is really what interviews are all about.

cholantesh 9y ago

>Whiteboard interviews have a very high false negative rate (rejecting people who are good), but they also have a low false positive rate (hiring people who are not good). That's a pretty bold assertion. Is there data to show this?

adrr 9y ago

Reasoning behind why whiteboard interviews suck is based that doesn't model real job skills. Maybe our org is an outlier but but we heavily rely on whiteboards to plan out architecture(ERDs,flow), problem solving, training/knowledge sharing. Our trello cards sometimes contains images of whiteboarded diagrams. Being able to explain and whiteboard your stuff is a very important skill at organization.

jay-anderson 9y ago
Sketching out ideas and preliminary diagrams is very different from programming on a whiteboard. If a whiteboard interview were talking at a higher level about the problems at hand then I don't think it'd be such a problem. However, that doesn't seem to be the case. In whiteboard interviews you're asked to write detailed code on a board.
rimliu 9y ago
How many images in your trello contain C code for reversing string?
- neeleshs 9y ago
  Yep, whiteboard programming - not good . Whiteboard interviews is too generic a name to use for that.
ng12 9y ago
I'm fairly convinced much of the hate comes from people who interview at firms who do WBI incorrectly. The question I give is tailored for the following points: You can pick up unfamiliar code/data structures easily You can understand a given problem space without too much explaining You can code the "easy stuff" without thinking about it too much You can distill and discuss the difficult parts of an algorithm All of which are traits I consider key for someone I'll be working with.

jaredcwhite 9y ago

WBIs will go away if the developer community rallies around not doing them. Companies can offer crazy incentives to work there like free massage, organic lunches, and game rooms, but they can't figure out a better way to hire than conducting WBIs? I call BS. They'll figure it out when they're pressured to do so...aka they can't find good talent anymore. Let's put on the pressure.

_dark_matter_ 9y ago
There is no benefit for people who are good at WBI's to join you in protest, and the bad people weren't going to pass them anyways, so the companies won't care either. On top of that, anyone giving a WBI got hired with a WBI, so the cycle continues. I am not saying WBIs are good or bad - just stating that this kind of rally won't have the desired effect.

krosaen 9y ago

In a similar vein, I view whiteboard interviews (and required prep) similar to professional exams, like the Bar exam for lawyers. While I think it's unreasonable to expect good professionals to be able to pass such exams without a lot of practice, it's perhaps not as unreasonable to expect good professionals to be able to pass such exams if they study. Nobody likes tests, but I suspect that a high percentage of the good programmers who belly ache about whiteboard interviewers could in fact get good at them if they put in the time. So yes, whiteboard interviews or not optimal and they are unfair, but if you are rejecting them out of hand and refusing to study for them, you are only taking away some opportunities. I say this as someone who hasn't had to do a whiteboard interview for 10+ years. The jury is out as to whether I'll take the pain to study for them again should I do a full job search among tech companies again. Also: note to author, I believe, "They have little co-relation" in your opening paragraph should be "They have little correlation".

lr4444lr 9y ago

I used to believe that I got rejected after many whiteboard interviews, not because I did poorly, but because I didn’t have the same (right) background as the interviewer. I served in the military, I was a Front End Engineer, I didn’t go to a fancy school, I used the wrong language, I’ve used the wrong framework etc. I think the author is hitting on some deep-seated psychological aversion people have to white board interviews. If you can identify that you were turned down based on your failure to do it well, you have to accept that it's your fault - not that you happened not to be good at that tech stack, or didn't have a "personality fit". (I know there are different kinds of white board interviews - I'm assuming the generic pseudo-code algos and data structures type in keeping with the article's generality.) Exposure to that kind of evidence of your inadequacy can be hard to accept. But it's also remediable. Get cracking and practice.

neeleshs 9y ago

I see a lot of naysayers for whiteboard, but pretty much no opinion on what they think is better than whiteboards.

hibikir 9y ago
Small project, on your own time, expected to take about 3 hours or so. Automated tests can do the very basics of evaluation. Then, 1 to 2 hours explaining the decisions made, and adding one more feature to the project. It lowers stress levels of the candidate, as they get to a lot of the thinking in advance. It's practical, as it requires more detailed programming skills. It provides a better sample of the candidate, as 5 20 lines of code snippets don't compare with a larger codebase, and makes the candidate explain something they really did themselves. Total time spent by the candidate is probably about the same, but the employers saves engineering time. I've been through that system a couple of times, it was both more enjoyable and gave people a far better picture of my skills than just how well I remember a tree traversal algorithm that I haven't had to write in forever, as data structures often handle their own traversal.
metaphorm 9y ago
need context to determine what would be better. giving the same interview for every position you're hiring for is not usually the right thing to do. Google is very unusual in their practice of hiring into a general pool. Most companies hire for more specific openings. In general though, when I interview it has three main sections, tailored for the role. First, is conversational. Talk about the resume, talk about work experience, talk about technology, etc. Second, is a practical example. Sometimes (but not always) this involves solving an algorithmic problem on a whiteboard. Sometimes it involves a guided tour of the codebase and functionality of a relevant project the candidate has worked on. Rarely it might be a take-home coding task. Third, is a more general example usually of some kind of system design problem. A lot of candidates like to write/draw on the whiteboard for this section but it's not strictly necessary nor do I ask them to do so. There's a board in the room if they want to but I don't ask them to use it.
falcolas 9y ago
Talk to them. Have a conversation about "how would you solve this (real) problem" - the same kind of conversation you have with a peer when they're stuck on a problem. If both you and they are professional developers, you will be able to get into a nice in-depth discussion of the pros and cons, and go into nice tangental discussions along the way. If they aren't a professional developer, it will show in no time at all; the longest I've let such a conversation go was 10 minutes just to see how long the interviewee would keep trying and snowball me. I've found this is simply much faster and more informative about their experience with subjects that matter in the daily work of a software dev.

cubano 9y ago

What I remember about much of that code I looked at in DS classes way back in the 80's was that, due to very limited memory space, all the variables were like I,J,K and X,Y,Z. You have no idea how much more difficult that made it to follow sorts and tree algorithms. Thankfully, I seemed to have some sort of coding gift that allowed me to write the damn things at the terminal and make them work. I got A's in all my CS classes, but now I guess I am so used to the magics of PHP object handling I've literally stopped caring about that shit. It's hard to see why I would want to again.

metaphorm 9y ago

I think one of the real problems that doesn't get discussed very much is Google-envy. A lot of companies that don't really have problems like Google, and don't really have organizations like Google, and don't really have applicant pools like Google, will yet interview the same way Google does. It really might be the correct way to interview for Google. Certainly they've put a lot of thought into how to manage their hiring process and they still lean heavily on the whiteboard. I'm almost positive, however, that most companies are really nothing like Google and really should not be using the same interviewing techniques. Smaller companies with more quotidian problems would be better served with a more conversational and less time-pressured interview style.

scrpn 9y ago
Google explains here their interview observations : https://www.google.ro/amp/s/www.wired.com/2015/04/hire-like-google/amp/ https://www.google.ro/amp/s/www.wired.com/2015/04/hire-like-...

nunez 9y ago

i 1000% agree with this. there is no good way to really know whether a candidate is good or not, but their ability to express their thoughts, in code, on a whiteboard is a really strong indicator that they can think through a problem and (more importantly) communicate a solution as a software engineer. the next best approach --- a take home assignment --- is also good at this (or better, IMO, since you're free to implement that solution however you damn well please and it more closely translates into the work they'll actually be doing), but people complain that this is too invasive of their time and that they are doing free work. which is it? I have to imagine that what most people want is to be asked about their experience, talk through it, and expect that to be enough. which is a very solid first pass filter. it's just not strong enough of a filter to extend an offer with confidence in many situations; it's too easy to bullshit and most of us engineers are damn expensive. (it's not just salary; we want dedicated offices and sit stand desks and super comfy chairs and free food and drinks and games and cadillac health insurance and 401ks and so on and so on) i also never understood why whiteboarding was hard. if you know how to code and know your language passably well, then why can't you write code on something that isn't a computer?

mbrodersen 9y ago

I have hired a lot of developers, working for different companies in different countries. And I have never ever seen a case where a developer did well in the WBI interview, was hired, and then turned out later to not be a good developer. However I have seen plenty of examples of developers (interviewed by somebody else) who didn't do a WBI interview and later was fired because they were not good developers. My point is that if you can't do a WBI, well then perhaps you are simply not as good a developer as you think you are. Developing is about thinking. And WBI style interviews force you to think for yourself without copy-pasting a solution from Google.

GhostVII 9y ago

I think whiteboard questions are great if you make them either pretty easy questions, or more complex questions that you just diagram and discuss. In the first case, you filter out poor coders very easily, and see how the better coders apporach the problem. In the second case, you learn about their thought process when approaching a problem where you can't just modify an algorithm you have already practiced and answer it.

samblr 9y ago

Im a 'competent' programmer who has done few whiteboard interviews in my time and I have failed in most of them. Here is why ? I consider myself a competent programmer since I have developed decent work@work and my peers recognise it too. But how would any interviewer can know about this ? My resume is like any other resume in tech world. They cannot. So I've to be put through WBI. For many years I couldn't have known what was wrong with me. Every time I failed I felt miserable to be honest. Eventually, I have narrowed down the reason. It is not just at interview I fail to problem solve - even when I try doing at home with a white board and 'explanation' I fail. English is not my region language but one would not doubt on my conversation abilities as I manage it ok. It is that I simply fail to explain solving programming puzzles in english since while programming I 'do' think in my regional language and most of the time in 'silent' mode in my head. Now 'silent' mode is again dreaded in interviews - as interviewers they want to know 'how you approach solving a problem'. I think a better solution to WBI would be to make interviewee do programming questions on a laptop (alone) where interview is held - say make it an hour worth of exercise. Have that program run through some time bound tests (like google code jam etc) and may be do a code walk through later with interviewer. This way interviewer is sure interviewee can code. Also, one of things that will definitely help would be to have some sort of evidence that you can code (git, code competitions etc). This gives some sort of confidence to interviewer.

tutufan 9y ago

If you select stringently for WBI ability, you'll get strong WBIers. However, you're also paying the opportunity cost of not selecting for other traits and skills that might matter a lot more. +1 though for "Senior Whiteboard Engineer"--that term is genius!