6 ms·

Why phone and whiteboard interviews are not the best way to evaluate candidates

64 points by tush726 12y ago

ArchD 12y ago
I give computer (coding) interviews instead of whiteboard interviews, and the candidates are allowed to freely use the compiler. This should be more natural than whiteboard interviews and closer to actual programming work. The candidate may tend to be less communicative than in a whiteboard interview, but I can still observe what they are typing and ask them questions.
- hessenwolf 12y ago
  I would really appreciate it if you could describe some of your experiences in this in more detail. How many approximately have you given? How is the success rate? How many do you turn away, and why? Have you been able to match up key successes and disappointments with particular signals in the interview, and what were they?
  - ArchD 12y ago
    There wasn't any A/B testing since when I think about whiteboard vs computer, I already have the notion that if you can write code on a whiteboard, you can write it more easily and naturally on a computer. I've given 5-10 such interviews.
hessenwolf 12y ago
And you know what? Maybe I'd like to work for Google or McKinsey. However, I suck at brainteasers. I've worked in research since 2002. I always solve the problem. I'm the one you go to, after the other teams ran into trouble (I work rather hard to position myself as that). I take the details of your issue, and offer to come back to you. Then I brainstorm and come back with a proposal. I will probably discuss the problem with several other specialists, one or two at a time. Ideas occur at a Poisson rate with time spent thinking. I find running helps to focus the time spent thinking. My most creative problem-solving ideas I've had while peeing. I think it is the interruption. These screening interviews will rule me out, with high probability. Fine.
- hueving 12y ago
  >I've worked in research since 2002. I always solve the problem. Those two sentences conflict heavily with each other. In research it is very rare to solve open problems. So either you are researching trivial things or your definition of "solving the problem" includes assuming it can't be solved.
- vacri 12y ago
  These screening interviews will rule me out, with high probability This sounds like a problem that you could put your mind to solving.
  - hessenwolf 12y ago
    The solution is like the blog author's (maybe it was a different blog I read this morning): practice interview questions. Study for the exam. Yeah, I can do that. Mmmm. And maybe I pass. But other companies won't make me do that... So, if the aim was screening for people that want to work for Google above all else, then, yes, the screening totally works. But... Google would downsize you and your project coldly if there were financial reason to do so, so...
    - throwawaykf05 12y ago
      Not a big fan of Google here, but while they may kill your project in cold blood, I highly doubt they'd downsize you with it. The other part you should consider is that Google and others of their whiteboarding ilk will likely pay you much more than other companies, except maybe finance. So if other factors don't prevent you, maybe it's worth jumping through the hoops for the pay upgrade.
k__ 12y ago
Best challenge I had at an interview was drawing UML diagrams. First I talked to the CEO who liked me and said: "Now your gonna be asked a few technical questions by our lead developer." The dev came in, we both hated a bit on Oracle and he told me to draw an UML diagram about a system of classes he will tell me. When he told me I didn't need to go into detail that much, the CEO looked impressed and I got the job. So strange...
Osiris 12y ago
The best interview I had was with my current employer. It was a half hour of discussion, some white boarding (just FizzBuzz), then an hour of "pair programming". What was great is that the three of us sat together, I wrote the code, but we asked each other questions and talked through it. It felt more like a real world scenario than a test. It was relaxing and enjoyable. I'm curious if anyone else has had the experience of a pair programming exercise for their interview?
- kasey_junk 12y ago
  I have done them, given them, and compared them to other interview techniques. I personally quite like them, but I like brain teasers, hostile phd style defenses and whiteboard "jam" sessions as well. My experience with them as a good tool in the hiring pipeline? Less compelling. Any work sample evaluation is better than techniques that don't involve work samples and pairing seems to pain less developers than on paper coding does. But unless you actively pair a lot during your day to day work, then paired programming is still fundamentally different than the kinds of work you are actually looking for and there are lots of developers who work poorly in paired environments.
rifung 12y ago
It definitely is strange that we still see these interviews when they don't do a particularly good job in terms of predicting success on the job. On the other hand, I'm on the opposite end of the spectrum from the author I suppose, I actually really enjoy interview questions. If anything I wish I would encounter those sort of questions day to day, but sadly real programming is much less demanding intellectually for most of us.
currywurst 12y ago
Adding anecdote: I know of a team that uses test-driven hiring, and they have found it to be a good indicator for subsequent performance. For javascript, you can probably set up a qunit jsfiddle (e.g. http://jsfiddle.net/IrisClasson/RMh78/ http://jsfiddle.net/IrisClasson/RMh78/)
malux85 12y ago
Whiteboard interviews can be good, but there's "One weird trick" to getting them right. Let the person code WITHOUT YOU BEING THERE. Interviews are stressful. The author says "My heart rate jumped" ... his heart rate jumped because his body released a bunch of adrenaline in reaction to a stressful event. Adrenaline prepares the body for fight or flight, and part of this preparation is re-routing blood to critial systems ... muscles, heart and lungs ... and things like digestion and higher level cortical function are not required for fight or flight. Programming requires higher level cortical thought - which is disabled during a stressful situation. Give the person a programming challenge, tell them to code it on a machine (or the whiteboard for the inferior option) and then leave the room and tell them you'll be back in 10 minutes. Let them relax, then their brain can think.
- abrugsch 12y ago
  This is pretty much analogous to urinal "stagefright" where your ability to pee (for those afflicted) goes right out the window as soon as another person steps up to an adjacent urinal. Likewise, my ability to do anything coherent on a keyboard drops to a "uhm what was I doing?" as soon as anyone starts hovering behind my shoulder. Doubly so for interviews. I wonder if there's a correlation between those susceptible to urinal (or stall perhaps...) stagefright and inability to code on show.
  - ryandvm 12y ago
    Amusing. I absolutely suffer from the urinal stagefright. I'm also terrible at whiteboard coding. Or playing piano in public. I recall from childhood the time my father invited me to play a song I knew by heart in front of one of his friends. I got a couple bars into it, flubbed a few keys and then the song just fell completely out of my memory. I had nothing. Pretty much any time I feel like I'm being evaluated by someone, my brain goes into an overly defensive, self-critical mode where I spend more cycles trying not to screw up than I do actually performing. It's generally a disaster. Strangely, I discovered my cure many years ago when I was able to crank out Mozart's Rondo Alla Turca in front of an entire wedding party. Under just the right level of inebriation, alcohol seems to dampen the self-critical part of my brain just before it starts affecting much else. Unfortunately, it takes quite a bit of alcohol for this effect to kick in. And sadly for me, this hack is a bit harder to pull off in a job interview situation without coming off like a high-functioning alcoholic. It certainly helps with the phone interviews though...
akanet 12y ago
I basically built an entire product to help make the phone-coding situation better. It's called CoderPad (https://coderpad.io https://coderpad.io) and it basically tries to mimic a more natural coding environment inside the browser, for the purposes of conducting interviews. For example, if you're used to Ruby, you'll get a full REPL with debug information and tab-assist, etc. It obviously can't do too much for the psychological side of the problem, but I believe that getting the interview environment as close as possible to your natural coding environment is definitely a needle-mover when it comes to candidate comfort during interviews.
- nsb1 12y ago
  We use a similar tool during phone screens: http://rextester.com/runcode http://rextester.com/runcode Doing so has dramatically improved the screening process for us. This site allows real-time multi-person interactive coding sessions in a variety of languages (note: If you try the "live cooperation" mode, you need to click the link yourself as well as give it to others. This wasn't obvious for some people. ) As for whether or not coding problems on the phone or the whiteboard are a good idea, in my opinion they are when applied properly. There is no way I am going to hire someone for a software development position without seeing them write some code, and there is no way I am hiring someone for an embedded C position without knowing that they understand how pointers work. There are not that many reliable ways to do this in a controlled environment. We know that everyone can use google to find answers - I'm more interested in where people are starting from. There is a balance though. I do not ask interviewees to solve brain teasers; that sort of problem only serves to increase stress in the interview. Most of the problems I ask are simple, such as printing out multiplication tables or simple string manipulation, or linked list type questions for embedded C. When doing these on a whiteboard, I am very clear up-front that trivial errors like missing punctuation or spelling do not matter. I am interested in seeing how the candidate approaches the problem logically, and it is surprisingly obvious what sort of candidate you are dealing with when seeing them attack a problem.
  - kasey_junk 12y ago
    > There is no way I am going to hire someone for a software development position without seeing them write some code How often do you watch your currently hired developers as they actually produce the code? How does the best of them handle you standing over their shoulder judging their typing technique? I'm unfairly giving you a hard time because you seem to have a sensible approach to hiring, but you should definitely know that watching people code, impacts the code they write. I've also found little evidence that people that are good at coding in front of someone, are necessarily better at the final product, than people that need some space to code.
LukeB_UK 12y ago
My current employer asks you to do an hour code test and submit it to them. The next stage is a phone interview to discuss what you did for the code test as well as a few questions around it. I feel this takes some of the pressure away but has the benefit of having an insight into how someone works technically.
- kasey_junk 12y ago
  This is largely what I always recommend. - Conversation about the company/position, not even a screen, usually more of a sales job, but done by someone who can answer actual real on the ground questions about the position and the hiring process. - Standard take home programming problem. - Code review session. That is all. My own personal bias has come down on the code review session isn't actually important either, but I get looked at like I have 3 heads when I suggest 0 interview style settings in a hiring pipeline.
wmkn 12y ago
Playing devil's advocate here. If there are two kinds of great programmers, those that can handle programming in a stressful situation and those that cannot. Given a choice I would want to hire the one that can handle the stress.
- spiralpolitik 12y ago
  This would hold if programming was done using a whiteboard. Luckily it is not.
- DanBC 12y ago
  Why? You have a duty to protect your employees from harm in the workplace and this includes protecting them from stress.
- kasey_junk 12y ago
  > If there are two kinds of great programmers There aren't. If there were, they wouldn't be split on the axis of "programming in a stressful situation". Even in development environments where front line stress can be a part of the job, its not full time, and the good developers work on 1) reducing the amount of time the organization spends in those modes and 2) removing human performance from the triage out of those modes. The reason for that is that under stress performance is not actually repeatable. Everyone breaks eventually and pushing the eventually back is not a cost effective way to mitigate the risks.
spiralpolitik 12y ago
The best coding interview I ever did was a pair programming exercise using a laptop with one of my prospective co-workers. We worked on a problem for 45 minutes, we conversed naturally about the problem, and there was very little stress. At the end both of us came away with an understand of the others skills and how we worked as part of team. I've since used a similar exercise when interviewing people as I've found I get a much better sense of what the candidate can do than placing them in from of a whiteboard.
stefanve 12y ago
I personally dislike whiteboard interviews as well. Sometime ago they ask me in an interview to code on the whiteboard in perfect Java code no pseudocode allowed. I'm not sure why you should want that.... I also done some challenges for different companies. Most of the time they are something like create X in N amount of time those I like they make so much more sense to me as you can see the result and talk about it especially when you can do it from your own home on your own computer.
- michaelt 12y ago
  no pseudocode allowed. I'm not sure why you should want that.... Some interview candidates, when invited to write pseudocode, will write pseudocode imprecise enough that you can't tell if they will make good programmers. For example, if a candidate is asked to write a bubble sort and they write: Take the input array, go through every element comparing it to the one after it and, if they're in the wrong order, swap them. Repeat until you get through without any swaps. Did I see the candidate think about off-by-one errors, like missing the last item or exceeding the array's bounds? Did the candidate think about common edge cases, like zero-item and one-item arrays? Did the candidate think about whether the function would work if the array contained two identical entries? Did the candidate check their loop will eventually terminate? Does the candidate know how to write a working computer program, rather than describing an algorithm in vague terms? With the pseudocode answer I can't tell if the candidate did those things, because the solution doesn't show me what I need to see. And yet, if you've asked them to use pseudocode, they've done exactly what you asked of them. That's why my preference is to say "use a real language and tell me the tests you'd perform on it, but I'm not fussed about semicolons and exact names of library functions." - it tends to get you a solution that lets you tell if the candidate knows how to program.
BSousa 12y ago
While I don't want to say if phone/whiteboard interviews are good or not, I can give you some pointers that worked great for me in the past. 1) Always, ALWAYS before writing a line of code say that you are trying to solve the problem first, by whatever means necessary and only then will you refactor. 2) Solve the problem by any means necessary. O(n*n)? Not freeing up resources? Allocating an array that is 100000 elements long just in case? No problem. Solve the problem. 3) Explain what you did/how you did it 4) Ask (if there is time) to refactor the code to be more idiomatic. Keep explaining that you wanted to solve the problem first, so later you can think of the right way to organize the code, and not the way around. I never had an interviewer complain about solving the problem in that order nor have cut me off (even when it ran out of time) when I suggested to refactor the code in the end. Also solving the problem first, then using idiomatic code takes a bit of the stress off since you don't have to go and write and rewrite each line everytime you see something wrong.
- matthewmacleod 12y ago
  That's very interesting—I'm much less interested in a candidate solving a problem in that situation, than I am in seeing how they approach it. That includes things like 'Hey, you made an unusable O(n!) algorithm, what gives?' That's always made clear though. I guess it goes to show that there's rarely one approach to this sort of thing.
  - BSousa 12y ago
    To be fully honest, that is usually how I solve problems anyway :) I start with the problem, whatever it maybe, then I 'solve' it by whatever means necessary. This lets me iterate very very quickly over various solutions if needed without much concern about pretty code, even if it is a simple api call, parse json, display something in the view. When I have that working, I refactor, sometimes I rewrite, I make the UI nice, I write tests, etc. In an interview, I believe you either want to see if the person can code/create and algorithm or you want to see his thought process on solving problems. If the latter, you don't need a whiteboard and a conversation with someone will work as well. If the first, I would do as I mentioned in the previous post. The O(n!) question for me wouldn't be important. I would probably reply "Yes, I know, let me just finish this part"... "Back to the O(n!), you are right/wrong, but seems to work this way, what we can do is change that for loop into a something/something reducing it to O(2n). And rewrite that part of the code. When I was interviewing people, after a quick phone screening, we would ask someone to come over, we had a quick chat with them (30 minutes maximum) and then had 2 quick exercises for them to do. 1) given a simple piece of code (200 lines or so) we would ask them to refactor it, based on some parameters like future extensibility, not being linked to some implementation, etc. Second one was a written spec, with 3-4 unit test written that would verify the spec, and we would ask for them to write the implementations (max. time 3h, good candidates finished it in 30-45minutes, bad ones couldn't finish it in time). With those two questions we were quite easily 'judge' the competence of the candidate. The ones that did very good would go home with job offers (or would have one when they got home), bad ones were rejected, and intermediate ones, depending on team feedback would be invited back to discuss the code so we could make up our minds. ps: I actually wrote the exercises and everyone already at the company did them as well and passed.
stressedcoder 12y ago
Went through two of these recently. Despite programming for more than a decade since university, I noticed the the unusual stress level and awkward situation (coding on google docs or a plain editor with no debugging ability) while the interviewer is umming and aahing (way to be judged) while you're thinking your way through a problem with stops and starts. I managed to solve all the problems given, but failed to get past the phone interview without the any feedback as to why. Thinking retrospectively, I'm almost never in that situation and pair programming with real tools and problems is not approximated at all by these types of phone or hangout screens. Neither are the time based coding tests like codility. That being said, the big software companies spend lots of money and time on hiring and swear by this process, so we're in a "like it or leave it" situation.
- army 12y ago
  I think the lack of highlighting and indentation support in some of those tools is a pain and adds unnecessary mental overhead. Whether a candidate can reason about the correctness of their code without using a debugger is a useful data point though. Both in a more abstract way and by manually tracing at least part of the program instead of stepping through it with a debugger. If you struggle to manually trace code in your favorite language, that raises some questions in my mind.
ukigumo 12y ago
I prefer that the initial interviews be about knowing what the candidate wants out of the job he's applying for, how it fits into his master plan etc. Then I want to try and figure out if I really want to spend a third of my waking life working with this person. Only towards the end of the process, and only if I'm not too sure about the candidate ability to do his job (eg: did not demonstrate interest in keeping himself up to date; was not able to articulate how X adds value over Y; etc) will I go into full-on technical interview. Sounds weird right? But the thing is, I'm not hiring rocket scientists so I don't need the best of the best, just someone who knows his stuff and isn't afraid of saying "I don't know (but I will find out)" when that's the case. I also take great care in planning work and defining deliverables so as to reduce the pressure on my colleagues' workday and there is hardly ever a situation where we need to rush to meet a deadline. If or when that does happen, I find that the culture of trust and a certain amount of friendship does wonders for the team's spirit.
deleted 12y ago
[deleted]
kasey_junk 12y ago
The author is absolutely right that interviews that involve someone to code over the phone, on paper or on a whiteboard bias against a certain part of the developer population. Unless it is something people actually need to do for the position, you would be wise to remove "paper" coding from your hiring pipeline. But, there is also a large part of the developer population that has the same reaction that the author did to "paper" coding, to any pair programming, including pairing with devs they've worked with for years. Developers in this population, who know that they are in it, generally will decline any job that actually involves large amounts of pair programming so you should reveal that you do lots of pairing early in your hiring pipeline. If you do not actually do pair programming a lot in practice, this technique is no better, and in some ways worse, than "paper" programming. My policy on any programming test is that it should be take home, non-timed and non-observed. Unless your environment has real, hard requirements about "how" someone writes code putting fake constraints on that part of the filtering process only biases against potentially great hires.
- dsacco 12y ago
  Devil's advocate here - how do you control against candidates plagiarizing code or cheating? Do you ask them about the code sample onsite in such a way as to determine if they understand it? I'm not sure where I stand on this, but I've heard many hiring managers complain that great phone and work sample candidates often fizz out entirely when tested on the sample in person - and not due to jitters, but due to e.g. not being able to comment their own code or explain one of their functions. I will say that I think a "homework" style coding test has always felt like a more fair assessment and been more enjoyable for me personally. I think a homework style work sample is probably one of the best methods for hiring, as long as it 1) won't take longer than an hour or so and 2) is technically exhaustive of typical job functions. That said, I know brain teasers are hated, and I dislike them too, but I really think most programmers should know standard algorithms and data structures. I don't think the current model of whiteboard coding is perfect, but knowledge of algorithm design is great for optimizing performant code.
  - kasey_junk 12y ago
    If you are in an environment that is large enough for your programming problems to be widely available, you are also large enough to build mechanical plagiarism detection systems (which are a pretty widely researched field oddly enough). As an aside, if you have that problem with submitted work, you also have it with whiteboard/paper problems. How do you distinguish between a "good" candidate and a "well coached" one in the whiteboard setting? I personally think if your hiring pipeline is generating lots of "cheaters" you need to address the candidate collection not the candidate evaluation. You are fishing in the wrong ponds, so to speak. I personally have no problem with candidates getting help on submitting their programming problem. I view effective help getting as a fundamental tool in a developers toolkit, so that in and of itself doesn't bother me. I think you can do an onsite code review, where the candidate presents their code sample, teaches you how to use it, explains the tradeoffs, responds to criticism, etc. This should be just like what you are doing for your "real" code reviews (if you don't do "real" code reviews, how do you know that any of your colleagues understand the code they write?) Over the years, I've grown a bias against even caring about that second step, because I have found so few people that do well on the programming problem part and then do poorly on the second part due to anything other than stage fright, but I understand I have an extreme position on that.
bsdpython 12y ago
Discussing almost anything highly technical over the phone is usually a disaster. When's the last time you solved a production problem without reference materials over the phone? Sounds hard right? Whiteboards are a crap shoot - we can all come up with lots of problems that are difficult to whip up without any reference materials but what's the point? Neither method reflects how anyone actually does their job so get rid of it. Every skill takes time to develop yet most people think they are good at giving interviews with zero experience, research, experimentation or methodology. Allow a typical junior employee to start interviewing candidates and it will go right to their head as they are given their first taste of power over others in a professional setting. It takes time to learn how to interview properly, figure out the specific attributes you want to screen for a particular role, remove your own personal biases and plain get over the fact that you have the power over someone else's career for probably no good reason. My philosophy on interviewing is that as an interviewer you want to avoid screening for anything that is not reflective of the actual job duties and to also avoid anything that the candidate is expecting to be asked. If you ask typical programming questions then you are likely screening for candidates that have the most time to prepare for interviews which screens out the lazy but also screens out the no-nonsense doesn't like to waste time types. Ask questions that get to the core of what interests them and figure out if they will be motivated enough to learn what they need to get the job done. Then get back to them quickly!
igammarays 12y ago
I think phone and whiteboard interviews are great for selecting the very smartest of the developer population, which is why companies like Google use it. Although they hardly reflect production environments, if someone is able to ace brain teasers in 45 min interviews, they must surely be smart enough to figure out pretty much anything technical, a priori. However I absolutely agree that phone/whiteboard interviews are not sufficient for determining a candidate's fit, because it doesn't say much about the person's work ethic, time management, and ability to manage large projects - which is a thing that many "born gifted" smart people struggle with, in my experience.
- ryandvm 12y ago
  Right. Google et al have never been concerned about false negatives when hiring. They have more than enough applicants to worry about whether a few good people get turned away.
- coldnebo 12y ago
  In my experience, I've seen some incredibly smart people (the kind who ace those brain teasers) make some incredibly bad technical gaffs. Like an architect who chose synchronous HTTP as a protocol for a service that later required the ability to cancel long running operations in progress (he unfortunately didn't realize why hitting the browser stop button wasn't the same as cancelling the process on the server). Or a senior dev who thought that a JSP scriptlet getting the current time would show the client's time, instead of the server's time. There is no guarantee that people who ace brain teasers will necessarily be able to figure out technical issues, although they are often excellent at sounding like they have figured out technical issues when they leave the details for the people who get the real work done.
  - igammarays 12y ago
    Those two examples sound like simple knowledge mistakes, not critical thinking failures. There is a difference between someone who is not too familiar with the HTTP protocol and someone who just doesn't have the mathematical mind for translating real world tasks into precise steps that a computer can understand.
munirusman 12y ago
Being a developer myself, I agree with most of the things. I learnt this in a different way though. When we started RemoteInterview.io, we started as real time coding interview tool [0]. Even Though, it's really helpful for interviewer to watch candidate code in real-time, we soon realized that a number of candidates don't like to code under stress. We then developed RemoteInterview.io Tests [1]. It's a screening platform based on coding tests where candidates solves the challenge in offline mode whenever they want. Some candidates still don't like to have time limits but I personally think it is important to have a deadline for every task because in real-world, no project can have infinite time. [0] https://www.remoteinterview.io/features-interview https://www.remoteinterview.io/features-interview [1] http://blog.remoteinterview.io/post/109864123971/screen-candidates-using-programming-tests http://blog.remoteinterview.io/post/109864123971/screen-cand...
- kasey_junk 12y ago
  What sort of time limits are you talking about? If its on the order of "we'd like to have this back in the next week or so, unless you have something else going on?" I'll buy it. If it is "you need to complete this problem in 2 hours from the time we on our server observe an arbitrary event", that would be a complete non-starter for me. A) those deadlines quite simply do not exist in the real world and B) it adds a whole level of stress to some candidates, much like the stage fright stress, that is unnecessary.
  - asadlionpk 12y ago
    I think it completely depends on the task given. If it's a quick question or some MCQs, I think giving X hours is fine. If it's something like "design this project/prototype" then a week should work.
alphanumeric0 12y ago
I recently had a similar waterboarding, err, whiteboarding experience. My first solution to the first part of the problem was laughably complex (but still correct). I soon arrived at a much simpler version the employer was likely looking for a few moments after. Immediately the interviewer should've known I was affected negatively by being put on the spot. I am an introverted person after all, and I don't program on a giant whiteboard with someone constantly looking over my shoulder. So if my job is to work with someone new every day on a programming problem, someone I am not entirely comfortable around then I guess I'm off the list? The second part of the problem was more complex, and I eventually worked through it, with some leading questions by the interviewer. I did trip up once or twice. Oops, that one part of the program was O(n/2). I did not arrive at that one correctly. I demonstrated I'm comfortable with multiple languages, can reason intelligently about a program and come up with some solutions, iterating on them while demonstrating understanding of big-O notation, but the employer cares about none of these things even if they tell you beforehand that they do. That's pretty much a lie. The employer is simply lazy and has decided the best metric for weeding out candidates are toy problems on a whiteboard.
PublicEnemy111 12y ago
I used to share the same sentiment about whiteboard interviews and the cs "fundamentals"(read: brainteasers) interviews. 2 years ago, I failed spectacularly in an interview for a Pinterest internship -- I couldn't answer a single technical question. Fast forward to this year and and I've breezed through every interview with Google and other companies. I spent months preparing for the interviews, but the preparation was one of the most rewarding things I could have done as a developer at my stage. I feel far more competent as a developer with a solid grasp of algorithms and data structures. I enjoyed the preparation so much, that I started reading white papers on the more cutting-edge cs theory :)
- ihireppl 12y ago
  I have hiring responsibility for a startup. Honest question...Over the period of time when you were preparing for interviews how much meaningful code did you ship? More? Less? Algorithms and data structures are dandy but I would have a hard time hiring somebody with this knowledge but no history of shipping production code.
  - PublicEnemy111 12y ago
    Well, I'm still an undergraduate. I was pretty much coding 24/7, but it was hackerrank, leetcode, codechef problems. My side projects slowed down tremendously during this time; however, once I started them back up, I was working at a much faster pace, producing higher quality code, and using design patterns/algorithms/data structures that scaled. For instance, there was a problem on my senior project that had me stumped for weeks before I stopped the project to focus on studying for interviews. I immediately knew how to solve the problem once I resumed the project. On top of my personal projects, I've had great internships, which treated me like a full-time engineer. I shipped production code regularly during those.
steverb 12y ago
I don't do code interviews over the phone, but I always do a phone interview. It's just a smell test to see if the person has obviously over-stated their experience, and make sure they have some exposure to the technical things we care about. Once they pass that gate, then we do an interview with our dev team. That way everyone gets a chance to talk to/ hear the candidate and get a feel for if they want to work with them. As part of that interview we do a group coding exercise, usually using Pex4Fun (http://pex4fun.com/default.aspx?language=CSharp&sample=ChallengeFibonacci2 http://pex4fun.com/default.aspx?language=CSharp&sample=Chall...). Basically, we give the interviewee time to start working on it, and if they get stuck, then start white-boarding the problem as a group and solve it together. Again, we want to see how the person works and whether we think they will fit well on the team. After the interview is over, we take a vote and if the result is not a unanimous yes, then we pass. It's tough, but I've had about 80% of interviewees tell me that it's the most fun they've ever had in an interview. So far, we've been very happy with the results.
- baggachipz 12y ago
  > I've had about 80% of interviewees tell me that it's the most fun they've ever had in an interview. Did you ask them this during/immediately following the interview? That may... skew your results. Then again, if they're offering that insight without being prompted, that's a very good sign. I think the execution of this strategy, and the general tone of the people involved is very important.
  - steverb 12y ago
    Yeah, it has always been unprompted. We all try and treat as a fun exercise, occasionally with beer although I've never had an interviewee accept the offer of any drink other than water. I also go out of my way to make sure I communicate with the candidate. We take the voice vote immediately after the interview, and I call the candidate within 24 hours. If there are any no votes I make sure to tell the candidate why, and try and give them pointers for specific areas they would need to improve on to work here and try and point them at resources they can use to improve. We're in a relatively small market, and it pays to be nice to everyone. Anyone we interview I am pretty much guaranteed to run into at some sort of dev event. And just because someone wasn't a good fit today, it doesn't mean they won't be the perfect candidate next year or two years from now.

64 points by tush726 12y ago

ArchD 12y ago

I give computer (coding) interviews instead of whiteboard interviews, and the candidates are allowed to freely use the compiler. This should be more natural than whiteboard interviews and closer to actual programming work. The candidate may tend to be less communicative than in a whiteboard interview, but I can still observe what they are typing and ask them questions.

hessenwolf 12y ago
I would really appreciate it if you could describe some of your experiences in this in more detail. How many approximately have you given? How is the success rate? How many do you turn away, and why? Have you been able to match up key successes and disappointments with particular signals in the interview, and what were they?
- ArchD 12y ago
  There wasn't any A/B testing since when I think about whiteboard vs computer, I already have the notion that if you can write code on a whiteboard, you can write it more easily and naturally on a computer. I've given 5-10 such interviews.

hessenwolf 12y ago

And you know what? Maybe I'd like to work for Google or McKinsey. However, I suck at brainteasers. I've worked in research since 2002. I always solve the problem. I'm the one you go to, after the other teams ran into trouble (I work rather hard to position myself as that). I take the details of your issue, and offer to come back to you. Then I brainstorm and come back with a proposal. I will probably discuss the problem with several other specialists, one or two at a time. Ideas occur at a Poisson rate with time spent thinking. I find running helps to focus the time spent thinking. My most creative problem-solving ideas I've had while peeing. I think it is the interruption. These screening interviews will rule me out, with high probability. Fine.

hueving 12y ago
>I've worked in research since 2002. I always solve the problem. Those two sentences conflict heavily with each other. In research it is very rare to solve open problems. So either you are researching trivial things or your definition of "solving the problem" includes assuming it can't be solved.
vacri 12y ago
These screening interviews will rule me out, with high probability This sounds like a problem that you could put your mind to solving.
- hessenwolf 12y ago
  The solution is like the blog author's (maybe it was a different blog I read this morning): practice interview questions. Study for the exam. Yeah, I can do that. Mmmm. And maybe I pass. But other companies won't make me do that... So, if the aim was screening for people that want to work for Google above all else, then, yes, the screening totally works. But... Google would downsize you and your project coldly if there were financial reason to do so, so...
  - throwawaykf05 12y ago
    Not a big fan of Google here, but while they may kill your project in cold blood, I highly doubt they'd downsize you with it. The other part you should consider is that Google and others of their whiteboarding ilk will likely pay you much more than other companies, except maybe finance. So if other factors don't prevent you, maybe it's worth jumping through the hoops for the pay upgrade.

k__ 12y ago

Best challenge I had at an interview was drawing UML diagrams. First I talked to the CEO who liked me and said: "Now your gonna be asked a few technical questions by our lead developer." The dev came in, we both hated a bit on Oracle and he told me to draw an UML diagram about a system of classes he will tell me. When he told me I didn't need to go into detail that much, the CEO looked impressed and I got the job. So strange...

Osiris 12y ago

The best interview I had was with my current employer. It was a half hour of discussion, some white boarding (just FizzBuzz), then an hour of "pair programming". What was great is that the three of us sat together, I wrote the code, but we asked each other questions and talked through it. It felt more like a real world scenario than a test. It was relaxing and enjoyable. I'm curious if anyone else has had the experience of a pair programming exercise for their interview?

kasey_junk 12y ago
I have done them, given them, and compared them to other interview techniques. I personally quite like them, but I like brain teasers, hostile phd style defenses and whiteboard "jam" sessions as well. My experience with them as a good tool in the hiring pipeline? Less compelling. Any work sample evaluation is better than techniques that don't involve work samples and pairing seems to pain less developers than on paper coding does. But unless you actively pair a lot during your day to day work, then paired programming is still fundamentally different than the kinds of work you are actually looking for and there are lots of developers who work poorly in paired environments.

rifung 12y ago

It definitely is strange that we still see these interviews when they don't do a particularly good job in terms of predicting success on the job. On the other hand, I'm on the opposite end of the spectrum from the author I suppose, I actually really enjoy interview questions. If anything I wish I would encounter those sort of questions day to day, but sadly real programming is much less demanding intellectually for most of us.

currywurst 12y ago

Adding anecdote: I know of a team that uses test-driven hiring, and they have found it to be a good indicator for subsequent performance. For javascript, you can probably set up a qunit jsfiddle (e.g. http://jsfiddle.net/IrisClasson/RMh78/ http://jsfiddle.net/IrisClasson/RMh78/)

malux85 12y ago

Whiteboard interviews can be good, but there's "One weird trick" to getting them right. Let the person code WITHOUT YOU BEING THERE. Interviews are stressful. The author says "My heart rate jumped" ... his heart rate jumped because his body released a bunch of adrenaline in reaction to a stressful event. Adrenaline prepares the body for fight or flight, and part of this preparation is re-routing blood to critial systems ... muscles, heart and lungs ... and things like digestion and higher level cortical function are not required for fight or flight. Programming requires higher level cortical thought - which is disabled during a stressful situation. Give the person a programming challenge, tell them to code it on a machine (or the whiteboard for the inferior option) and then leave the room and tell them you'll be back in 10 minutes. Let them relax, then their brain can think.

abrugsch 12y ago
This is pretty much analogous to urinal "stagefright" where your ability to pee (for those afflicted) goes right out the window as soon as another person steps up to an adjacent urinal. Likewise, my ability to do anything coherent on a keyboard drops to a "uhm what was I doing?" as soon as anyone starts hovering behind my shoulder. Doubly so for interviews. I wonder if there's a correlation between those susceptible to urinal (or stall perhaps...) stagefright and inability to code on show.
- ryandvm 12y ago
  Amusing. I absolutely suffer from the urinal stagefright. I'm also terrible at whiteboard coding. Or playing piano in public. I recall from childhood the time my father invited me to play a song I knew by heart in front of one of his friends. I got a couple bars into it, flubbed a few keys and then the song just fell completely out of my memory. I had nothing. Pretty much any time I feel like I'm being evaluated by someone, my brain goes into an overly defensive, self-critical mode where I spend more cycles trying not to screw up than I do actually performing. It's generally a disaster. Strangely, I discovered my cure many years ago when I was able to crank out Mozart's Rondo Alla Turca in front of an entire wedding party. Under just the right level of inebriation, alcohol seems to dampen the self-critical part of my brain just before it starts affecting much else. Unfortunately, it takes quite a bit of alcohol for this effect to kick in. And sadly for me, this hack is a bit harder to pull off in a job interview situation without coming off like a high-functioning alcoholic. It certainly helps with the phone interviews though...

akanet 12y ago

I basically built an entire product to help make the phone-coding situation better. It's called CoderPad (https://coderpad.io https://coderpad.io) and it basically tries to mimic a more natural coding environment inside the browser, for the purposes of conducting interviews. For example, if you're used to Ruby, you'll get a full REPL with debug information and tab-assist, etc. It obviously can't do too much for the psychological side of the problem, but I believe that getting the interview environment as close as possible to your natural coding environment is definitely a needle-mover when it comes to candidate comfort during interviews.

nsb1 12y ago
We use a similar tool during phone screens: http://rextester.com/runcode http://rextester.com/runcode Doing so has dramatically improved the screening process for us. This site allows real-time multi-person interactive coding sessions in a variety of languages (note: If you try the "live cooperation" mode, you need to click the link yourself as well as give it to others. This wasn't obvious for some people. ) As for whether or not coding problems on the phone or the whiteboard are a good idea, in my opinion they are when applied properly. There is no way I am going to hire someone for a software development position without seeing them write some code, and there is no way I am hiring someone for an embedded C position without knowing that they understand how pointers work. There are not that many reliable ways to do this in a controlled environment. We know that everyone can use google to find answers - I'm more interested in where people are starting from. There is a balance though. I do not ask interviewees to solve brain teasers; that sort of problem only serves to increase stress in the interview. Most of the problems I ask are simple, such as printing out multiplication tables or simple string manipulation, or linked list type questions for embedded C. When doing these on a whiteboard, I am very clear up-front that trivial errors like missing punctuation or spelling do not matter. I am interested in seeing how the candidate approaches the problem logically, and it is surprisingly obvious what sort of candidate you are dealing with when seeing them attack a problem.
- kasey_junk 12y ago
  > There is no way I am going to hire someone for a software development position without seeing them write some code How often do you watch your currently hired developers as they actually produce the code? How does the best of them handle you standing over their shoulder judging their typing technique? I'm unfairly giving you a hard time because you seem to have a sensible approach to hiring, but you should definitely know that watching people code, impacts the code they write. I've also found little evidence that people that are good at coding in front of someone, are necessarily better at the final product, than people that need some space to code.

LukeB_UK 12y ago

My current employer asks you to do an hour code test and submit it to them. The next stage is a phone interview to discuss what you did for the code test as well as a few questions around it. I feel this takes some of the pressure away but has the benefit of having an insight into how someone works technically.

kasey_junk 12y ago
This is largely what I always recommend. - Conversation about the company/position, not even a screen, usually more of a sales job, but done by someone who can answer actual real on the ground questions about the position and the hiring process. - Standard take home programming problem. - Code review session. That is all. My own personal bias has come down on the code review session isn't actually important either, but I get looked at like I have 3 heads when I suggest 0 interview style settings in a hiring pipeline.

wmkn 12y ago

Playing devil's advocate here. If there are two kinds of great programmers, those that can handle programming in a stressful situation and those that cannot. Given a choice I would want to hire the one that can handle the stress.

spiralpolitik 12y ago
This would hold if programming was done using a whiteboard. Luckily it is not.
DanBC 12y ago
Why? You have a duty to protect your employees from harm in the workplace and this includes protecting them from stress.
kasey_junk 12y ago
> If there are two kinds of great programmers There aren't. If there were, they wouldn't be split on the axis of "programming in a stressful situation". Even in development environments where front line stress can be a part of the job, its not full time, and the good developers work on 1) reducing the amount of time the organization spends in those modes and 2) removing human performance from the triage out of those modes. The reason for that is that under stress performance is not actually repeatable. Everyone breaks eventually and pushing the eventually back is not a cost effective way to mitigate the risks.

spiralpolitik 12y ago

The best coding interview I ever did was a pair programming exercise using a laptop with one of my prospective co-workers. We worked on a problem for 45 minutes, we conversed naturally about the problem, and there was very little stress. At the end both of us came away with an understand of the others skills and how we worked as part of team. I've since used a similar exercise when interviewing people as I've found I get a much better sense of what the candidate can do than placing them in from of a whiteboard.

stefanve 12y ago

I personally dislike whiteboard interviews as well. Sometime ago they ask me in an interview to code on the whiteboard in perfect Java code no pseudocode allowed. I'm not sure why you should want that.... I also done some challenges for different companies. Most of the time they are something like create X in N amount of time those I like they make so much more sense to me as you can see the result and talk about it especially when you can do it from your own home on your own computer.

michaelt 12y ago
no pseudocode allowed. I'm not sure why you should want that.... Some interview candidates, when invited to write pseudocode, will write pseudocode imprecise enough that you can't tell if they will make good programmers. For example, if a candidate is asked to write a bubble sort and they write: Take the input array, go through every element comparing it to the one after it and, if they're in the wrong order, swap them. Repeat until you get through without any swaps. Did I see the candidate think about off-by-one errors, like missing the last item or exceeding the array's bounds? Did the candidate think about common edge cases, like zero-item and one-item arrays? Did the candidate think about whether the function would work if the array contained two identical entries? Did the candidate check their loop will eventually terminate? Does the candidate know how to write a working computer program, rather than describing an algorithm in vague terms? With the pseudocode answer I can't tell if the candidate did those things, because the solution doesn't show me what I need to see. And yet, if you've asked them to use pseudocode, they've done exactly what you asked of them. That's why my preference is to say "use a real language and tell me the tests you'd perform on it, but I'm not fussed about semicolons and exact names of library functions." - it tends to get you a solution that lets you tell if the candidate knows how to program.

BSousa 12y ago

While I don't want to say if phone/whiteboard interviews are good or not, I can give you some pointers that worked great for me in the past. 1) Always, ALWAYS before writing a line of code say that you are trying to solve the problem first, by whatever means necessary and only then will you refactor. 2) Solve the problem by any means necessary. O(n*n)? Not freeing up resources? Allocating an array that is 100000 elements long just in case? No problem. Solve the problem. 3) Explain what you did/how you did it 4) Ask (if there is time) to refactor the code to be more idiomatic. Keep explaining that you wanted to solve the problem first, so later you can think of the right way to organize the code, and not the way around. I never had an interviewer complain about solving the problem in that order nor have cut me off (even when it ran out of time) when I suggested to refactor the code in the end. Also solving the problem first, then using idiomatic code takes a bit of the stress off since you don't have to go and write and rewrite each line everytime you see something wrong.

matthewmacleod 12y ago
That's very interesting—I'm much less interested in a candidate solving a problem in that situation, than I am in seeing how they approach it. That includes things like 'Hey, you made an unusable O(n!) algorithm, what gives?' That's always made clear though. I guess it goes to show that there's rarely one approach to this sort of thing.
- BSousa 12y ago
  To be fully honest, that is usually how I solve problems anyway :) I start with the problem, whatever it maybe, then I 'solve' it by whatever means necessary. This lets me iterate very very quickly over various solutions if needed without much concern about pretty code, even if it is a simple api call, parse json, display something in the view. When I have that working, I refactor, sometimes I rewrite, I make the UI nice, I write tests, etc. In an interview, I believe you either want to see if the person can code/create and algorithm or you want to see his thought process on solving problems. If the latter, you don't need a whiteboard and a conversation with someone will work as well. If the first, I would do as I mentioned in the previous post. The O(n!) question for me wouldn't be important. I would probably reply "Yes, I know, let me just finish this part"... "Back to the O(n!), you are right/wrong, but seems to work this way, what we can do is change that for loop into a something/something reducing it to O(2n). And rewrite that part of the code. When I was interviewing people, after a quick phone screening, we would ask someone to come over, we had a quick chat with them (30 minutes maximum) and then had 2 quick exercises for them to do. 1) given a simple piece of code (200 lines or so) we would ask them to refactor it, based on some parameters like future extensibility, not being linked to some implementation, etc. Second one was a written spec, with 3-4 unit test written that would verify the spec, and we would ask for them to write the implementations (max. time 3h, good candidates finished it in 30-45minutes, bad ones couldn't finish it in time). With those two questions we were quite easily 'judge' the competence of the candidate. The ones that did very good would go home with job offers (or would have one when they got home), bad ones were rejected, and intermediate ones, depending on team feedback would be invited back to discuss the code so we could make up our minds. ps: I actually wrote the exercises and everyone already at the company did them as well and passed.

stressedcoder 12y ago

Went through two of these recently. Despite programming for more than a decade since university, I noticed the the unusual stress level and awkward situation (coding on google docs or a plain editor with no debugging ability) while the interviewer is umming and aahing (way to be judged) while you're thinking your way through a problem with stops and starts. I managed to solve all the problems given, but failed to get past the phone interview without the any feedback as to why. Thinking retrospectively, I'm almost never in that situation and pair programming with real tools and problems is not approximated at all by these types of phone or hangout screens. Neither are the time based coding tests like codility. That being said, the big software companies spend lots of money and time on hiring and swear by this process, so we're in a "like it or leave it" situation.

army 12y ago
I think the lack of highlighting and indentation support in some of those tools is a pain and adds unnecessary mental overhead. Whether a candidate can reason about the correctness of their code without using a debugger is a useful data point though. Both in a more abstract way and by manually tracing at least part of the program instead of stepping through it with a debugger. If you struggle to manually trace code in your favorite language, that raises some questions in my mind.

ukigumo 12y ago

I prefer that the initial interviews be about knowing what the candidate wants out of the job he's applying for, how it fits into his master plan etc. Then I want to try and figure out if I really want to spend a third of my waking life working with this person. Only towards the end of the process, and only if I'm not too sure about the candidate ability to do his job (eg: did not demonstrate interest in keeping himself up to date; was not able to articulate how X adds value over Y; etc) will I go into full-on technical interview. Sounds weird right? But the thing is, I'm not hiring rocket scientists so I don't need the best of the best, just someone who knows his stuff and isn't afraid of saying "I don't know (but I will find out)" when that's the case. I also take great care in planning work and defining deliverables so as to reduce the pressure on my colleagues' workday and there is hardly ever a situation where we need to rush to meet a deadline. If or when that does happen, I find that the culture of trust and a certain amount of friendship does wonders for the team's spirit.

deleted 12y ago

[deleted]

kasey_junk 12y ago

The author is absolutely right that interviews that involve someone to code over the phone, on paper or on a whiteboard bias against a certain part of the developer population. Unless it is something people actually need to do for the position, you would be wise to remove "paper" coding from your hiring pipeline. But, there is also a large part of the developer population that has the same reaction that the author did to "paper" coding, to any pair programming, including pairing with devs they've worked with for years. Developers in this population, who know that they are in it, generally will decline any job that actually involves large amounts of pair programming so you should reveal that you do lots of pairing early in your hiring pipeline. If you do not actually do pair programming a lot in practice, this technique is no better, and in some ways worse, than "paper" programming. My policy on any programming test is that it should be take home, non-timed and non-observed. Unless your environment has real, hard requirements about "how" someone writes code putting fake constraints on that part of the filtering process only biases against potentially great hires.

dsacco 12y ago
Devil's advocate here - how do you control against candidates plagiarizing code or cheating? Do you ask them about the code sample onsite in such a way as to determine if they understand it? I'm not sure where I stand on this, but I've heard many hiring managers complain that great phone and work sample candidates often fizz out entirely when tested on the sample in person - and not due to jitters, but due to e.g. not being able to comment their own code or explain one of their functions. I will say that I think a "homework" style coding test has always felt like a more fair assessment and been more enjoyable for me personally. I think a homework style work sample is probably one of the best methods for hiring, as long as it 1) won't take longer than an hour or so and 2) is technically exhaustive of typical job functions. That said, I know brain teasers are hated, and I dislike them too, but I really think most programmers should know standard algorithms and data structures. I don't think the current model of whiteboard coding is perfect, but knowledge of algorithm design is great for optimizing performant code.
- kasey_junk 12y ago
  If you are in an environment that is large enough for your programming problems to be widely available, you are also large enough to build mechanical plagiarism detection systems (which are a pretty widely researched field oddly enough). As an aside, if you have that problem with submitted work, you also have it with whiteboard/paper problems. How do you distinguish between a "good" candidate and a "well coached" one in the whiteboard setting? I personally think if your hiring pipeline is generating lots of "cheaters" you need to address the candidate collection not the candidate evaluation. You are fishing in the wrong ponds, so to speak. I personally have no problem with candidates getting help on submitting their programming problem. I view effective help getting as a fundamental tool in a developers toolkit, so that in and of itself doesn't bother me. I think you can do an onsite code review, where the candidate presents their code sample, teaches you how to use it, explains the tradeoffs, responds to criticism, etc. This should be just like what you are doing for your "real" code reviews (if you don't do "real" code reviews, how do you know that any of your colleagues understand the code they write?) Over the years, I've grown a bias against even caring about that second step, because I have found so few people that do well on the programming problem part and then do poorly on the second part due to anything other than stage fright, but I understand I have an extreme position on that.

bsdpython 12y ago

Discussing almost anything highly technical over the phone is usually a disaster. When's the last time you solved a production problem without reference materials over the phone? Sounds hard right? Whiteboards are a crap shoot - we can all come up with lots of problems that are difficult to whip up without any reference materials but what's the point? Neither method reflects how anyone actually does their job so get rid of it. Every skill takes time to develop yet most people think they are good at giving interviews with zero experience, research, experimentation or methodology. Allow a typical junior employee to start interviewing candidates and it will go right to their head as they are given their first taste of power over others in a professional setting. It takes time to learn how to interview properly, figure out the specific attributes you want to screen for a particular role, remove your own personal biases and plain get over the fact that you have the power over someone else's career for probably no good reason. My philosophy on interviewing is that as an interviewer you want to avoid screening for anything that is not reflective of the actual job duties and to also avoid anything that the candidate is expecting to be asked. If you ask typical programming questions then you are likely screening for candidates that have the most time to prepare for interviews which screens out the lazy but also screens out the no-nonsense doesn't like to waste time types. Ask questions that get to the core of what interests them and figure out if they will be motivated enough to learn what they need to get the job done. Then get back to them quickly!

igammarays 12y ago

I think phone and whiteboard interviews are great for selecting the very smartest of the developer population, which is why companies like Google use it. Although they hardly reflect production environments, if someone is able to ace brain teasers in 45 min interviews, they must surely be smart enough to figure out pretty much anything technical, a priori. However I absolutely agree that phone/whiteboard interviews are not sufficient for determining a candidate's fit, because it doesn't say much about the person's work ethic, time management, and ability to manage large projects - which is a thing that many "born gifted" smart people struggle with, in my experience.

ryandvm 12y ago
Right. Google et al have never been concerned about false negatives when hiring. They have more than enough applicants to worry about whether a few good people get turned away.
coldnebo 12y ago
In my experience, I've seen some incredibly smart people (the kind who ace those brain teasers) make some incredibly bad technical gaffs. Like an architect who chose synchronous HTTP as a protocol for a service that later required the ability to cancel long running operations in progress (he unfortunately didn't realize why hitting the browser stop button wasn't the same as cancelling the process on the server). Or a senior dev who thought that a JSP scriptlet getting the current time would show the client's time, instead of the server's time. There is no guarantee that people who ace brain teasers will necessarily be able to figure out technical issues, although they are often excellent at sounding like they have figured out technical issues when they leave the details for the people who get the real work done.
- igammarays 12y ago
  Those two examples sound like simple knowledge mistakes, not critical thinking failures. There is a difference between someone who is not too familiar with the HTTP protocol and someone who just doesn't have the mathematical mind for translating real world tasks into precise steps that a computer can understand.

munirusman 12y ago

Being a developer myself, I agree with most of the things. I learnt this in a different way though. When we started RemoteInterview.io, we started as real time coding interview tool [0]. Even Though, it's really helpful for interviewer to watch candidate code in real-time, we soon realized that a number of candidates don't like to code under stress. We then developed RemoteInterview.io Tests [1]. It's a screening platform based on coding tests where candidates solves the challenge in offline mode whenever they want. Some candidates still don't like to have time limits but I personally think it is important to have a deadline for every task because in real-world, no project can have infinite time. [0] https://www.remoteinterview.io/features-interview https://www.remoteinterview.io/features-interview [1] http://blog.remoteinterview.io/post/109864123971/screen-candidates-using-programming-tests http://blog.remoteinterview.io/post/109864123971/screen-cand...

kasey_junk 12y ago
What sort of time limits are you talking about? If its on the order of "we'd like to have this back in the next week or so, unless you have something else going on?" I'll buy it. If it is "you need to complete this problem in 2 hours from the time we on our server observe an arbitrary event", that would be a complete non-starter for me. A) those deadlines quite simply do not exist in the real world and B) it adds a whole level of stress to some candidates, much like the stage fright stress, that is unnecessary.
- asadlionpk 12y ago
  I think it completely depends on the task given. If it's a quick question or some MCQs, I think giving X hours is fine. If it's something like "design this project/prototype" then a week should work.

alphanumeric0 12y ago

I recently had a similar waterboarding, err, whiteboarding experience. My first solution to the first part of the problem was laughably complex (but still correct). I soon arrived at a much simpler version the employer was likely looking for a few moments after. Immediately the interviewer should've known I was affected negatively by being put on the spot. I am an introverted person after all, and I don't program on a giant whiteboard with someone constantly looking over my shoulder. So if my job is to work with someone new every day on a programming problem, someone I am not entirely comfortable around then I guess I'm off the list? The second part of the problem was more complex, and I eventually worked through it, with some leading questions by the interviewer. I did trip up once or twice. Oops, that one part of the program was O(n/2). I did not arrive at that one correctly. I demonstrated I'm comfortable with multiple languages, can reason intelligently about a program and come up with some solutions, iterating on them while demonstrating understanding of big-O notation, but the employer cares about none of these things even if they tell you beforehand that they do. That's pretty much a lie. The employer is simply lazy and has decided the best metric for weeding out candidates are toy problems on a whiteboard.

PublicEnemy111 12y ago

I used to share the same sentiment about whiteboard interviews and the cs "fundamentals"(read: brainteasers) interviews. 2 years ago, I failed spectacularly in an interview for a Pinterest internship -- I couldn't answer a single technical question. Fast forward to this year and and I've breezed through every interview with Google and other companies. I spent months preparing for the interviews, but the preparation was one of the most rewarding things I could have done as a developer at my stage. I feel far more competent as a developer with a solid grasp of algorithms and data structures. I enjoyed the preparation so much, that I started reading white papers on the more cutting-edge cs theory :)

ihireppl 12y ago
I have hiring responsibility for a startup. Honest question...Over the period of time when you were preparing for interviews how much meaningful code did you ship? More? Less? Algorithms and data structures are dandy but I would have a hard time hiring somebody with this knowledge but no history of shipping production code.
- PublicEnemy111 12y ago
  Well, I'm still an undergraduate. I was pretty much coding 24/7, but it was hackerrank, leetcode, codechef problems. My side projects slowed down tremendously during this time; however, once I started them back up, I was working at a much faster pace, producing higher quality code, and using design patterns/algorithms/data structures that scaled. For instance, there was a problem on my senior project that had me stumped for weeks before I stopped the project to focus on studying for interviews. I immediately knew how to solve the problem once I resumed the project. On top of my personal projects, I've had great internships, which treated me like a full-time engineer. I shipped production code regularly during those.

steverb 12y ago

I don't do code interviews over the phone, but I always do a phone interview. It's just a smell test to see if the person has obviously over-stated their experience, and make sure they have some exposure to the technical things we care about. Once they pass that gate, then we do an interview with our dev team. That way everyone gets a chance to talk to/ hear the candidate and get a feel for if they want to work with them. As part of that interview we do a group coding exercise, usually using Pex4Fun (http://pex4fun.com/default.aspx?language=CSharp&sample=ChallengeFibonacci2 http://pex4fun.com/default.aspx?language=CSharp&sample=Chall...). Basically, we give the interviewee time to start working on it, and if they get stuck, then start white-boarding the problem as a group and solve it together. Again, we want to see how the person works and whether we think they will fit well on the team. After the interview is over, we take a vote and if the result is not a unanimous yes, then we pass. It's tough, but I've had about 80% of interviewees tell me that it's the most fun they've ever had in an interview. So far, we've been very happy with the results.

baggachipz 12y ago
> I've had about 80% of interviewees tell me that it's the most fun they've ever had in an interview. Did you ask them this during/immediately following the interview? That may... skew your results. Then again, if they're offering that insight without being prompted, that's a very good sign. I think the execution of this strategy, and the general tone of the people involved is very important.
- steverb 12y ago
  Yeah, it has always been unprompted. We all try and treat as a fun exercise, occasionally with beer although I've never had an interviewee accept the offer of any drink other than water. I also go out of my way to make sure I communicate with the candidate. We take the voice vote immediately after the interview, and I call the candidate within 24 hours. If there are any no votes I make sure to tell the candidate why, and try and give them pointers for specific areas they would need to improve on to work here and try and point them at resources they can use to improve. We're in a relatively small market, and it pays to be nice to everyone. Anyone we interview I am pretty much guaranteed to run into at some sort of dev event. And just because someone wasn't a good fit today, it doesn't mean they won't be the perfect candidate next year or two years from now.