10 ms·
Interviewing candidates
- jkaptur 11y agoThis is almost exactly the template that I apply to my own interviewing. However, I switched away from asking about a project on the candidate's resume because I found that that question - specifically meant to put people at ease - did not in fact put them at ease. Instead, I found that people became surprisingly flustered - including one who said that "like a lot of things on there, it sounds cooler than it is" and went on to question his own commitment to the field. I asked about the project because it sounded awesome, and, when I pressed him to please tell me about his role, it was awesome. I think that many people are not prepared to discuss every item on their resume. I don't think that's great, but I don't want to base a large amount of my decision on that fact. This question threw them off for the rest of the interview, so I've stopped asking it.
- ericlippert 11y agoThat's interesting feedback, thanks. I have found that most of the time people are happy to talk about their work, but I have also encountered the flustered reaction you describe.
- eli 11y agoSome people -- i.e. introverts -- are uncomfortable talking about themselves and their strengths and accomplishments. Unfortunately that's kinda the point of the interview. Not sure there's much you can do about that besides (as you suggest) pressing them to answer questions and cutting them some slack for not being super forthcoming.
- deleted 11y ago[deleted]
- warcode 11y agoAs an introvert I can tell you that most of the time anything in an interview will be flustering if unknown. Tell them to pick a project or two and prepare to talk about it ahead of time of the interview. Actually many of the issues I have with the current "standard" interviewing process is all about the fact that its a "school test" environment instead of a "prepare for a specific task" environment. The latter being what real work is actually about.
- tptacek 11y agoI feel like this is how everyone thinks they interview candidates, and that it doesn't really work. I wrote about this at length, specifically so I wouldn't write the same long HN comment every time this comes up. :) http://sockpuppet.org/blog/2015/03/06/the-hiring-post/ http://sockpuppet.org/blog/2015/03/06/the-hiring-post/
- ericlippert 11y agoThanks for writing that up; really interesting stuff. I agree with many of the critiques you make. I suspect that it may be easier to get the sort of standard process you describe implemented in a relatively small organization that has a relatively consistent set of products and commensurate challenges than in a large, diverse organization like Microsoft, where I learned to interview. The Microsoft interviewer has the challenge of finding candidates that are good both for the specific team they're interviewing for, and also have the general skills necessary to work with other teams in the future. As you say, it's a hard problem.
- epsylon 11y agoBut Microsoft is a large organization, with deep pockets! Is time spent crafting meaningful challenges for each team really that much more than, say, the man-hours that each team will have to spend to interview candidates?
- jkaptur 11y agoI appreciate that you're tired of writing the same comment over and over, but I'd really appreciate your going into some depth about what "this" is that you're referring to and how it contrasts with your approach. It seems like the original article describes a very scripted, standardized interview that tries hard to put the candidate at ease. If you were to condense your article into a sort-of "Joel test" for interviews, I imagine that this process would score reasonably well.
- tptacek 11y agoIt's based on the assumption that it's reasonably possible to put a candidate so at ease that they can solve interesting programming problems in front of an interviewer. But in-person interviews are inherently hostile, if for no other reason that they're timed and adversarial. It's better that we jettison the entire notion that they can be a venue for demonstrating programming ability. I'm not opposed to interviews. I just think you need to factor out the programming from them.
- doragcoder 11y agoI feel like I follow this template as well. Except, I like to add a small self assessment at the beginning of the interview. Like: Q: On a scale from 1 to 10 (1 being low) how do you rate yourself on "Programming Language Du Jour"? Then I can see if their self assessment is inline with mine. Which can help in reviewing the resume. I also like to ask for them to choose a project in the past to talk about, because then it's easy to see what they are passionate about. Or easily tell if they are choosing to talk about something because they think, you think, it's cool.
- EliRivers 11y agoI was asked by a hip, happening, popular web-app related startup to rate myself in C++. I pointed out that Bjarne Stroustrup rated himself 7 out of 10 once. I wasn't as good as him, and I wasn't nearly as good as him, which left me the field of 5 and below. I can guarantee that some ham-fisted chancer who'd flicked through the "Teach Yourself C++" chapter summaries put himself down as an 8, though.
- stygiansonic 11y agoAgreed - the answer to this question is really the result of how much one thinks one knows. Because a little knowledge can result in false confidence, those that rate highly will fall into two categories: Those that actually know, and those that don't. If the interviewer can distinguish between these two categories, then what value does the question add? If the interviewer cannot and is relying on the answer to this question (even partially), then it's an example of relying on a false signal. Whenever I'm asked this question, I always respond with the following questions: 1) What does a "10" mean? 2) What does a "1" mean? 3) What's the difference between 5, 6 and 7? 4) (Optional, this could easily come across as being snarky) What's the range of levels in your team/org/dept for this metric?
- doragcoder 11y agoI would welcome the challenge in what rating is what. Because I know they are highly subjective. This is not an adversarial question, it's mainly used as an indicator to gauge where to direct the interview. I like to interview people across a range of subjects (as I have not had the chance to hire a pure specialist). So knowing what they feel confident in answering will help me tailor the questions to what their perceived strengths are. So I think it gives me a feel of your confidence level in a language. If you flipped through a "Teach yourself C++ in 24 hours" book once, then rate yourself a 10 on a scale from one to 10. Then not answering basic C++ questions will raise red flags. If they answer a "1" on something. (And I re-assure them that answering 1 is perfectly ok). Then I can feel a lot more comfortable giving them more hints, or a lot more supporting detail in a question that they have indicated they are not very strong in.
- RogerL 11y ago"Red flags often surface at this point. I’ve had candidates with PhDs in computer science, for instance, who did not know that on a 64 bit architecture, pointers are 64 bits wide." Because learning 64-bit architecture is impossible for somebody that have proven that they can learn a field, perform research, and successfully write and defend a dissertation? '32 < 64' is beyond them?
- bzz01 11y agoI guess author assumes that if you don't know basic things like that, it is questionable that your degree is a proof that you "can learn a field etc." Not sure that I agree on the choice of this specific "red flag"; on the other hand, having PhD in CS is a red-ish flag to me per se.
- mediocrejoker 11y ago"... on the other hand, having PhD in CS is a red-ish flag to me per se. " I would be interested in hearing your reasons for that.
- bzz01 11y agoSo of course I'm generalizing, not everyone is like that, not everyone's CS degree is like that, etc. etc. Also, more true for recent grads. But in my experience people with advanced "pure CS" degrees seem to be focused on the process not the result. I mean, they went to CS because they like to tinker with type systems, elegant algebraic concepts, nice abstract problems -- and not just as a hobby, they like it so much so they commited several years of their lifes to do that. Not that this is bad area of research, but generally when you look for a software engineer you look for someone not only smart but also pragmatic, who can get stuff done in a simple and efficient way, and this is kind of the opposite. Of all PhDs, in my experience people who have degrees in areas that use programming as a tool not the goal in it self, e.g. physicis/EE, make the best software engineers.
- teraflop 11y agoYou could say that about almost any kind of knowledge-based evaluation. If I ask somebody ten factual questions and they know the answers to five of them, how should I interpret that? Sure, it would be easy for me to just fill in those particular gaps in their knowledge. But assuming my questions were well-chosen, it's reasonable for me to expect that they'll continue to hit similar gaps in the future, about 50% of the time. That's not to say that a good interview consists of drilling someone on facts; more often, they come up implicitly in the context of solving a problem. And as an interviewer, I'll look much more favorably on a candidate who recognizes what they don't know, because I give them the benefit of the doubt and assume that in a real-world situation, they would be able to do the necessary research. But even so, if there are basic things they don't understand, it's a red flag because it tells me they're encountering these concepts for the first time.
- at-fates-hands 11y ago"Second, what actually did they do on this project?" A lot of developers I know when in interviews say, "Well, the "team" did this and "we" did that. You always want to use singular personal pronouns such as, "I did this", or "The team expected me to do such and such." This is something a lot of potential hiring managers like to hear. If they have to ask what you did, many believe you're role or your work may not have been that important.
- reagency 11y agoYes, parsing "I/we" is a great way to filter out collaborative team players and hire antisocial narcissists. Is that what you want, though?
- ryandrake 11y agoI don't care as much about what the rest of the team did, because I'm not interviewing them. I'm more interested in what you, the interviewee, personally did to help the team achieve success.
- dvt 11y agoI don't really get it. This is sounds like a very pretty typical software engineer interview process. The fact that these kinds of interviews suck, don't produce good hires, and yield a ridiculous amount of false negatives has been beaten to death. Unless you're Google (or Facebook), you are not getting thousands of applications a day. You don't need to emulate their hiring process. They do it for a reason (practicality) whereas it seems others do it out of pure snobbery. Whiteboard coding is worthless (barring simple fizzbuzz tests which can be done over the phone anyway). I mean how pompous does this shit sound: "All candidates eventually figure out that they will need to add an auxiliary data structure that maintains a map from a 32 bit integer to a 64 bit pointer, because of the aforementioned ten pound sack. Do they know that there are off-the-shelf map classes? If not, do they have confidence that they could write one?" Eric Lippert is originally from Microsoft though, so I guess that shouldn't surprise me. After all, they're the ones that started asking "Why is a manhole cover round?" in software engineering interviews. Much respect goes to tptacek that outlines a much better and enlightened alternative.
- ericlippert 11y agoWell, my intention was not to sound pompous, but writing is hard, you know? Thanks for your cogent critique of my tone.
- reagency 11y agoI hope you are being coy, because dvt's post was obnoxious and ill-founded.
- ximeng 11y agoRegardless of whether you or tptacek has the better method for hiring coders, I have much more problem with dvt's tone in his post than yours in the article. Hope this kind of response won't discourage you from posting more, as I've had a lot of value from your writing in the past and found this article interesting as well.
- AtmaScout 11y ago
- gfodor 11y ago"How are things going?" is actually a terrible question to ask someone in an interview, if that's literally how you ask it. It is saying to them, in so many words, "tell me how you think you are doing, performance-wise, so far?" Horrible, and about the direct opposite of a question that will put someone at ease. Questions that will put someone at ease are ones that have nothing that can be construed as something you are assessing their answers to, like asking them about their visit to the area or if they enjoyed lunch or something like that. Break the ice, tell them about what you work on, see if they need a break, and then start digging into stuff. I also like to tell the person up-front what we'll be covering in the interview so there are no surprises. Walking in the door and asking them a question they could interpret to mean to reflect on their interactions with the previous interviewers makes me cringe.
- ericlippert 11y agoWell as I said in the article, the intention of the question is emphatically not a "gotcha" about how they are actually doing. It's intended to find out if there are any problems that need addressing.
- gfodor 11y agoYeah it's a tough one I think to ask well -- maybe you've got the tone and wording down in a way that candidates won't misinterpret, but I've avoided that type of question because I'm not able to do so.
- lukasm 11y agoWhy not do 2 phone screens to filter out completely incompetent people and then let them choose: - paid work for a week - writing patch for a project or making simple tool(proxy of the job) and invite them to talk about it.
- civilian 11y agoBecause candidates have lives outside of code. They probably already have a job, so taking a week off for "vacation" which will actually be a "paid interview" sounds like a crummy deal. Writing a patch-- similarly, it's eating into their own time. You're going to lower your hire rate from that.
- dagw 11y agoIf they can take half a day or a day out of their life to show up for an interview then certainly they can take the same time out of their life to write some code instead.
- lukasm 11y agobut it's paid so you can take a week off between jobs. As I said, give them an option. Interview can take a whole day, doing a project pretty much the same, but you can split it into a few day or do it over the weekend.
- douche 11y agoWe did this for our most recent hire - gave them a short project to implement over a long weekend. I think we shot for a target of having it take the applicants at most four hours; I threw together the reference example in an hour or two. I think it was a success, and we'll probably use this method going forward. There's just no substitute for seeing actual code, and by extension, the way someone structures the problem in their mind. It also give you some easy jumping off points to discuss with the applicant - Can you explain why you used data structure X here instead of Y? Change this requirement, and how would you do this differently? etc.
- mparramont 11y agoI also wrote a thing about coding interviews, from the employee side: http://www.developingandstuff.com/2015/05/why-i-dont-do-coding-tests.html http://www.developingandstuff.com/2015/05/why-i-dont-do-codi...
- Bartweiss 11y agoIt had never occurred to me until I read this, but I have a sudden urge to ask my interviewers simple coding questions like "reverse a linked list". In most cases it would be a silly waste of time, but it would also have saved me from one or two truly horrific jobs with skill-free managers.
- m3talridl3y 11y agoHere's how you hire a candidate, in two steps: 1) If they provide a github/bitbucket/sourceforge link, look at what's there. Skip step 2. 2) Literally anything else. It won't work anyway.
- therobot24 11y ago> Leave the candidate with a positive impression of the company I feel like I haven't experienced this enough in interviews. Mostly I assumed that if I'm applying then interviewer already thinks that I have a good view of the company (or otherwise I wouldn't want to work there right?), but it would be nice to get a better outlook of the company. Maybe I'm just asking the wrong questions.
- cechner 11y agoThis was my thoughts on the matter: http://seshbot.com/blog/2014/01/24/the-best-way-for-programmers-to-interview-programmers/ http://seshbot.com/blog/2014/01/24/the-best-way-for-programm... summary: ask the open ended question ‘what was the most recent interesting project in which you were heavily involved?’ and then focus most of your technical questions on the system the then describe. If a programmer can’t talk throughly about a topic which they chose themselves and are purportedly very interested, they probably aren’t right for the job (at whatever company I’m working for anyway!) The benefits are that you wont be randomly hitting a gap in their knowledge, you get a better idea of what they feel are their greatest strengths, and because you can raise your expectations about the quality of their answer you can probe very deeply and find out how good they are at communicating, and whether they are actually blagging. The problems you can encounter are: they might end up running the interview if you aren't careful; you must be very engaged and perhaps talk about things you aren't that knowledgable yourself, and it is harder to compare one candidate with another (which is VERY important at large companies where you must justify your decisions to HR)
- ausjke 11y agoLove this, very useful for me, especially the gradually elevated technical interview steps/details, though it's less meaningful for junior engineers, who normally can't think that deep yet be it at interview or on the job.
- ksk 11y agoI agree with what was said in the article. I'd say one half of programming in general is objective in the sense that either the program works or doesn't. The other half is completely opinion based - be it a design philosophy, choosing algorithms, future proofing APIs, hardening code against malicious actors, etc. This half is very fuzzy because there are no right/wrong answers as it goes more towards taste. IMHO you should try to evaluate the following - "What has the candidate done that makes their opinion worth something". If you can't come up with a very strong reason to give any kind of weight to their opinion, then I'd say do not hire them.
- dbenhur 11y ago> if we make a bad hire, that can drag down the productivity of a team for years This is the essential cowardice of employers. Interviews are insufficient contexts to judge if someone can help move your business forward. Screen the incompetent, but just hire people and don't be afraid to let them move on if they don't work out after 2/=4/6 months.