4 ms·
Another problem is the disconnect between "recruiting is extremely important to the company" and "the interview process should be as thorough and regimented as
by btilly 14y ago
Another problem is the disconnect between "recruiting is extremely important to the company" and "the interview process should be as thorough and regimented as possible", as if the latter followed from the former.
There is a reason for the regimented process that you're talking about.
The challenge for any large company is to delay, for as long as possible, the inevitable reversion to hiring the mean. Under normal circumstances, different teams will have different standards, and by the time you realize this you'll have a lot of teams whose standards have slipped, and you will have a lot of marginal employees.
Therefore smart large companies try to create a minimal process that aims to create uniform process across the whole company. The process tries to be lightweight as possible on individual people, but fights the good fight against entropy.
Unfortunately for Rachel, Google's process centers around the idea of having well-calibrated interviewers whose interview characteristics had been so well tested against others that hiring committees know exactly what to make of their interview feedback. (What absolute scale you grade on does not matter much. How your scale compares to everyone else, and how well it predicts everyone else, does.) Once an employee winds up as a well-calibrated interviewer, HR would like to use that person as much as possible (and each use does a better job calibrating the employee, and increases their desirability as an interviewer).
This is inconvenient for the employee to whom it happens. But, on average across all developers, the process is quite light-weight.
Instead, the formal interview processes I've seen are gauntlets of trivia and case studies and successive whiteboarding exercises that exhaust everybody. They don't generate value proportional to their cost. And they're unsustainable, so that whatever marginal effectiveness they have is compromised totally within a year when everyone who participates gets sick of it and starts phoning things in.
Google's process has been sustained company-wide for a number of years now. There is much that is imperfect about it, but evidence suggests that it is sustainable.
- tptacek 14y agoSo first, I should be more careful. Google is operating at a scale that I don't understand. My experience tops out at ~100 employees. Let's stipulate that Google has a hiring problem that very few other companies have, and let me shift the focus back on to startup hiring, which is what I care about. And then: I want my point to stand: for a given role, there are good hires and there are bad hires (where "bad" includes "meh"). There is not a lot of value in grading past that. But lots of hiring processes are premised on doing exactly that; they take candidates who can demonstrably perform the task required for the role and then beat them up with trivia questions, with puzzles, with nerd dominance quizzes, and with subjective stuff. I point that out because if you can find a way to accept that, you can streamline a lot of recruiting process and make interviews less painful for everyone. The way I finally managed to make that leap of faith was to reflect back on everyone I've had a hand in hiring that didn't work out, and then consider the process that hired them. Then I think back on everyone I've hired who has surprised me with their awesomeness (some of them are even people I'd negged). I am now sold on the fact that most tech interviews are voodoo rituals. What I would do/am doing instead: * Get people out of HR/phone screening as quickly as possible. * Have an explicit up-front call with me or someone senior in the hiring process to level-set people, try to defuse some of the tension, explain the process, and answer questions. Things go faster and more smoothly when candidates are convinced they want the role. The amount of sales effort it takes to get a candidate into the pipeline is not enough; you need to engage them on what the job is like, let them ask what the day-to-day is, and get them revved up before they interview. * Minimize phone screening, both in length and in depth. I am on a mission to get rid of them altogether but am not convinced I'll succeed. In the interim, what I'd like to do is take all the effort and expense we'd sink into hour-long phone screens and redirect it to coming up with a small rotating battery of really, really good questions that we standardize on. * Replace phone screens with challenge problems. Calibrate them for the length of time a series of elaborate phone screens would take. Try to make them fun. Here I admit I'm not sure what I'd do back in my pure dev jobs to make this work, but I'm sure I could come up with something. * Give the on-site interview team a full dossier of objective facts about what our up-front process learned about the candidate, so that nobody is trying to read tea leaves out of resumes. Candidly: I barely glance at resumes anymore, and I wince when I read interview reports that go "I read XXX on the candidate's resume and so asked YYY". The most important thing I think I've learned about recruiting in the last year --- apart from getting better at getting people in the door --- is to generate sets of facts that are easy to compare between candidates. I'm learning a lot by reviewing the history of how people do on challenges, and I really very much want to be in the same place by the end of the year with all person-to-person screening interviews. A nice side effect I'm hoping for is for prep time on phone screens to approach zero, and for actual phone time for interviews to approach ~20-30 minutes. We'll see.
- nbm 14y agoSelf-selection can potentially be powerful for a startup (although maybe less so recently), meaning that it may be possible to spend significantly more time considering each candidate. Skipping initial recruiter calls for at least a subset of candidates, for example, and having a hiring manager reach out and work on expectations setting and sell is something that might happen less often in a larger company. Similarly, self-deselection is something you want to avoid - ie, you need to prove yourself to them a lot more. Making sure people you are interested in see the sorts of information they might need to consider working for you is important - maybe it's talking about your product/opportunity, your development/product/management/business philosophy, or even just mentioning who works with you and what they've done (although focusing on the VC team, or just saying "A bunch of former employees at Google, Apple, Microsoft, Facebook, Dropbox, ..." is probably not enough). One thing I've found is that certain people are better at asking certain types of questions. I love "exploring" conversational-style interviews that are highly interactive and where not knowing something is an opportunity for the candidate to figure it out themselves or to learn a new thing. One colleague does an awesome debugging scenario that I sometimes think could continue for hours if he wanted to. I assume that certain candidates like particular interview styles - finding some way to match those two together would be great. (A recent Facebook start and I spent an additional half-hour during his interview because we both enjoyed the subject we were discussing. Making each interview that enjoyable...) I'm not entirely sure what you mean by challenge problems - is this a technical phone interview with a specific set of broader topics/challenges, or is this something that gets sent to the candidate and they get some time to return a result? I prefer the term "phone interview" when a non-recruiter/sourcer is evaluating the quality of the response to some question or challenge, and "phone screen" for the initial recruiting discussions where wholly unsuitable candidates are weeded out due to skill or experience mismatch to the role based more on treating commentary provided by the candidate as fact (although it does go a lot further than that).
- Dn_Ab 14y agoI don't know if you have read Thinking Fast and Slow but your observations are in strong agreement with research. From what I have read (mostly Kahneman) most interviews as performed are useless and suffer from the illusion of validity. Instead, structured interviews and scoring that are the same for all candidates are strongly recommended. Which is, draw up a short list ~= 6 independent factors which from experience are reflective of the job and whose answers you can easily judge objectively. For each factor come up with a small number of questions with a score, say from 1 - 5 and then simply add up the scores. No fancy weighting required - research shows that regression does not add much to this simple model. To avoid the "halo effect" do not jump question categories. This will vastly improve upon the more common unstructured interview and though still far from perfect you will be hard pressed to improve on that short of an IQ or equivalent test. Interestingly, and reminiscent to bagging in machine learning, unstructured interviews can be brought nearly up to par by averaging multiple independently scoring interviewer opinions.