19 ms·

The Utter Uselessness of Job Interviews

• cricfan 10y ago
  I wonder if we can extrapolate to marriages and to how arranged marriages (at least in India) having a higher success rate. Usually the parents on either side decide on a match based on family background, financial stability, education background etc,. rather than letting the to-be married decide.
  • keyboardhitter 10y ago
    Perhaps, but then one would have to define a "successful" marriage, which is very subjective (even if only Indian families are scrutinized).
  • cthalupa 10y ago
    What is 'success'? My evidence is anecdotal, but I've spoken with multiple Indian women who are in arranged marriages, have zero real love in the relationship, and aren't particularly happy, but the idea of divorce is entirely unthinkable to them. The lack of divorce probably makes this a 'success' by most metrics, but doesn't seem particularly successful to me.
• akhilcacharya 10y ago
  It's interesting, because I'd argue that the best companies in tech have have interviews that are very structured and predictable.
  • PlaceFan 10y ago
    Is that true? Isn't the generic accusation against AmaGooFaceSoft that your interview performance largely depends on whether you happened to study some interviewer's pet topic (e.g. parallelism, dynamic programming, networking, etc.)?
    • rb2k_ 10y ago
      Having done interviews at a few of time, I'd say that's mostly false, at least for the last 2-3 years. As far as I can tell, there's always a pre-defined pool of questions. Some of them are pretty 'open ended', but still targeted towards getting a good impression of a certain area of knowledge. That being said, I had an interview with Google at some point in the past where one of the interviewers almost seemed appalled that I didn't know the exact list of items that can be found in a filesystem superblock. But at all other companies it seemed a bit more sane. I guess it's partially a function of the type of personalities a company is willing to hire :)
      • anonymoushn 10y ago
        Recently I failed an interview at Facebook because I chose to serialize a binary tree into a list of N items, rather than a list of N items and O(N) sentinels. Most of the interview was spent convincing the interviewer that this could possibly be correct.
        • dastbe 10y ago
          was it a binary tree or a binary search tree? a BST can be uniquely identified by one of pre/post-order traversal, whereas a binary tree requires an in-order traversal and one other. or placing sentinels that specify a state machine [PUSH_LEFT, PUSH_RIGHT, POP] that specify how to move down and up the tree structure.
          • anonymoushn 10y ago
            It was a BST.
        • z3t4 10y ago
          i recently failed a programming test after acing all tests except one where i needed a text diff and pulled in a library instead of writing my own in less then ten minutes. they said i need to get better at algos. which is fair. but i dont know anyone that can program a decent working text diff in less then ten minutes.
  • tptacek 10y ago
    As I said upthread, I think this is not true, at least not in the sense that the author is using. Could you describe some of the structured processes you've seen?
• rb2k_ 10y ago
  It should probably have been titled "The Utter Uslessness of Unstructured Job Interviews", because that's the kind of interview the author criticizes. In my personal experience, structured interviews can be very helpful in determining a candidates abilities.
  • sverige 10y ago
    From the hiring side, I agree. There's no other way to smoke out those whose creative writing skills extend to their resumes, and you get a pretty good idea of personality differences between candidates if you use the same pool of questions for a given position.
  • tptacek 10y ago
    Are you sure you're using the same definitions as the author? In practice, across a pretty decent sample of large tech companies, I've never seen a truly structured interview outside of Matasano, where I was almost murdered by my employees for instituting them. I think we'd hear far more complaints about them if they were common. It's possible that what you consider to be a structured interview is in fact what this author (and I) would call unstructured. Specifically: if the interviewer has any discretion about questions at all, the interview is probably fundamentally unstructured. In a structured interview, the interview is less an interrogator than a proctor or a referee. Every candidate gets identical questions. The questions themselves are structured to generate answers that facilitate apples-apples comparisons between candidates: they generate lists of facts. The most common interview questions, those of the form "how would you solve this problem", are themselves not well suited to these kinds of interviews. It takes a lot of work to distill a structured interview out of those kinds of problems.
    • cwyers 10y ago
      How did you feel about your experience using structured interviews for hiring?
      • tptacek 10y ago
        Truly structured interviews are better than even the most rigorous traditional interviews. They are also more expensive to design and much more painful to deliver. Some kind of interview is probably necessary for every serious tech hiring process. Organizations should be realistic about the low quality signal they'll get even from structured interviews. Take time away from interviews and feed that time to off-site work-sample testing.
• inopinatus 10y ago
  Articles like this - and the comments that follow - always overlook the primary value of job interviews, which to me is answering the question: "Do I want to work for this company?"
  • sverige 10y ago
    And articles like this are almost universally written from the perspective of someone being interviewed, rather than the one interviewing.
    • IIIIIIIIIIII 10y ago
      The article that I just read following the submitted link was from the perspective of neither, but of someone looking at the hiring process as an observer. The anecdotes are anecdotes and they don't matter except to lighten up a newspaper article for the masses, so we can ignore them - I'm talking about what was actually studied.
  • throwaway9475 10y ago
    Sure, because poor management, crippling technical debt, and a toxic work environment are totally plainly presented to you in an interview. /s
    • inopinatus 10y ago
      Not plainly, no, but easily inferred.
      • throwaway9475 10y ago
        You should publish because I know I and a lot of other people would be incredibly interested in divining this information during a job interview.
        • flukus 10y ago
          The best I've come up with is a few proxies, like their deployment process. Someone doing continuous or frequent deployments probably have pretty decent code quality. A company that takes several weeks, several rounds of QA and a dozen release documents that need to be signed in triplicate, they have that process because they've been burned and need scapegoats when the inevitable happens. The best part is that most people at companies like this aren't aware that it's not normal, so they'll be open about it.
• Boothroid 10y ago
  But surely structured interviews just test a candidate's ability to improvise plausible stories? Whether they are truthful or not is a different matter..
  • scarmig 10y ago
    Being able to improvise plausible stories is an important aspect of job performance, both internally and externally.
    • Boothroid 10y ago
      One aspect amongst many?
  • threatofrain 10y ago
    Structured interviews improve the ability to systematically test the relationship between policy and results. Real experiments are then possible. As opposed to a situation where your interviewers are asking off-the-cuff questions, and you're uncertain whether the questions have a systematic bias toward a bad direction -- racism, sexism, or just selecting for something idiosyncratic that adds unnecessary constraint to your candidacy pool. You also now don't know whether it's your policy that's systematically bad (or good!), or that your interviewers are systematically bad.
    • vacri 10y ago
      And yet unstructured interviewing basically works and the world keeps spinning. If 'random' truly was a better result (as the article suggests), then in a hiring round for a programmer last year, we might have discarded our candidate that won and is awesome for the one that couldn't conceptually 'get' FizzBuzz. Or put another way: if 'random' was better than 'unstructured', you'd never have a round of hiring where there were no unsuitable candidates - one will have been chosen.
      • Boothroid 10y ago
        Better to give someone a job related task perhaps?
• gatsby 10y ago
  Laszlo Bock (former SVP of People at Google) did a great job summarizing decades of research around structured interviewing in his book 'Work Rules!' For a quick reference, the two defining criteria for a structured interview are: 1.) They use one or several consistent set(s) of questions, and 2.) There are clear criteria for assessing responses That second point is really important. You can't only ask candidates the same sets of questions and have a structured process: you need to understand what a "one-star" response vs. a "five-star" response actually looks or sounds like. Training and calibrating all of the interviewers in a large company around a similar rating system is nightmarish, so most companies don't bother. The book also outlines that pairing a work sample with a structured interview is one of the most accurate methods of hiring. If anyone is interested in some in-depth structured interview questions or work sample ideas, feel free to email me. I've spent the last few years working on a company in the interviewing space and would love to chat.
  • andrewmutz 10y ago
    How does this work in practice? If there are consistently used questions and specific criteria for assessing responses, can a candidate just learn the likely questions and what constitutes the "right" answer?
    • tptacek 10y ago
      This is a good and important question. Also: while I have a lot of respect for people at Google trying to innovate on hiring, make no mistake: Google's heart is in the right place, but they aren't at the forefront of structured hiring, and their hiring processes are notoriously capricious. The reality is that generating good questions for a structured interview is difficult. You can't just pose a programming problem. As you've noted, most programming problems have multiple good answers. Differentiating between multiple good answers from different candidates re-introduces subjectivity. Your brain would rather convince you that one valid answer is less good than another, even when it's not, that to admit to you that it can't differentiate or generate a narrative for you. The part of your brain that generates narratives is incredibly powerful and does not care about how accurate your hiring process ends up being. What we tried to do was create questions that generated lists of facts. "Spot all the errors in this code" would be an example of this approach (but none of the three we used). We went into the process wanting to embrace epistemological uncertainty, generating a historical trail of data that we could retrofit to candidate performance. In the end, work sample testing was so much more powerful a predictor for ourselves that we never fully got around to analyzing the data. Sometimes we'd get candidates that clearly generated inferior "lists of facts"; I think there may have been 1-2 instances where that outcome actually overruled work-sample testing delivered prior (out of a few tens of engineering hires and probably ~100 interviews).
• DrNuke 10y ago
  It is some time we are going towards word of mouth and peer recommendations as the preferred way to hire new personnel. Cold applications are for outsiders and as such a very different market, with all the strings and the bulls.it attached.
• hobls 10y ago
  I've been a programmer for a bit over ten years. I've worked at scrappy little startups, midsized companies, now for a tech giant for a few years. The engineers I work with at the tech giant are consistently better engineers than my other coworkers have been, and I credit the very structured interview process. We're trained to ask specific questions, look for specific types of answers, and each interviewer is evaluating different criteria. Also, it is quite often not the technical questions that end up making us decide not to hire someone. That's just one area. I know it's the part that sticks out, and candidates give a lot of weight to it in their memory of the interview, but you shouldn't just assume it was that your whiteboard code wasn't quite good enough. I actually don't think that's the most common thing we give a "no hire" for.
  • dikdik 10y ago
    Which of the companies paid the best/has the best benefits? If it's the tech giant, it's very possible they attract better candidates because they offer more.
    • NikolaeVarius 10y ago
      In my experience, the big Corps don't tend to offer anywhere near as much compensation but better days off etc etc. The best sofware guys I've ever seen work at those big corps. The big corps are the ones that have the resources to work on the REALLY hard problems and not writing the same CRUD app over and over again. Those same companies also provide tons of educational benefits so that you become an expert in your field. Things like fighter jet software are fantastically complex. You hear about the dumb mistakes that are made (International Date Line Issues), but you never hear about the thousands of hours of testing every change to code goes through and the insane calculations that are made every second in even standard level flight. I have no idea why there is such a backlash against interviews. What's so hard about studying for a job you want? Even if you don't USE knowledge, you should at least be able to rederive things using your base knowledge. Thoe interviews I've seen at aero companies are damn hard. Tons of grilling on if you actually understand mechanics and fliud dynamics.
      • 10y ago
• erikpukinskis 10y ago
  Once we get to the point where most people have several jobs with separate contracts, interviews become superfluous because you can just hire someone for a few hours at a time and then fire them. The only reason that doesn't work today is we're still clinging to the idea that you only work for one entity at a time. Never mind that most people already manage at least a couple bosses within the same company.
  • __s 10y ago
    It took months to become familiar with the existing tech at the company I'm currently working for. Still learning their internals
  • wpietri 10y ago
    I've done a bunch of consulting and contract work, and that can be fine. But it's expensive, because to be continually finding new work, you must continuously invest in marketing, and every time you do something new there's a lot of switching cost. A company also needs a lot more supervisory capacity to be monitoring people who come and go, so it's not cheap for companies, either. It's also limited. I can be much more productive when I do one thing full time than a bunch of things part time. Depth takes time, as does keeping up with changes. You just can't get as far with fractional attention as with serious focus. And uncertainty is also uncomfortable. Many people just want to settle into a solid, reliable situation and do the work. They want to be able to plan their future with some confidence. Even if they can make more money juggling a variety of things, they'd rather make less and have less chaos. So I don't think we'll ever get to the point you suggest.
    • pdimitar 10y ago
      You both have a good point. I too quit my freelancing for a stable remote job but looking at it 18 months later I found out that I sucked a lot in managing my freelancing gigs. So IMO with some good contract and budget management -- and confidence, and actually having a choice -- you can reap most the benefits of freelancing and almost none of the drawbacks, if you can take the thought of switching customers every 3-6 months. I understand not everybody can pull this off -- I am not sure I am yet ready to do this that well. But I've seen people doing it very successfully and almost stress-free.
• damagednoob 10y ago
  > In one experiment, we had student subjects interview other students and then predict their grade point averages for the following semester. Not sure how using inexperienced interviewers proves anything. Would have been more interesting to have lecturers interview the students.
  • manarth 10y ago
    There's a reference in the article that covers that kind of scenario. The additional 50 students that the school interviewed but initially rejected, did just as well as their other classmates in terms of attrition, academic performance, clinical performance, and honors earned.
• redthrow 10y ago
  Matt Mullenweg advocates audition/tryouts instead of job interviews. https://hbr.org/2014/04/the-ceo-of-automattic-on-holding-auditions-to-build-a-strong-team https://hbr.org/2014/04/the-ceo-of-automattic-on-holding-aud...
  • Merad 10y ago
    That has to be one of the most employee-hostile hiring strategies I've ever read.
    • cwyers 10y ago
      You're pretty much limited to hiring either only the jobless or someone with the ability to work two jobs at once for an extended period of time while you hold your tryout. There are a lot of people who can't devote that kind of time to a second job (some candidates burn a week's vacation!) And it's not some mere formality, according to their stats they hire about a third of the people they try out.
      • gjkood 10y ago
        And don't forget anti-moonlighting clauses in current employment contracts.
      • Ensorceled 10y ago
        I presume they only hire about a third of the people who try out for the same reasons you mention; the only candidates are the jobless or people willing to screw their current employer and moonlight while "working from home". Sounds like a recipe for a bunch of bad candidates.
      • Merad 10y ago
        > or someone with the ability to work two jobs at once for an extended period of time while you hold your tryout. This is the part that's crazy to me. If you think you can accurately judge someone based the work they perform after a full normal workday/week, I have some beachfront property on the moon to sell you... I have a feeling their conversion rate is so low because a lot of people get another offer during their trial and immediately jump ship.
• dkarapetyan 10y ago
  > The key psychological insight here is that people have no trouble turning any information into a coherent narrative. This is true when, as in the case of my friend, the information (i.e., her tardiness) is incorrect. And this is true, as in our experiments, when the information is random. People can’t help seeing signals, even in noise. People see patterns where there are none. I think this is fundamentally why humans fail at statistics. If every fiber of your being wants to see patterns then you will see patterns. Probably why people hallucinate when in sensory deprivation tanks as well. The brain will make up patterns just so it can continue to see them. The paragraph right after follows up with the statistical failure that pattern seeking leads to > They most often ranked no interview last. In other words, a majority felt they would rather base their predictions on an interview they knew to be random than to have to base their predictions on background information alone. So people would rather do busy work in order to continue to satisfy established pattern seeking habits than figure out a better way.
• tptacek 10y ago
  Take a second to read about the experiments this author conducted. They included: Dummy candidates mixed in with the interview flow that gave randomized answers to questions (interviews were structured to somewhat blind this process), and interviewers lost no confidence from those interviews. Interviewers, when told about the ruse, were then asked to rank between no interview, randomized interview, and honest interview. They chose a ranking (1) honest, (2) randomized, (3) no interview. Think about that: they'd prefer a randomized interview to make a prediction with over no interview at all. Of course, the correct ranking is probably (1) no interview, (2) a tie between randomized and honest. At least the randomized interview is honest about its nature.
  • vacri 10y ago
    The interview is also selecting for a single thing - GPA. You can be an utter arsehole and have a high GPA. You can have personal hygiene that stinks out any room you're in and have a high GPA. Basically, you can be completely impossible to work with and have a high GPA. The research they've done is suspect, because they weren't interviewing for an ongoing role, but for a single KPI. Similarly, the people doing the prediction were other students rather than teachers who know better what to look for. The research would better fit HR people hiring for roles they know nothing about than experienced team members hiring for roles similar to what they do. The research results are massively over-applied, and in no way whatsoever is sufficient to use the term "utter uselessness of X". Unsurprising, given it is research by a 'professor of marketing'.
    • tptacek 10y ago
      Like everyone else, I agree a face to face interview is the best way to verify a candidate doesn't smell bad. I assume that interview could be pretty short.
      • vacri 10y ago
        Well, I suppose you could fish out the most trivial part of the comment and refute that. Why not set up a battery of short unit tests for the candidates? "Follow me, Jones, and we'll see how you handle the 'coworker with chewing gum' test next..." The study was done on students, who almost universally have zero experience selecting people to work under them in an industry setting (or at all). Drawing conclusions from this particularly inexperienced subject pool and then extrapolating out is bogus, particularly given the extremely certain language of the article. The subject pool is at an age (18-22) where people are still figuring out what to make of themselves and others; they have extremely little adult experience of the workplace and judgment of character - indeed, at this age, people are notorious for making bad decisions in their personal lives. When you look at the actual paper they link, out the window goes the declarative language, and instead the article is unusually full of weasel-words ('can', 'may'). There's a major difference between "utterly useless" and the actual conclusion of their paper: "interviewers probably over-value unstructured interviews".
• uncensored 10y ago
  How do you know that the person you'll be marrying won't cheat on you and won't leave you in hard times? If you apply the methods we use today for interviews you'll end up with a 50/50 chance at best, a coin toss. Yet why do some marriages last forever (till death do us apart) while others fail miserably or crumble even after 20 years? The search for the global optimum cannot be performed by asking a set of questions. I argue that it cannot be done consciously. It's gut/instinct thing. If you have a mechanical approach, anyone can game the system and get a job because humans can be like chameleons to present themselves as the right candidate, and they can study for the interview. Only way IMO is to have that 3rd eye or whatever you call it... instinct, gut feeling, etc The problem with this conclusion is that instinct and sexism/racism are often conflated. No good answer.
  • devoply 10y ago
    It goes both ways. Companies are often not trustworthy and will fire people for their own reasons. They may not pay very well. There are so many things. I am not Mr. Right, I am Mr. Right Now accurately describes the business arrangement. It's a one or a few nights stand. Not a lifetime commitment.
  • manarth 10y ago
    instinct and sexism/racism are often conflated. What's perceived as instinct is often the outcome of social conditioning, which gives outcomes such as "People See Black Men as Larger, More Threatening Than Same-Sized White Men" [1] [1] http://www.apa.org/news/press/releases/2017/03/black-men-threatening.aspx http://www.apa.org/news/press/releases/2017/03/black-men-thr...
    • chmln 10y ago
      Could also be due to the disproportionally high rates of crime among black men. That should never be an excuse for prejudice, however. https://ucr.fbi.gov/crime-in-the-u.s/2012/crime-in-the-u.s.-2012/tables/43tabledatadecoverviewpdf https://ucr.fbi.gov/crime-in-the-u.s/2012/crime-in-the-u.s.-... https://infogr.am/Black-34991937313 https://infogr.am/Black-34991937313
• ordinaryperson 10y ago
  The problem is GPA itself isn't necessary a valid data point. It's less fallible than "gut instinct", as the author here seems eager to claim, but personality type can be more important than ability to memorize facts. I'd rather hire a programmer who knew less and could get along with others than a master dev who's a total a-hole.
• kasey_junk 10y ago
  Structured interviews are better than unstructured ones, but in my experience they are really a Trojan horse for the idea that interviews in all forms are largely worthless (as predictors for good hiring). Once you start collecting data on your hiring pipeline work sample hiring becomes so much obviously better that it makes little sense to spend the time to do the hard work of making a good structured interview process.
• Zigurd 10y ago
  It's not surprising. Employee selection is basically voodoo. Outcomes don't get fed back into redesign of the process, and the process is far more based in tradition than data. When the process gets challenged it's ripe to fall apart.
  • YZF 10y ago
    The sample size is too small to do any useful feedback and humans are too complex to be mapped/binned in the way interviews attempt to do. Not only that but things that happen after hiring may have a bigger influence on the success/failure of the hire than the factors the interview is looking for. E.g. who you end up working with, internal politics, the specifics of the job. Kahenman's work on the illusion of validity comes to mind. It kind of boils down to our hiring process must be right because we're doing it.
• ravitation 10y ago
  I have some major issues with their conclusions... and the title of the article (which is mostly nonsensical clickbait). The real conclusion should be that "unstructured interviews provide a variable that decreases the average accuracy of predicting GPAs, when combined with (one) other predictive variable(s) (only previous GPA)." This conclusion seems logical. When combined with an objective predictive measure of a person's ability to maintain a certain GPA (that person's historical ability to maintain a certain GPA), a subjective interview decreases predictive accuracy when predicting specifically a person's ability to maintain a certain GPA. To then go on to conclude that interviews then provide little, to negative, value in predicting something enormously more subjective (and more complicated), like job performance, is absurd - and borderline bad science. There are numerous (more subjective) attributes that an unstructured interview does help gauge, from culture fit to general social skills to one's ability to handle stressful social situations. I'd hypothesize all of these are probably better measured in an unstructured (or structured) interview than in most (any) other way. To recommend the complete abandonment of unstructured interviews (which is done in the final sentence of the actual paper) is ridiculous.
  • emodendroket 10y ago
    > There are numerous (more subjective) attributes that an unstructured interview does help gauge, from culture fit to general social skills to one's ability to handle stressful social situations. I'd hypothesize all of these are probably better measured in an unstructured (or structured) interview than in most (any) other way. To recommend the complete abandonment of unstructured interviews (which is done in the final sentence of the actual paper) is ridiculous. Well, based on what? Is there any evidence for any of these hunches?
    • ravitation 10y ago
      I didn't say it was based on hard data, that's why it's a hypothesis... But, if you really believe that it's a huge leap to hypothesize that interviews would be a better measure of subjective social skills than metrics like GPA, I don't know what to tell you.
• daenz 10y ago
  As someone with many interviews coming up in the near future, this scares me. It's easy to get in a self-conscious feedback loop when you know every behavior, response, and gesture is being fed into a fundamentally irrational character-judging process. The best interview I've ever been on was one for a young startup. They gave essentially a homework problem, a day to solve it, and then in the interview we talked about the problem and my solution. The worst interview I've been on was sitting in front of multiple engineers as each one threw out a random CS question (from seemingly the entire space of CS) and asked me to talk intelligently about it. When I seemed unsure of myself, they glanced around nervously and disapprovingly. Interviews are the worst. I've spent my time trying to bolster my OSS projects, so that I can point to them as evidence of my competence, but I can't help but prepare for the worst anyways.
  • crispyambulance 10y ago
    "...fundamentally irrational character-judging process" is really the wrong impression to go into interview with as a candidate. Seriously, you will handicap yourself if you see it this way. Competency is only one component of a hiring decision. After some base threshold of competency, the question then becomes whether or not you, the candidate, is going to be someone that the team WANTS to work with. Bolstering your OSS projects is fine but you'll get a better return on investment for your time to practice "behavioral interview" questions. This is absolutely the hardest type of interview but in the hands of experienced hiring managers it works better than any other technique, IMHO. If you can find an experienced mentor who will do mock behavioral interviews with you and give you honest constructive feedback, that will boost your interview skills more than anything else.
    • daenz 10y ago
      Good advice, thanks. My experience so far has been difficulty appearing confident on the technical portions of the interviews. Other devs seem to sense blood in the water when nerves and shyness have you fumbling through whiteboarding obscure topics.
  • alfalfasprout 10y ago
• m-j-fox 10y ago
  Known useless indicators: * Resumes * Skills tests (hacker rank) * Whiteboard interviews * Unstructured interviews * Employee referrals No wonder headhunters have such a good business. Not that they're more discriminating, but they can pretend to be the solution to an intractable problem.
  • taway_1212 10y ago
    > No wonder headhunters have such a good business. Not that they're more discriminating, but they can pretend to be the solution to an intractable problem. Do companies view them this way though? I've always thought that hiring external recruiters to bring in a bunch of candidates is done to outsource the scummy task of bothering people via phone, linkedin etc.
  • tianlins 10y ago
    I agree with most, but referrals are one of the most important indicators. "A hire As, B hire Cs". A known issue, however, is the workforce diverse given the heavy use of referral methods.
    • m-j-fox 10y ago
      All my referral tells you is I have a friend who needs a job and that I want $5000. Whether my friend is any good or not is something you'll find out after I cash my check.
• numinary1 10y ago
  This discussion misses an important element, the skill of the interviewer. It is unsurprising that unskilled interviewers' assessments are poor predictors of future performance. It would be interesting to measure the accuracy of interviewers who have had years of experience interviewing, hiring, and managing people. Here's how I think it works. Skilled interviewers are biased toward rejecting candidates based on any negative impression. Structured interviewing has the same effect. It's the precision versus recall tradeoff. For this use case only precision matters. Extremely low recall is fine. Also, in the GPA prediction example, the interviewer is penalized for predicting a low GPA for a person who performed well. But in hiring, there is no penalty for failing to hire someone who would have performed adequately. (Yes, I understand there is an implicit assumption in my argument that candidates are not in short supply, but that's usually true, certainly at Google)
  • ScottBurson 10y ago
    > For this use case only precision matters. Extremely low recall is fine. Only if you have arbitrarily large amounts of time you can spend interviewing. Most of us don't.
• deleted 10y ago
  [deleted]
• throwaway71958 10y ago
  One thing interviews can't select for: creativity and motivation. And in tech those two criteria are the most vital, especially motivation. I can easily fill in the skills gap in someone who's motivated. I can't do anything with someone who doesn't give a shit, even if they're the second coming of Albert Einstein. So folks, please, don't apply for jobs you don't really care about. Save yourself and your prospective employer time, aggravation, and the opportunity cost.
  • pdimitar 10y ago
    I find it unfair that you're downvoted. I mostly agree with you. However, "don't apply for jobs you don't really care about" is a very 50/50 advice. Right now I have zero money reserves. If I somehow got fired tomorrow, I'll be in the red even after 2 days of unemployment. So sometimes you have to make a hard choice. That being said, it's good to be open about this after you get your act together months later and decide what to do with your current employer, during a lunch for example.
    • throwaway71958 10y ago
      Well, don't wait to get fired then. Find a better paying job you like and go for it. It doesn't seem like you have much to lose anyway, and the best way to increase your paycheck is by moving around and not letting employers take you for granted. Just don't sell your soul for a buck in the process. This game is a marathon, and grinding it out never really works in the long term.
      • pdimitar 10y ago
        I fully agree, and that's exactly what I am doing -- even if an offer from another company ends up only being a leverage to force a raise in my current company (see below for clarification). However, I am taking it slow and I am patient (even though NOT being able to randomly go to the cinema or a restaurant with my girlfriend is getting on the nerves of both of us lately; money is tight and I'm very unhappy with my current compensation) because I don't want to replace one problem with the same problem in another company. So I am picky, I am clear in my requirements, I don't accept terms I know will make me hate the job, and I am perfecting my negotiating skills during this entire process. CLARIFICATION on the leverage remark: it's my opinion that 99% of the time leveraging an offer from another company that wants to give you more money, to make your old company give you more money, is a huge mistake. Most businessmen HATE being strong-armed, or, to use a milder language, hate being shown that their employees have power over them, and this makes them hate you even if they very much need you in a business sense. They end up actively looking for a way to get rid of you, even if it costs them more money and/or stress in the long-term. I've witnessed it. SOURCE: 4 of my stupider younger acquantainces from 7-12 years in the past. And an observation from my first job. After I "strong-armed" my first employer to double my then pretty measly salary, he went on a hunt to replace me (even though it took him around a year to really do it), but I was smart enough to detect the signs and resigned long before he had the chance. No regrets.
• santoshalper 10y ago
  As someone who has worked at every level of IT (startup to Fortune 500 executive), hired thousands of people, and personally interviewed hundreds of candidates of all levels of experience, the conclusion I have come to is that interviews are almost entirely worthless.
• Spooky23 10y ago
  It's the whole process that's useless. Remember a few years ago when this forum was drowning about how to find and hire "10x" people? 98% of employees were useless in the face of the 10xer. The reality is, most of the time screening for general aptitude, self-motivation and appropriate education is good enough. I've probably built a dozen teams where 75% of the people were random people who were there before or freed up from some other project. They all work out. IMO, you're better off hiring for smart and gets thing done and purging the people who don't work.
  • Top19 10y ago
    This I think is a very a good point. The scariest point about Lazlo Block's book is that he is very against the idea that people can improve or be trained better. To him there is this idea of predestination. You're either always good and always have been, or you're not good and won't get better and there is nothing Google can do. I feel like as a company you could exploit a lot of value by just hiring people and training them. Training btw doesn't mean at work. Think how much you learned in college lectures vs. reading the actual textbook. It means going home and studying, and the incentive should be that you're getting things that make you more employable, so this shouldn't count towards hours (if it was say a very proprietary old programming language though I could see this argument falling apart). Not saying this is the best way but shocked that everyone is "trying to find the best". What kind of world will that leave us in if all companies want to hire the same 3% of people? Businesses move slower, talent is lost, and inefficiencies accrue.
    • Apocryphon 10y ago
      One wonders what exactly happened to this industry, why instead of training, companies are offloading the work to universities, MOOCs, and boot camps.
      • pdimitar 10y ago
        All about cost and trying to offload it somewhere else. Sadly it's very predictable, most businesses work that way. Then again, theoretically this gives a huge edge to the proactive learners.
• thomastjeffery 10y ago
  > Alternatively, you can use interviews to test job-related skills, rather than idly chatting or asking personal questions. Alternatively? How is this not the focus of an interview? Sure, confidence and social skills are important, but obviously they cannot predict a person's actual ability.
• sgt101 10y ago
  Odd - I was trained to do competence based interviews 20 years ago, apparently this is now rediscovered knowledge or something!
• evervevdww221 10y ago
  We had 2 slackers on the team. One jumped directly to Google. The other jumped around for few years, got laid off by some company, recently joined FB.
  • d357r0y3r 10y ago
    One of the most incompetent developers I've ever known got hired at Facebook, then Apple. Every line of code the guy right was just...exceptionally bad and poorly thought out. I guess he was good at whiteboard exercises, though?
• schmit 10y ago
  I find it quite problematic that researchers get to talk about their own research and present it as facts without anyone taking a critical look. Over time I’ve become more skeptical about this kind of psychology research (as more studies fail to replicate) and, as is often the case, here the sample size is quite small (76 students, split across 3 groups), predicting something relatively noisy as GPA. It is unclear to me that one would be able to detect reasonable effects. Furthermore, some claims that make it into the piece are at odds with the data: > Strikingly, not one interviewer reported noticing that he or she was conducting a random interview. More striking still, the students who conducted random interviews rated the degree to which they “got to know” the interviewee slightly higher on average than those who conducted honest interviews. While Table 3 in the paper shows that there is no statistical evidence for this claim as the effects are swamped by the variance. My point is not that this article is wrong; verifying/debunking the claims would take much more time than my quick glance. But that ought to be the responsibility of the newspaper, and not individual readers. Politicians don’t get to write about the successes of their own policies. While there is a difference between researchers and politicians, I think we ought to be a bit more critical.
  • trishume 10y ago
    The claim you mention about getting to know the person better in random interviews as well as "People can’t help seeing signals, even in noise." is misleading and unsurprising. People ask questions in interviews that they want to know the answer to and that could go either way. All questions have equivalent expected surprise, either both answers are unsurprising, or one answer is surprising, but you think you already know the answer is the other one. If interviewers were asking questions like "is 2+2=4" they would have detected random interviewers way easier, but they wouldn't be trying very hard to get to know the person. As for getting to know the person, the more surprising someone's answers are, if you believe they are telling the truth, the more distinguishes them from "average person who gave answers I expected", so you say you "got to know" them. This is unsurprising. This isn't to defend unstructured interviews, other studies for a long time have shown them to be worse than structured interviews and test scores. If I had to guess the only reason the research in the article got published as novel was the random interview part. Edit: Here's a table from a meta-analysis of lots of studies on correlation between different factors and job performance: http://imgur.com/a/YRFTh http://imgur.com/a/YRFTh. Basically unstructured interviews aren't as good as structured ones, but they are better than nothing. Work samples are the best, structured interviews and IQ tests tie for second. Note that this meta-analysis is combining a bunch of different fields to yield general observations, a specific field may have different results. But in expectation for a randomly selected field these are fairly solid results, and I don't expect fields vary from them too much.
• douglasjsellers 10y ago
  All that this article says is that past performance, in terms of GPA, is the best performance of future performance - rather than a 30 minute interview predicting future performance. This sees like a basic truism to me and the main lesson that tech hiring processes can take away from this article. In my experience (having hired > 100 engineers) one of the basic problems that tech hiring, as a whole, has is that it misunderstands the point of a technical interview. Organizations and hiring managers see the interview process as a way of improving the brand of the engineering organization - "We have super high standards and to prove this our interview process is really hard - therefore if you think you meet these standards you should apply". This leads to the current interviewing trends of super academic/puzzle/esoteric technology based interviews. Applicants leave those interviews saying that it was super hard reinforcing the brand messaging (classic marketing). Rather, in my experience, the best results come from viewing the hiring/interviewing process for what it is - an attempt to predict future performance (and specifically performance at your organization) using a variety of techniques which interviewing is one. In this context, of attempting to predict future performance, interviews are not a great tool - better to look at specific past performance. Past performance is always the best predictor of future performance and the point of a technical interview, in my mind, is to critically inspect that past performance to understand how closely it relates to the future performance that your organization needs.
• donovanm 10y ago
  It certainly does feel like job interviews are a coin flip to me after having done many interviews.
• vonnik 10y ago
  The traditional recruiting and hiring process is broken. I say this as a former technical recruiter. I wrote about the problems of recruiting for a closed-source startup here: https://www.linkedin.com/in/chrisvnicholson/recent-activity/posts/ https://www.linkedin.com/in/chrisvnicholson/recent-activity/... I mention closed-source for a reason. For technical hiring, there is nothing better than open source. Open-source projects allow engineers and their potential employers to collaborate in depth over time. The company can experience whether the engineer is competent, reliable and friendly. The engineer can judge the team's merits in the same way. And they can both decide whether the fit is right. Closed-source and/or non-engineering jobs are the opposite. You get a resume, a Github repo if you're lucky, and a half-day's worth of interviews and tests. Then you roll the dice on that imperfect information. This is one reason why a lot of recruiting and hiring happens through the networks of people that a company can tap into. It may seem corrupt or nepotistic, but the advantage of those referrals is that someone with more information than you is willing to stake their reputation on a candidate's performance. Large companies with lots of historical data have the opportunity to train algorithms to learn how job applications and long-term performance/flight risk/etc. actually correlate. From what I can tell, most haven't.
  • kiranmova 10y ago
    Hiring candidates through interviews or through known contacts list is just about taking a chance. There are several unknowns which can only come out in the course of time. I agree with vonnik, that running the Project in Open Source definitely has an edge in the interviewing process. The potential employee has already seen your code (or you can make sure that he has), knows what he/she is getting into. I would seek for some links of their contributions to other projects which will help me evaluate if their code. Of-course less than 5% of the job applicants now have had the opportunity to develop for open source. (Either the folks contributing in the open source love their job so much, that they hardly look around or they are already being offered jobs without looking..).
• peterwwillis 10y ago
  No interview will tell you the future, so to my mind, the only thing the interview can tell you immediately is how much someone knows and whether their personality will mesh with the company's culture. In order to ascertain this, I propose job-hiring hackathons. Have the company hold a mini-hackathon, once every 2 weeks or once a month, where all job applicants must show up and work on projects (corporate employees' presence can be optional). Just watch them complete the projects and hire the best candidates.
• mck- 10y ago
  If you care about your company's culture, a person's humbleness, art of concise debating, etc - more important parameters than sheer GPA or coding skills imho - you can never do away with in-person interviews. Calling them "utterly useless" is an utter click-bait.
• nameisu 10y ago
  i have an interview with apple for a mechanical engineer. i will report back once its done.
  • pdimitar 10y ago
    Please do. I'll be watching for your response to this message.
• WalterBright 10y ago
  More accurately, the article is about the use of unstructured free-form interviews.
• exabrial 10y ago
  They're useless if it's attempt to prove you know more tham them about some algorithm that's been implemented 150x times (every job interview in California). I'd rather work with someone pleasent, hard working, and concerned with everyone's well being.
  • alfalfasprout 10y ago
    Hell, all I want is someone that could easily learn and understand an algo they're not already familiar with if needed. I remember being asked to basically implement the Day-Stout-Warren algorithm in an interview a few years ago and wondering what that person really learned about me from that memorization exercise.
• bahmboo 10y ago
  When interviewing for smaller orgs you do have to answer the question: can I work with this person everyday? Subjective and arguably harder to predict than technical performance, and oftentimes more important.
• deleted 10y ago
  [deleted]
• dba7dba 10y ago
  I believe a lot of places hire a candidate through a consensus, meaning some members in the team accept or reject a potential candidate. When enough accept the candidate, the hiring is done. Is there any company that tracks who rejects a particular candidate during an interview process, and how often that negative feedback turned out to be true. I guess with the turnover rate at todays tech places, such tracking of a record of an interviewer is not really possible? I always wonder about this.
• nebabyte 10y ago
  > not one interviewer reported noticing that he or she was conducting a random interview. More striking still, the students who conducted random interviews rated the degree to which they “got to know” the interviewee slightly higher on average Yeah well, when you're asking questions of someone who looks thoughtful very briefly, then answers almost immediately after, it sounds like you might have more reason to think you know them better than the one actually considering their answer Might be introducing a confound or two that you then proceed to completely ignore and even conclude past lest someone accidentally draw other conclusions
• RichardHeart 10y ago
  You hear lots of these stories about stupid interviews. You rarely hear the stories about the horrible, terrible employees that weren't weeded out, got hired and did great harm to the company and their coworkers. Interviews can be good and bad, I'd venture to say that many the horrible hire has been avoided by any interview at all. Thus, don't make perfect the enemy of good, and try to improve on good. The set of potential bad hires is vast compared to the good hires, and that ratio is only remedied by good filtering before and during the interview.
  • dragonwriter 10y ago
    > You rarely hear the stories about the horrible, terrible employees that weren't weeded out, got hired and did great harm to the company and their coworkers. Actually, stories about horrible co-workers that weren't weeded out by interview or any other hiring process are quite common workplace stories in every field, including tech.
  • flukus 10y ago
    We have entire websites dedicated to them: http://thedailywtf.com/ http://thedailywtf.com/ . If they did great harm to the company though, that's the companies fault for keeping them and/or not monitoring them.
• ppidugu 10y ago
  Such a careless claim and writing articles and throwing on peoples faces gives just a chance to everyone to debunk ny times articles
• andrewstuart 10y ago
  I am a recruiter. Recently, I started working with a new employer. We could not get anyone through their interview process. Eventually I asked the HR person to clarify precisely what was being asked in these interviews. She said that, essentially, the interviews were ad-hoc, with the interviewer just coming up with whatever questions they thought relevant based on the resume - often asking the candidate to go through their career history. I explained that the only effective approach I have found with recruiting is to have a set of pre-defined questions, and each question is specifically designed to give insight into how the candidate meets the pre-defined job requirements. Very much like software development, where test cases are related to software requirements. I explained also that it is not critical to stick precisely to these questions, but that should mostly be the case - interviews are human interactions and some flexibility is required depending on circumstance. The HR person then explained this to the hiring managers at the company, and worked with the hiring managers to define interview questions that give insight into the job requirements. The next two people interviewed got the jobs, after months of no one getting through the interviews. In the early days of software development, the business was often dissatisfied with software delivered because it simply did not meet the requirements of the business. So the software development process matured and came up with the idea of tests that can be mapped back to the requirements via a requirements traceability matrix. Thus the business has a requirement, the developers write code to meet the requirement, and a test is designed to verify that the software meets the defined requirement. Recruiting currently has no such general understanding in place of the relationship between job position requirements and definition of quantifiable questions that identify to what extent a given job candidate meets a requirement. Once you get your head around the idea that recruiting should be very similar to software development in this regard, then it is easy to see that ad-hoc interviews do nothing to verify in any organised way to what extent a candidate meets the requirements of a given job opening.
  • flukus 10y ago
    > I explained that the only effective approach I have found with recruiting is to have a set of pre-defined questions, and each question is specifically designed to give insight into how the candidate meets the pre-defined job requirements. Very much like software development, where test cases are related to software requirements. This creates a system that can be gamed though. The interviewees can pass this information back to the recruiter and the recruiter can couch future interviewees. The recruiter has a vested interest in placing people. It would be better to have a thousand questions that are randomly selected.
• woodandsteel 10y ago
  My basic problem with interviewing is you are observing behavior in one sort of situation, and on that basis trying to predict behavior in a very different sort of situation, namely job performance, which is actually a whole bundle of different types of situations. It seems like it would be much better to instead put the job prospect in situations that model the sorts that would come up at work.
• JacksonGariety 10y ago
  Does it bother anyone else that the example given in the article (showing up 25 minutes late) is judging interviewing by its worst rather than its best?
• cordite 10y ago
  I'm sure it has been discussed before, but what factors push people away from say two-day internships? Is it because these things are so short that they too can eventually be gamed? Or is it because you need to have staff of the same specialty dedicating their resources to a potentially short-lived investment?
• dlwdlw 10y ago
  The issue with silicon valley interviews is that it's leading a new paradigm of management styles that deal with knowledge and creative work This shift MUST be accepted by everybody ornostracism is risked. (Like trump supporters) But paradigm shifts take time and the majority of managers still want cogs. But instead of filtering dor cogs they have to dress the filters up as filtering for a "i give smart people freedom team" and the convoluted mental gymnastics needed for this creates shitty interview processes. All "well this technique worked for us" stories are mostly Not useful because tehy are just N=1 stories about managers using their preferred filters. The issue isn't with the filters themselves (all sorts exist) but with a culture that obligates everyone to out on false facades. People who just want to be paid and can work well need to pretend to be passionate. Managers who want well-paid cogs need to pretend to promote individualistic thinking etc...
• jasonthevillain 10y ago
  Well, the worst interview I've ever had was the ones where the interviewer wouldn't deviate from the script, even after he recognized that these questions made no sense for someone with my background (I was self-taught, and had never managed my own memory or written a sort algorithm. They were also irrelevant to the position in question). It was painfully awkward. It was also a fantastic way to accidentally discriminate against women and older candidates. I'm not saying anyone should conduct an interview completely by the seat of their pants, but please don't encourage this foolish consistency.
• tempodox 10y ago
  > So great is people’s confidence in their ability to glean valuable information from a face to face conversation that they feel they can do so even if they know they are not being dealt with squarely. But they are wrong. If people utterly refuse to learn from proven mistakes, then all hope is lost. Einstein was right, human stupidity is infinite.
• geebee 10y ago
  I'm coming late to this discussion, but there is one point I'd like to make. Our "interviews" in the world of software engineering, are immensely different from "interviews" in the standard sense of the word. I've worked in different fields, and I talk to people who work in other fields. Most of those fields work in a way that is described in this article - interviews are question and answer sessions, where people are evaluated by a number of highly subjective criteria. "Tell me about your fundraising experience?" "How do you deal with difficult clients or coworkers". That kind of thing. Software interviews are exams. They're not "more like" exams, they are flat out exams. There is very little banter. The closest I've come in google and Netflix interviews has been the more open-ended system design style question they often put in there, but even that has an academic test quality to it. It's pretty much 5 hours of technical exam. "How do you find all matching subtrees in a binary tree" might be a question - and you really are expected to get it written at the whiteboard. "Find all permutations of a set". "Find all square sub matrices in an NXM matrix." The "top" companies are good at modifying the question so that you must know how to do this but can't just regurgitate it. Alternatively, you may do a "take home" exam. Most recently, I did a mini rails project. I actually liked my result, I kind of enjoyed writing it. However, it was a no-hire, one reason given was that my routing was non-standard. True. I hadn't really thought about it, it was a take-home, so I mainly focused on the UI and code, and just chucked in a couple of named routes for demo and testing purposes. The other reason was that there was some duplicate code (I disagreed and had a reason for this, but there is no chance to defend your code, you write it and send it in, and they say "no hire"). I have no idea if it was a real piece of crap and they were just being nice. It had 100% test coverage and git for version control, and implemented a few features. Unfortunately, like I said, I never got a chance to defend the code. Our processes in high tech are badly broken. I'm probably done interviewing, my next job will have to be one that doesn't involve a software interview. The routing and duplicate code, along with a google interview, pretty much sealed the deal for me. My advice to people is (this isn't my idea) be an X that programs, not an X programmer. Coding is an amazing tool for a job, but avoid making it your job. For instance, I actually know a fundraiser who does a lot of data science, and he's a rock star in his field, but I guarantee you nobody asks him to reverse a binary tree in an interview! Best of luck out there. Our interviewing processes are their own special version of horridness, just not uselessness described here.
• tdelet 10y ago
  The problem is that most people have no training in how exactly to interview someone. It's like asking people with no training in building rockets to get you to the Moon and then deciding that because because they failed tickets are "utterly useless".