15 ms·

People are still bad at gauging their own interview performance

158 points by nns 10y ago

RyanCavanaugh 10y ago
I would draw the opposite conclusion from that data. The mode difference was 0, and 91% of people were within 1 star of the interviewer. Without knowing what the interviewer's bar is for any particular star level, I don't see how the interviewees could do any better. Rounding to 1-star increments amplifies relatively small changes -- if the interviewer rounds up 3.6 to 4 and you round 3.4 down to 3.
- searine 10y ago
  "You probably interviewed as bad as you think you did" is not a catchy clickbait title.
- Retric 10y ago
  +/- 1 star is a huge range. That's 50th to 90th percentile.
- throw_away_777 10y ago
  Yeah, the way the data was presented was not informative. The heat maps are interesting to look at, but don't really convey the information well - the colors are basically useless. You have to hover over everything, at which point it should be presented as a table or a better picture. It is pretty clear from one of the plots that interviewers and interviewees grade on a different curve - the mean interviewee scores themselves lower than the mean interviewer does. But once you correct for this, it seems that people are actually pretty good judges of how well they did. I'm honestly surprised by how low the R-squared was, I wonder if they included an intercept?
- nck4222 10y ago
  There's also the possibility that interviewees are better at rating themselves than interviewers. For the purpose of this article, interviewer ratings are considered the absolute truth while interviewee ratings are not, which is obviously not always the case in reality.
p4wnc6 10y ago
Maybe I'm missing something, but why isn't the implication going the other direction? In my experience, the problem is that interviewers have no idea how to correctly value a candidate's performance. Maybe the candidates are closer to being well-calibrated, but their self-assessments don't match up with the interviewers' because the interviewers don't know how to gauge what they are looking for? Making the assumption that an interviewer knows how to measure the response of a candidate, even in cases of extremely quantitative questions with well-defined answers, is highly suspect to me. I think virtually no one knows how to do that effectively.
- GoToRO 10y ago
  :)) Just the other day a manager said "HR is overwhelmed. They have a student that filters the CVs and she is doing all she can." The senior programmer exploded "How is a student any good at filtering programmers, she does not know anything about programming!".
  - hnal943 10y ago
    Based on your comments in this thread, it sounds like you may need a new job.
    - GoToRO 10y ago
      I already have it. Not a long time ago I was searching for a job and this is how I learned all these things. Because I always learn :). Strange experience.
- rifung 10y ago
  If you consider how well someone does as how likely they are to be hired, then it makes sense that interviewers are the source of "truth", because it's their grade or opinion which will decide whether you get hired or not. We certainly can speculate whether whose opinions are more correct or valid, but if we just objectively consider that doing perfectly means you get hired, then what the interviewer thinks matters and what the interviewee thinks doesn't. I do think you raise an interesting question though. I do wonder whether, in a scenario where many interviewers saw the same interview performance, how varied their scores would be. After all, most people fail at many job interviews before landing one they get, but is that because of variation in the performance of the interviewee or because of the differences in interviewers?
andrewstuart2 10y ago
In related news, "People are still bad at knowing the unknown" and "Everybody thinks they're average; perhaps because their sample size can only be 1."
- EliRivers 10y ago
  "Everybody thinks they're average; perhaps because their sample size can only be 1." It's my understanding that this is not the case. People typically think they're above average. https://en.wikipedia.org/wiki/Illusory_superiority https://en.wikipedia.org/wiki/Illusory_superiority http://www.livescience.com/26914-why-we-are-all-above-average.html http://www.livescience.com/26914-why-we-are-all-above-averag...
GoToRO 10y ago
I often wonder how would these people hire soldiers? "So I see you have no previous experience in killing people... We are looking for people that have at least 5 years of experience in killing people." My point is you need smart enough people for your job and nothing more. You do an IQ test, a personality one and that's all. The rest can be learned.
- ng12 10y ago
  There's a line there somewhere. Hiring someone for knowing Node is pretty silly, but hiring someone who's spent 5 years developing web apps is a useful metric.
  - GoToRO 10y ago
    In my experience, not that much. The 5 years experience might mean one year repeated 5 times. The person might be bored but he has no choice but to continue with web development and so on. Get somebody who is interested and smart enough for the job. Note I didn't say "the smartest".
    - deleted 10y ago
      [deleted]
    - red_blobs 10y ago
      5 years of 1 year experience repeated 5 times is better than someone that will introduce spaghetti code and security flaws simply because they've never experienced it. It takes years to figure out patterns (when to use and not use them), good coding practices, avoiding newbie mistakes like sql injection and xss and how to actually make a deadline (its not easy to finish a project when you also have to learn the language and other skills in addition to the task at hand). As a business owner, I don't want someone learning the above on the job and on my dime. There is a company in MI that tried this called Compuware back in the 90s. It was a complete and utter disaster.
- civilian 10y ago
  I think the personalty trait we call "grit" or "stick-to-it-ive-ness" is hard to test for. And I think the best way to test for it is by looking at someone's track record. And it's one of the big three requirements. (which are: smart, works hard, someone you want to work with. )
fecak 10y ago
>some portion of interviewees are losing interest in joining your company just because they didn’t think they did well, despite the fact that they actually did. This is the key takeaway for me. This is IMO and experience probably based on a "sour grapes" type defense mechanism, where candidates will internally try to talk themselves out of wanting a job that they felt they didn't get. They walk out of the interview feeling they didn't perform, and instead of regretting the lost opportunity they start to focus on even the most minute "negatives" about the job/company. The more time you give a candidate to stew about those reasons they don't want the job, the more time they'll have to find reasons that may not even exist. Positive feedback should be given almost instantly. Some companies and candidates are reluctant to give positive feedback quickly because they feel it may hurt their negotiation leverage. I don't have data on this, but anecdotally (over almost 20 years in recruiting) I've seen this countless times while collecting post-interview feedback from candidates. The ones that feel they did poorly are likely to mention a negative about the interviewer or company, which becomes somewhat awkward when we come back with a job offer.
- Yhippa 10y ago
  I absolutely judge a company by the interview process and the questions they ask. "Would I want to work for a company where the employees pass their given process?"
  - fecak 10y ago
    >I absolutely judge a company by the interview process and the questions they ask. And rightfully so. The term these days is "candidate experience" for the overall interview process, which can include everything from a company's handling of logistics and time spent in a waiting room to the types of questions being asked. That said, what I'm referring to in my comment isn't necessarily related to the interview process at all. A candidate can be asked questions that he/she felt were relevant and entirely fair for the job at hand, yet if he/she performed poorly there is still a tendency to try and find flaws in the overall opportunity. My reference wasn't to candidates performing poorly and thus giving a bad review to the interview process. Rather, the candidate might perform poorly and say that the vacation time was inadequate or the commute would have been too long.
sushid 10y ago
> Therefore, every bar above 0 is impostor syndrome country, and every bar below zero belongs to its foulsome, overconfident cousin, the Dunning-Kruger effect. Dunning-Kruger effect is attributed to "a metacognitive inability of the unskilled to recognize their ineptitude." [1] Seems like a harsh accusation for being off by a single star. And the author makes a generalization that the imposter effect is "better" than Dunning-Kruger effect. I don't clearly see why this would be the case for the interviewee.
- sushisource 10y ago
  I'm so mind-bendingly sick of these two terms (Dunning-Kruger and Imposter Syndrome). They're very much in-vouge as of late and it seems like an overwhelming number of /r/programming and HN posts have at least some reference to it. Pulling some piece of psychology jargon out of thin air doesn't make it immediately applicable or accurate. Presumably, the terms get used so frequently for the "look how smart I sound" effect, which serves only to piss me off. I would categorize these two groups as "overconfident" and "under-confident", at most. They're not suffering from some genuine long term psychological condition just because you think it sounds cool to use the terms. /rant
  - Myrmornis 10y ago
    Yep. You can pretty much tell when people use those terms that they're more driven by a desire to use those terms than out of having anything interesting to say.
    - AstralStorm 10y ago
      Isn't a single instance not indicative of a syndrome? The syndrome presence or not can only be inferred if the same candidate heard multiple times on various interviews is consistently under- or overvaluing. And by a noticeable amount too.
EdiX 10y ago
Mmmhhh I would have drawn the opposite conclusion, most people estimate ±1 star away from the real result, it's not bad considering the actual performance is not entirely objective. Glad to see that once again Dunning-Krüger proved wrong.
- tom_mellior 10y ago
  > not bad considering the actual performance is not entirely objective It's not even clear (to me) what's being compared here. What's the "actual performance" that the blog post very carefully does not define? The screenshot highlighting the technical performance question seems to suggest that it might be only that one point out of the three that interviewers fill in. But it's completely possible to be technically competent and still feel that one didn't do well overall because one came across as nervous or boring or whatever. In other words, the comparison here seems to be between apples and undefined oranges.
Kephael 10y ago
These interview training programs seems to be really reminiscent of coding bootcamps and appear to have grown rapidly in a very short period of time. I think I'd almost prefer interviewees being judged based on the schools they attended and companies they worked at. I'm not sure that I find the ability to cram for data structures and algorithms questions to be indicative of intelligence and problem solving skills. Schools attended and companies worked at could be a very strong potential signal of applicant performance and not involve several rounds of "implement that algorithm from memory". Are interviews becoming more difficult as a result of the proliferation of all the interview test prep material?
- askAwayMan 10y ago
  Maybe if you've been fortunate to have attended an elite school. For the rest of us no name state school alumns, there's the nice illusion of meritocracy either way.
- kristopolous 10y ago
  Algorithms and elite schools doesn't tell me whether you'll use the right-sized solution for a problem and be able to prioritize and focus on tasks accordingly. It doesn't tell me if you can perceive and extract requirements well and have a process of communication and a professionalism of management in how you go about getting to solutions that work for the humans that will be using it. It also doesn't tell me about your personal value system and whether you are passionate about creating good user experiences or if you are just interested in a technical playground or if you are a social climber or what. It also doesn't tell me if you are highly opinionated and want to do things your own way or if you are flexible enough to deal with the imperfections of the real world. I don't know whether you value the mantra that the perfect shouldn't be the enemy of the good. No, none of these brain teasers or abstract bootcamp style things tell me that - and those are the actual things that will make or break a company and determine whether your product will hit the market on time. Those are the things that will get you a customer base and will create a company culture of productivity and not something that doesn't lead to results. Until interviews become more about how people approach work and their ethic and process of conducting it (which is a major part of classical engineering), these are ineffective tools for finding the right people. --- [1] although as a disclaimer, I strongly prefer companies of under 20 employees. If you have 200, then the employees can be more insulated from the exposure to these types of risk and the necessity of this type of vertical thinking. But it's an "insulation" - everyone is working either directly or tangentially on something that is intended to be used by something in the world (usually humans).
Sleaker 10y ago
>some portion of interviewees are losing interest in joining your company just because they didn’t think they did well, despite the fact that they actually did. Maybe this is because the interviewee feels partly abused, as if communication isn't a two-way street. I think interviews would be less terrible if interviewers actually told you at the end that you did above/aswell as/below expectations, and/or why instead of just waiting 1-2 weeks before getting back to you and then telling you that you are/aren't moving on to the next step. A huge part of the interview process produces emotional ups/downs that no one wants to deal with and which ostracize potential employees, but the interviewers don't really do much to help with this. I also think that having a 1-4 range is not a good guide for gathering sweeping datasets. Adding any amount of numbers will help to show actually how much. Or actually using human language to define them would be better. 1) Terrible/Unhirable. 2) Unhirable/Not Passing 3) Passing 4) Exemplar would actually be beneficial right now arbitrary numbering doesn't help to give people much context as to what they should choose, and the numbers are so close that you're going to get outliers where maybe some people think that a 2 is passable but not great, and interviewer thinks 3 is passable but not great, or vice-versa.
- fecak 10y ago
  >I think interviews would be less terrible if interviewers actually told you at the end that you did above/as well as/below expectations This happens quite often in phone interviews. My clients have regularly told candidates "We'll be inviting you in" at the end of a phone interview. The issue with this for on-site interviews is that they tend to be with multiple individuals/groups, and getting those groups together to discuss the candidate's performance while the candidate is still on-site is a mess logistically. It would be ideal, but usually there will need to be some discussions among the panels.
  - bcbrown 10y ago
    > usually there will need to be some discussions among the panels. The way we've handled that (startup, ~30 devs) is that the VP of Engineering goes last, and will go around getting quick reactions from everyone on the loop. If everyone says "I'm all for hiring", the VP will go ahead and let the candidate know we'll be making an offer. If anyone says "I'd like to discuss", then we don't give feedback until after getting together to discuss the candidate.
collyw 10y ago
Surely a lot depends timing and circumstances. One interviewer said they gave me the job based on asking if the guys accent was Dutch (he was Norwegian and found that hilarious). It was a maintenance programming job and had a high turnover of staff. Other interviews you may have done well but had some serious competition.
Fricken 10y ago
I've never interviewed for a job I didn't get. Possibly because I aim too low. But if I were to present myself honestly as a potential employee, I would enter the interview without greeting or making eye contact, then I would sit as far away from the interviewer as possible and doodle in a notebook for 15 minutes before raising my head and explaining everything I think is wrong about the company.
kafkaesq 10y ago
What a hatchet job. Unfortunately we don't have the raw data available; but all we can infer from these heatmaps is that yes (and exactly as one would expect) there's a bit of variance between between the two sets of ratings (in both directions). Maybe a bit more than can be explained by the sample size, or from the simple fact people (on both sides of the process) are forced to discretize their assessments (so even if both parties were to objectively agree that the "real" performance was in the 2.4-2.7 range, which it is a good chunk of the time.... inevitably you'll find them be off by +/- 1 in their quantile assignments a good chunk of the time -- which is exactly what the data seem to show). In particular: independent of how we explain the variance (whether due to sampling effects, or bona fide DK/IS) note that the vast majority of the variance is in the +/- 1 range. And on the alleged DK side (-2 variance) the measure is apparently quite miniscule. However, that's not the original D-K effect. What D-K specifically predicted (and were able to show with much better data and methodology -- in their very specific sample population, at least) is that, among the general population there will be a striking degree of -2 variance, specifically in the bottom quartile. Going by the oft-quoted abstract from their original paper: Across 4 studies, the authors found that participants scoring in the bottom quartile on tests of humor, grammar, and logic grossly overestimated their test performance and ability. Although their test scores put them in the 12th percentile, they estimated themselves to be in the 62nd. In other words: "Performers in the bottom quartile estimated themselves to be performing, on average, at mean performance in the third quartile." So if were to look for something analogous to a D-K effect, we'd expect to find a "hot spot" (or at least the strongest amount of red ink, for that quartile) in the upper left of the first chart. Instead we find most performers in the bottom quartile (correctly) identifying themselves as in the bottom quartile; with a smaller portion bleeding up into the second quarter; and an an almost negligible portion placing themselves in the D-K zone -- namely, the third quarter. If anything what the data show is that (even assuming there's nothing to contend with in how the actual data were taken), while inevitably there's some variance (both ways) in the two sets of rankings, and probably some of due to distorted self-perception -- on the whole there's actually much less of a D-K effect among engineers than among the general population (or rather, "Cornell undergraduates taking psychology classes" -- which, lest we forget, were the actual subject class used in the original D-K study). So basically the opposite of what the author of the original article is claiming to infer from this data.
Ologn 10y ago
After having interviewed dozens of people, the idea occurred to me that my ratings for interviewees were a Gaussian curve with a normal distribution. Most people blended into each other - they knew the topics the same amount. Enough to be employed in their current job. Then one in six was a standard deviation below or worse, and one in six was a standard deviation above or better. That might be one reason for the impostor syndrome. Because if we interviewed six people, one would be unqualified, four would be OK but not spectacular, and one would know the subjects better than anyone. So in an interview of six people the best qualified one of the six would have the job. The average four were not truly impostors, but would not get the job unless they had an inside recommendation or if we were really needed someone and there was a shortage of candidates coming in.
merb 10y ago
That there still needs to be discussions about that topic. On a workplace people should be treated equally. No matter how many topics will be brought up. A world with different genders, stereotypes, whatever can only work if both sides won't interpret too much into it and just enjoy their work with their colleagues. Actually we don't employ a woman (I'm working at a 3 person company). And we actually hire, but the only woman will probably decline on their side, she actually has experience in things we actually not need yet, so I guess we can't provide her a new challenge. way probably the best application.
delphinius81 10y ago
Once you have an engineering job, a typical project tasking involves: some team design discussion, some engineering research, some work, some review, more work. The requirements are either known upfront, or determined as a team. I do not understand why interviews are not conducted in this manner. Pick some arbitrary idea and have the candidate work with the interviewer to design something, then let the candidate do a little implementation research/design, review with the next interviewer, wipeboard code and review some part of the project with next reviewer. Have a coffee break at some point to get to know the person. Treat the interview process like a normal day at work.
- jomamaxx 10y ago
  "I do not understand why interviews are not conducted in this manner. " It truly is crazy.
- dogma1138 10y ago
  That's a good idea but logistically is hard, both requiring the interviewer to spend considerable time on site as well as costing your own employee's time. It might work well for a small company especially with a small number of candidates but it doesn't really scale, unless you are a really big company (googleish size) that can dedicate so many resources to the interviewing process. Doing design and research takes time, you can't do it in 30 min, you can't also ask someone to spend 8 hours interviewing, and if you have 10 candidates you can't spend half a man month or more on interviewing them either. I've seen some companies that "interview" by contracting candidates for 1-3 days of work and paying them for it, but it really only works on positions where you hiring effectively interns/fresh graduates, but as most candidates especially those who end up in the face to face phase are currently employed you can't really do that.
  - delphinius81 10y ago
    I dunno, I've personally spent on average half a day (3-4 hours) with on-site interviews as a candidate. When on the other side of the table, it's been about the same (~3 hours). With a tractable enough problem, I think you can fit it into a half-day. You could also begin the design/research process as part of a phone screen, have the candidate further prepare on their own time, and then present/discuss as part of the on-site, along with the more in-depth technical stuff. After you've run this process enough times, your interviewers will have a good grasp on the various solutions and technical problems. Most people's training on how to interview comes from how they themselves were interviewed - which was probably poorly done. Improve the interview process and you can train people to be better interviewers just by being interviewed. :)
mark-r 10y ago
Don't miss the footnote: "I’m always terrified of misspelling “Dunning-Kruger” and not double-checking it because of overconfidence in my own spelling abilities."
Apocryphon 10y ago
I understand companies are unwilling to give negative feedback to applicants for fear of legal repercussion. But often not getting any form of feedback is highly detrimental to an applicant who's invested days if not weeks of going through the process, only to receive a boilerplate "thank you for applying, please try again" response. One wonders if it's possible to devise some sort of neutral, yet helpful, assessment that could be given to applicants without fear of legal retaliation. Or, if eventually an applicant will sue a company for not providing feedback after they were rejected.
matmann2001 10y ago
Probably because interviewing companies have stopped giving performance feedback.
nfriedly 10y ago
I used to do a lot of interviews for front-end engineers. The process I settled on was to give them the following setup: - some mock-ups from when we first designed our product gallery - a json file that listed the products names, pictures, description, etc. - a blank css file - a html file that loaded the css and jQuery, and then made an ajax request to load the json file. (This was back when jQuery was pretty much state of the art for front-end.) With all of that in place, I'd explain that I'd like them to work on building a product gallery that looked similar to the mockups. I would further explain that, since we only had an hour, I didn't expect a polished or even fully functional product, but I'd just like them to dive in and see how far they got. They could use their editor of choice and ask any questions they wanted. I felt like this gave better insight to candidates skill and workflow. My colleagues all gave more traditional interviews, so I also felt like there was a good counterbalance if someone didn't do well with this kind of interview.
- xxbondsxx 10y ago
  Doesn't that heavily bias towards candidates who have used jQuery before and/or built a product gallery before? Skewing interview performance based on some specific previous experience might be what you want if you expect your candidates to hit the ground running, but might not be the best to gauge long-term performance.
  - nfriedly 10y ago
    This was back in the day when everyone used jQuery for everything, and the company was only looking for experienced people, so a candidate who hadn't used jQuery before probably wouldn't have been a good match for the position anyways. I don't think a single candidate had that issue though. As far as the product gallery goes, I wasn't looking for a pixel-perfect product. Just spit out a bit of html and matching CSS that look vaguely like the mockup. The mockup images had things like the hex values of the colors written on the image to make that part easy. Basically, it was trying to focus on things that would be the main part of the candidate's job there. The folks that we did hire all did well on both that interview and long-term at the company.
ethicalconsult 10y ago
4 For University Grades changing +Erase Criminal Records hack +Driver License +Facebook hack, Twitter hack, Instagram hack, Gmail, Yahoo hack +Already hacked Paypal, MasterCard, Credit Card untraceable balance +Email interception get a hold of consultanthackers@outlook.com , certified , professional and highly time conscious... call (302) 365-0294? and its done !

158 points by nns 10y ago

I would draw the opposite conclusion from that data. The mode difference was 0, and 91% of people were within 1 star of the interviewer. Without knowing what the interviewer's bar is for any particular star level, I don't see how the interviewees could do any better. Rounding to 1-star increments amplifies relatively small changes -- if the interviewer rounds up 3.6 to 4 and you round 3.4 down to 3.

searine 10y ago
"You probably interviewed as bad as you think you did" is not a catchy clickbait title.
Retric 10y ago
+/- 1 star is a huge range. That's 50th to 90th percentile.
throw_away_777 10y ago
Yeah, the way the data was presented was not informative. The heat maps are interesting to look at, but don't really convey the information well - the colors are basically useless. You have to hover over everything, at which point it should be presented as a table or a better picture. It is pretty clear from one of the plots that interviewers and interviewees grade on a different curve - the mean interviewee scores themselves lower than the mean interviewer does. But once you correct for this, it seems that people are actually pretty good judges of how well they did. I'm honestly surprised by how low the R-squared was, I wonder if they included an intercept?
nck4222 10y ago
There's also the possibility that interviewees are better at rating themselves than interviewers. For the purpose of this article, interviewer ratings are considered the absolute truth while interviewee ratings are not, which is obviously not always the case in reality.

p4wnc6 10y ago

Maybe I'm missing something, but why isn't the implication going the other direction? In my experience, the problem is that interviewers have no idea how to correctly value a candidate's performance. Maybe the candidates are closer to being well-calibrated, but their self-assessments don't match up with the interviewers' because the interviewers don't know how to gauge what they are looking for? Making the assumption that an interviewer knows how to measure the response of a candidate, even in cases of extremely quantitative questions with well-defined answers, is highly suspect to me. I think virtually no one knows how to do that effectively.

GoToRO 10y ago
:)) Just the other day a manager said "HR is overwhelmed. They have a student that filters the CVs and she is doing all she can." The senior programmer exploded "How is a student any good at filtering programmers, she does not know anything about programming!".
- hnal943 10y ago
  Based on your comments in this thread, it sounds like you may need a new job.
  - GoToRO 10y ago
    I already have it. Not a long time ago I was searching for a job and this is how I learned all these things. Because I always learn :). Strange experience.
rifung 10y ago
If you consider how well someone does as how likely they are to be hired, then it makes sense that interviewers are the source of "truth", because it's their grade or opinion which will decide whether you get hired or not. We certainly can speculate whether whose opinions are more correct or valid, but if we just objectively consider that doing perfectly means you get hired, then what the interviewer thinks matters and what the interviewee thinks doesn't. I do think you raise an interesting question though. I do wonder whether, in a scenario where many interviewers saw the same interview performance, how varied their scores would be. After all, most people fail at many job interviews before landing one they get, but is that because of variation in the performance of the interviewee or because of the differences in interviewers?

andrewstuart2 10y ago

In related news, "People are still bad at knowing the unknown" and "Everybody thinks they're average; perhaps because their sample size can only be 1."

EliRivers 10y ago
"Everybody thinks they're average; perhaps because their sample size can only be 1." It's my understanding that this is not the case. People typically think they're above average. https://en.wikipedia.org/wiki/Illusory_superiority https://en.wikipedia.org/wiki/Illusory_superiority http://www.livescience.com/26914-why-we-are-all-above-average.html http://www.livescience.com/26914-why-we-are-all-above-averag...

GoToRO 10y ago

I often wonder how would these people hire soldiers? "So I see you have no previous experience in killing people... We are looking for people that have at least 5 years of experience in killing people." My point is you need smart enough people for your job and nothing more. You do an IQ test, a personality one and that's all. The rest can be learned.

ng12 10y ago
There's a line there somewhere. Hiring someone for knowing Node is pretty silly, but hiring someone who's spent 5 years developing web apps is a useful metric.
- GoToRO 10y ago
  In my experience, not that much. The 5 years experience might mean one year repeated 5 times. The person might be bored but he has no choice but to continue with web development and so on. Get somebody who is interested and smart enough for the job. Note I didn't say "the smartest".
  - deleted 10y ago
    [deleted]
  - red_blobs 10y ago
    5 years of 1 year experience repeated 5 times is better than someone that will introduce spaghetti code and security flaws simply because they've never experienced it. It takes years to figure out patterns (when to use and not use them), good coding practices, avoiding newbie mistakes like sql injection and xss and how to actually make a deadline (its not easy to finish a project when you also have to learn the language and other skills in addition to the task at hand). As a business owner, I don't want someone learning the above on the job and on my dime. There is a company in MI that tried this called Compuware back in the 90s. It was a complete and utter disaster.
civilian 10y ago
I think the personalty trait we call "grit" or "stick-to-it-ive-ness" is hard to test for. And I think the best way to test for it is by looking at someone's track record. And it's one of the big three requirements. (which are: smart, works hard, someone you want to work with. )

fecak 10y ago

>some portion of interviewees are losing interest in joining your company just because they didn’t think they did well, despite the fact that they actually did. This is the key takeaway for me. This is IMO and experience probably based on a "sour grapes" type defense mechanism, where candidates will internally try to talk themselves out of wanting a job that they felt they didn't get. They walk out of the interview feeling they didn't perform, and instead of regretting the lost opportunity they start to focus on even the most minute "negatives" about the job/company. The more time you give a candidate to stew about those reasons they don't want the job, the more time they'll have to find reasons that may not even exist. Positive feedback should be given almost instantly. Some companies and candidates are reluctant to give positive feedback quickly because they feel it may hurt their negotiation leverage. I don't have data on this, but anecdotally (over almost 20 years in recruiting) I've seen this countless times while collecting post-interview feedback from candidates. The ones that feel they did poorly are likely to mention a negative about the interviewer or company, which becomes somewhat awkward when we come back with a job offer.

Yhippa 10y ago
I absolutely judge a company by the interview process and the questions they ask. "Would I want to work for a company where the employees pass their given process?"
- fecak 10y ago
  >I absolutely judge a company by the interview process and the questions they ask. And rightfully so. The term these days is "candidate experience" for the overall interview process, which can include everything from a company's handling of logistics and time spent in a waiting room to the types of questions being asked. That said, what I'm referring to in my comment isn't necessarily related to the interview process at all. A candidate can be asked questions that he/she felt were relevant and entirely fair for the job at hand, yet if he/she performed poorly there is still a tendency to try and find flaws in the overall opportunity. My reference wasn't to candidates performing poorly and thus giving a bad review to the interview process. Rather, the candidate might perform poorly and say that the vacation time was inadequate or the commute would have been too long.

sushid 10y ago

> Therefore, every bar above 0 is impostor syndrome country, and every bar below zero belongs to its foulsome, overconfident cousin, the Dunning-Kruger effect. Dunning-Kruger effect is attributed to "a metacognitive inability of the unskilled to recognize their ineptitude." [1] Seems like a harsh accusation for being off by a single star. And the author makes a generalization that the imposter effect is "better" than Dunning-Kruger effect. I don't clearly see why this would be the case for the interviewee.

sushisource 10y ago
I'm so mind-bendingly sick of these two terms (Dunning-Kruger and Imposter Syndrome). They're very much in-vouge as of late and it seems like an overwhelming number of /r/programming and HN posts have at least some reference to it. Pulling some piece of psychology jargon out of thin air doesn't make it immediately applicable or accurate. Presumably, the terms get used so frequently for the "look how smart I sound" effect, which serves only to piss me off. I would categorize these two groups as "overconfident" and "under-confident", at most. They're not suffering from some genuine long term psychological condition just because you think it sounds cool to use the terms. /rant
- Myrmornis 10y ago
  Yep. You can pretty much tell when people use those terms that they're more driven by a desire to use those terms than out of having anything interesting to say.
  - AstralStorm 10y ago
    Isn't a single instance not indicative of a syndrome? The syndrome presence or not can only be inferred if the same candidate heard multiple times on various interviews is consistently under- or overvaluing. And by a noticeable amount too.

EdiX 10y ago

Mmmhhh I would have drawn the opposite conclusion, most people estimate ±1 star away from the real result, it's not bad considering the actual performance is not entirely objective. Glad to see that once again Dunning-Krüger proved wrong.

tom_mellior 10y ago
> not bad considering the actual performance is not entirely objective It's not even clear (to me) what's being compared here. What's the "actual performance" that the blog post very carefully does not define? The screenshot highlighting the technical performance question seems to suggest that it might be only that one point out of the three that interviewers fill in. But it's completely possible to be technically competent and still feel that one didn't do well overall because one came across as nervous or boring or whatever. In other words, the comparison here seems to be between apples and undefined oranges.

Kephael 10y ago

These interview training programs seems to be really reminiscent of coding bootcamps and appear to have grown rapidly in a very short period of time. I think I'd almost prefer interviewees being judged based on the schools they attended and companies they worked at. I'm not sure that I find the ability to cram for data structures and algorithms questions to be indicative of intelligence and problem solving skills. Schools attended and companies worked at could be a very strong potential signal of applicant performance and not involve several rounds of "implement that algorithm from memory". Are interviews becoming more difficult as a result of the proliferation of all the interview test prep material?

askAwayMan 10y ago
Maybe if you've been fortunate to have attended an elite school. For the rest of us no name state school alumns, there's the nice illusion of meritocracy either way.
kristopolous 10y ago
Algorithms and elite schools doesn't tell me whether you'll use the right-sized solution for a problem and be able to prioritize and focus on tasks accordingly. It doesn't tell me if you can perceive and extract requirements well and have a process of communication and a professionalism of management in how you go about getting to solutions that work for the humans that will be using it. It also doesn't tell me about your personal value system and whether you are passionate about creating good user experiences or if you are just interested in a technical playground or if you are a social climber or what. It also doesn't tell me if you are highly opinionated and want to do things your own way or if you are flexible enough to deal with the imperfections of the real world. I don't know whether you value the mantra that the perfect shouldn't be the enemy of the good. No, none of these brain teasers or abstract bootcamp style things tell me that - and those are the actual things that will make or break a company and determine whether your product will hit the market on time. Those are the things that will get you a customer base and will create a company culture of productivity and not something that doesn't lead to results. Until interviews become more about how people approach work and their ethic and process of conducting it (which is a major part of classical engineering), these are ineffective tools for finding the right people. --- [1] although as a disclaimer, I strongly prefer companies of under 20 employees. If you have 200, then the employees can be more insulated from the exposure to these types of risk and the necessity of this type of vertical thinking. But it's an "insulation" - everyone is working either directly or tangentially on something that is intended to be used by something in the world (usually humans).

Sleaker 10y ago

>some portion of interviewees are losing interest in joining your company just because they didn’t think they did well, despite the fact that they actually did. Maybe this is because the interviewee feels partly abused, as if communication isn't a two-way street. I think interviews would be less terrible if interviewers actually told you at the end that you did above/aswell as/below expectations, and/or why instead of just waiting 1-2 weeks before getting back to you and then telling you that you are/aren't moving on to the next step. A huge part of the interview process produces emotional ups/downs that no one wants to deal with and which ostracize potential employees, but the interviewers don't really do much to help with this. I also think that having a 1-4 range is not a good guide for gathering sweeping datasets. Adding any amount of numbers will help to show actually how much. Or actually using human language to define them would be better. 1) Terrible/Unhirable. 2) Unhirable/Not Passing 3) Passing 4) Exemplar would actually be beneficial right now arbitrary numbering doesn't help to give people much context as to what they should choose, and the numbers are so close that you're going to get outliers where maybe some people think that a 2 is passable but not great, and interviewer thinks 3 is passable but not great, or vice-versa.

fecak 10y ago
>I think interviews would be less terrible if interviewers actually told you at the end that you did above/as well as/below expectations This happens quite often in phone interviews. My clients have regularly told candidates "We'll be inviting you in" at the end of a phone interview. The issue with this for on-site interviews is that they tend to be with multiple individuals/groups, and getting those groups together to discuss the candidate's performance while the candidate is still on-site is a mess logistically. It would be ideal, but usually there will need to be some discussions among the panels.
- bcbrown 10y ago
  > usually there will need to be some discussions among the panels. The way we've handled that (startup, ~30 devs) is that the VP of Engineering goes last, and will go around getting quick reactions from everyone on the loop. If everyone says "I'm all for hiring", the VP will go ahead and let the candidate know we'll be making an offer. If anyone says "I'd like to discuss", then we don't give feedback until after getting together to discuss the candidate.

collyw 10y ago

Surely a lot depends timing and circumstances. One interviewer said they gave me the job based on asking if the guys accent was Dutch (he was Norwegian and found that hilarious). It was a maintenance programming job and had a high turnover of staff. Other interviews you may have done well but had some serious competition.

Fricken 10y ago

I've never interviewed for a job I didn't get. Possibly because I aim too low. But if I were to present myself honestly as a potential employee, I would enter the interview without greeting or making eye contact, then I would sit as far away from the interviewer as possible and doodle in a notebook for 15 minutes before raising my head and explaining everything I think is wrong about the company.

kafkaesq 10y ago

What a hatchet job. Unfortunately we don't have the raw data available; but all we can infer from these heatmaps is that yes (and exactly as one would expect) there's a bit of variance between between the two sets of ratings (in both directions). Maybe a bit more than can be explained by the sample size, or from the simple fact people (on both sides of the process) are forced to discretize their assessments (so even if both parties were to objectively agree that the "real" performance was in the 2.4-2.7 range, which it is a good chunk of the time.... inevitably you'll find them be off by +/- 1 in their quantile assignments a good chunk of the time -- which is exactly what the data seem to show). In particular: independent of how we explain the variance (whether due to sampling effects, or bona fide DK/IS) note that the vast majority of the variance is in the +/- 1 range. And on the alleged DK side (-2 variance) the measure is apparently quite miniscule. However, that's not the original D-K effect. What D-K specifically predicted (and were able to show with much better data and methodology -- in their very specific sample population, at least) is that, among the general population there will be a striking degree of -2 variance, specifically in the bottom quartile. Going by the oft-quoted abstract from their original paper: Across 4 studies, the authors found that participants scoring in the bottom quartile on tests of humor, grammar, and logic grossly overestimated their test performance and ability. Although their test scores put them in the 12th percentile, they estimated themselves to be in the 62nd. In other words: "Performers in the bottom quartile estimated themselves to be performing, on average, at mean performance in the third quartile." So if were to look for something analogous to a D-K effect, we'd expect to find a "hot spot" (or at least the strongest amount of red ink, for that quartile) in the upper left of the first chart. Instead we find most performers in the bottom quartile (correctly) identifying themselves as in the bottom quartile; with a smaller portion bleeding up into the second quarter; and an an almost negligible portion placing themselves in the D-K zone -- namely, the third quarter. If anything what the data show is that (even assuming there's nothing to contend with in how the actual data were taken), while inevitably there's some variance (both ways) in the two sets of rankings, and probably some of due to distorted self-perception -- on the whole there's actually much less of a D-K effect among engineers than among the general population (or rather, "Cornell undergraduates taking psychology classes" -- which, lest we forget, were the actual subject class used in the original D-K study). So basically the opposite of what the author of the original article is claiming to infer from this data.

Ologn 10y ago

After having interviewed dozens of people, the idea occurred to me that my ratings for interviewees were a Gaussian curve with a normal distribution. Most people blended into each other - they knew the topics the same amount. Enough to be employed in their current job. Then one in six was a standard deviation below or worse, and one in six was a standard deviation above or better. That might be one reason for the impostor syndrome. Because if we interviewed six people, one would be unqualified, four would be OK but not spectacular, and one would know the subjects better than anyone. So in an interview of six people the best qualified one of the six would have the job. The average four were not truly impostors, but would not get the job unless they had an inside recommendation or if we were really needed someone and there was a shortage of candidates coming in.

merb 10y ago

That there still needs to be discussions about that topic. On a workplace people should be treated equally. No matter how many topics will be brought up. A world with different genders, stereotypes, whatever can only work if both sides won't interpret too much into it and just enjoy their work with their colleagues. Actually we don't employ a woman (I'm working at a 3 person company). And we actually hire, but the only woman will probably decline on their side, she actually has experience in things we actually not need yet, so I guess we can't provide her a new challenge. way probably the best application.

delphinius81 10y ago

Once you have an engineering job, a typical project tasking involves: some team design discussion, some engineering research, some work, some review, more work. The requirements are either known upfront, or determined as a team. I do not understand why interviews are not conducted in this manner. Pick some arbitrary idea and have the candidate work with the interviewer to design something, then let the candidate do a little implementation research/design, review with the next interviewer, wipeboard code and review some part of the project with next reviewer. Have a coffee break at some point to get to know the person. Treat the interview process like a normal day at work.

jomamaxx 10y ago
"I do not understand why interviews are not conducted in this manner. " It truly is crazy.
dogma1138 10y ago
That's a good idea but logistically is hard, both requiring the interviewer to spend considerable time on site as well as costing your own employee's time. It might work well for a small company especially with a small number of candidates but it doesn't really scale, unless you are a really big company (googleish size) that can dedicate so many resources to the interviewing process. Doing design and research takes time, you can't do it in 30 min, you can't also ask someone to spend 8 hours interviewing, and if you have 10 candidates you can't spend half a man month or more on interviewing them either. I've seen some companies that "interview" by contracting candidates for 1-3 days of work and paying them for it, but it really only works on positions where you hiring effectively interns/fresh graduates, but as most candidates especially those who end up in the face to face phase are currently employed you can't really do that.
- delphinius81 10y ago
  I dunno, I've personally spent on average half a day (3-4 hours) with on-site interviews as a candidate. When on the other side of the table, it's been about the same (~3 hours). With a tractable enough problem, I think you can fit it into a half-day. You could also begin the design/research process as part of a phone screen, have the candidate further prepare on their own time, and then present/discuss as part of the on-site, along with the more in-depth technical stuff. After you've run this process enough times, your interviewers will have a good grasp on the various solutions and technical problems. Most people's training on how to interview comes from how they themselves were interviewed - which was probably poorly done. Improve the interview process and you can train people to be better interviewers just by being interviewed. :)

mark-r 10y ago

Don't miss the footnote: "I’m always terrified of misspelling “Dunning-Kruger” and not double-checking it because of overconfidence in my own spelling abilities."

Apocryphon 10y ago

I understand companies are unwilling to give negative feedback to applicants for fear of legal repercussion. But often not getting any form of feedback is highly detrimental to an applicant who's invested days if not weeks of going through the process, only to receive a boilerplate "thank you for applying, please try again" response. One wonders if it's possible to devise some sort of neutral, yet helpful, assessment that could be given to applicants without fear of legal retaliation. Or, if eventually an applicant will sue a company for not providing feedback after they were rejected.

matmann2001 10y ago

Probably because interviewing companies have stopped giving performance feedback.

nfriedly 10y ago

I used to do a lot of interviews for front-end engineers. The process I settled on was to give them the following setup: - some mock-ups from when we first designed our product gallery - a json file that listed the products names, pictures, description, etc. - a blank css file - a html file that loaded the css and jQuery, and then made an ajax request to load the json file. (This was back when jQuery was pretty much state of the art for front-end.) With all of that in place, I'd explain that I'd like them to work on building a product gallery that looked similar to the mockups. I would further explain that, since we only had an hour, I didn't expect a polished or even fully functional product, but I'd just like them to dive in and see how far they got. They could use their editor of choice and ask any questions they wanted. I felt like this gave better insight to candidates skill and workflow. My colleagues all gave more traditional interviews, so I also felt like there was a good counterbalance if someone didn't do well with this kind of interview.

xxbondsxx 10y ago
Doesn't that heavily bias towards candidates who have used jQuery before and/or built a product gallery before? Skewing interview performance based on some specific previous experience might be what you want if you expect your candidates to hit the ground running, but might not be the best to gauge long-term performance.
- nfriedly 10y ago
  This was back in the day when everyone used jQuery for everything, and the company was only looking for experienced people, so a candidate who hadn't used jQuery before probably wouldn't have been a good match for the position anyways. I don't think a single candidate had that issue though. As far as the product gallery goes, I wasn't looking for a pixel-perfect product. Just spit out a bit of html and matching CSS that look vaguely like the mockup. The mockup images had things like the hex values of the colors written on the image to make that part easy. Basically, it was trying to focus on things that would be the main part of the candidate's job there. The folks that we did hire all did well on both that interview and long-term at the company.

ethicalconsult 10y ago

4 For University Grades changing +Erase Criminal Records hack +Driver License +Facebook hack, Twitter hack, Instagram hack, Gmail, Yahoo hack +Already hacked Paypal, MasterCard, Credit Card untraceable balance +Email interception get a hold of consultanthackers@outlook.com , certified , professional and highly time conscious... call (302) 365-0294? and its done !