15 ms·

Does stress impact technical interview performance? (2020) [pdf]

109 points by kevbin 6y ago

kevbin 6y ago
> A technical interview has an uncanny resemblance to the trier social stress test [39], a procedure used for decades by psychologists and is the best known “gold standard” procedure [1] for the sole purpose of reliably inducing stress. > The trier social stress test involves having a subject prepare and then deliver an interview-style presentation and perform mental arithmetic, all in front of an audience. > … rather than avoiding unwanted stress, technical interviews may be inadvertently designed with the sole purpose of inducing it. 39: https://www.karger.com/Article/Abstract/119004 https://www.karger.com/Article/Abstract/119004
- madhadron 6y ago
  I mean, that's what the interview format was designed to do. Schockley decided that how he should hire engineers was to stress them and ask them brain teasers. He had no basis but his own opinion, but he had to hire fast because none of his colleagues from Bell Labs were willing to work with him again. Then a bunch of his engineers all quit because he was such a terrible boss and founded Fairchild Semiconductor. We've at least abolished the brain teaser part in most cases.
  - gfxgirl 6y ago
    We really haven't though. I've been going through the leetcode questions and I'd say > 30% are basically brain teasers and either you've seen the answer or related answer before or you haven't. If you're lucky and have some magical insight then maybe you solve it. Otherwise you look up the answer, possibly implement it yourself to try to cement it in your mind, and now if you're lucky it's added to your internal list of known solutions. To give just one example, I don't think I'd have ever solved the "find the largest rectangle" in a bitmap. Certainly not in a 45 minute interview. The brute force solution is O(N^3 * M^3). Me, I couldn't think of any spatial structures that would make the problem all that much smaller. For any structural solution it seems to be the degenerate case still ends up using too much space and too much time. But, there's an optimal "magic" solution is O(N*M). If you happen to know it or have seen then then you bang it out. If you haven't it's arguably just luck that you'll have the insight during your interview. I've seen several others of the same vane. Given 4-24 hours and a debugger to help my thoughts and directly my mistaking assumptions I might solve some of them but in 45 minutes on a whiteboard, it's just a lucky role of the dice that I've seen something similar before. The Jan 2021 google foobar was similar. I didn't figure it out nor did a friend of mine. We both spent 3-4 days. I bought it up with another friend. His first solution was wrong. When I pointed out why he was like "Oh yea, this the XYZ-algo" (meaning he didn't solve it, he just had read the solution before). The wikipedia page for the arguably obscure algo basically showed the exact same example as the foobar one.
nabla9 6y ago
It seems that technical interview is a great way to do psychological evaluation. How does this person manage stress and deal with it?
- kevbin 6y ago
  The authors address this: the induced stress is not necessarily similar to the stress experienced on the job they're applying for.
  - planet-and-halo 6y ago
    This has been my experience. Time is crunched in a rather unrealistic way. Not that time crunches don't happen, but it's never once been the case in my real job that I was approaching an unpracticed academic algorithm problem which had a 30 minute time cap. Recent example: I was asked to do a density map with about ~20 minutes remaining in an interview. I could immediately see the performance issue of the naive solution (re-summing the same positions), but I needed some time to think through an efficient solution. But because of the time limit and the fact that I had someone watching my every step (i.e. there was more performance than problem solving going on), we agreed that I would take the naive approach and simply sum all of the positions as quickly as I could. When I wasn't asked back for a second round, it was obviously in part because I had resorted to such an ugly and inefficient solution. I don't blame the interviewer—he was actually one of the best I've ever interacted with—and I actually think the problems selected fit the role very well. But at the end of the day I don't feel that format gave a very good snapshot of my abilities. (Case in point, I looked up solutions afterwards and an intuitive approach I had thought of was very close to the optimal solution, but I didn't feel I had time to go down that route.) I don't think there is a perfect interview format. They all have downsides. But this is definitely a tradeoff you're making with timed algos.
- hinkley 6y ago
  Why do you want all of your employees to be the ones most resilient to stress? Are you hiring for a fire department? The Navy Seals? Engineering section on a nuclear submarine? Maybe, just maybe, things would improve if you had a few people who said 'no way' when someone tries to crank up the drama on your team instead of just grinning and bearing it. Capitalists don't reward the quiet workers. They exploit them. Billionaires and wannabees do it remorselessly.
tchalla 6y ago
Previous Discussion 8 months ago : https://news.ycombinator.com/item?id=23848039 https://news.ycombinator.com/item?id=23848039
louisstow 6y ago
My least favourite interview technique has to be "talk me through a complex problem on a whiteboard". It's not so much the stress but the fact that you can't have a train of thought longer than 5 seconds before someone finds the silence awkward or interprets it as you're lost and breaks the silence and train of thought. It's a test for solving complex problems in 5 second blocks while being constantly interrupted.
- Germanika 6y ago
  Working through a problem out loud is definitely a bit of a different skill from just solving a complex problem on your own. I do find that saying something along the lines of "OK now I just need a minute to think through this next part" goes over well enough and can give you a minute to relax and think though.
  - planet-and-halo 6y ago
    I think the issue is "a minute." You still feel pressured to get moving and keep talking, even if it buys you a little time. My favorite part of the job is whiteboarding with colleagues, but in an interview there is something that feels inherently adversarial about it. This isn't necessarily the fault of the interviewer either, it's just sort of baked into the format. There's a natural power discrepancy: the interviewer makes the final call, and they selected the problem to begin with, which means you can't possibly be on equal footing. That imbalance means you spend the whole time on the back foot. I'm not sure there's a better format available, but that's definitely an unnatural environment and can lead people including me to perform much differently than they do in a normal work environment. Unfortunately it may be the best we can do.
    - bmitc 6y ago
      I think the better format is to have the interviewee talk through a complex problem that they have already solved, whether it's a side project, a work project they can talk about, or whatever. It balances out the power imbalance, and it showcases how well the interviewee can explain and talk through problems, design decisions, what they learned, what worked, what didn't, etc. through a problem they are already familiar with. These whiteboarding interviews where the interviewers pose some random question does nothing for either party.
_greim_ 6y ago
So, it's a long PDF. Can someone summarize the recommendation from the study?
- tchalla 6y ago
  The abstract of a paper is an executive summary. In case, that is not enough - you can look into the press release [0]. [0] https://news.ncsu.edu/2020/07/tech-job-interviews-anxiety/ https://news.ncsu.edu/2020/07/tech-job-interviews-anxiety/
heroHACK17 6y ago
Or, an alternative title: Is Water Wet?
- ChrisMarshallNY 6y ago
  You do realize that the correct answer to any headline that ends with a question mark, is "No." So I guess water is not wet. I will say that I am comfortable with people not wanting to work with me, because I don't fit their "cultural criteria." Take-home tests are a good idea, but you still have the issue that so many folks here have mentioned: They don't just want you to solve the problem. They want you to solve it the way they want. Orthogonal approaches are generally not met with support. I once had a take-home, where I was asked to produce an iOS app that used a certain dependency (that I had never used), and was asked to do so, using MVVM. In four hours, I had it done. It was a localized, ship-ready app. I didn't use MVVM, because UIKit is designed specifically for MVC, and using other models actually makes the app larger and more prone to problems. It was a tiny app, and I didn't want to add any code that wasn't 100% necessary (basic Quality 101). I also encapsulated the dependency (Quality 101, again). Did I mention that I had never even looked at the dependency API, before starting that project? During the process, my home had an electrical emergency, and almost burned down, so I guess there was a bit of stress. I could smell the burning rubber, as they peeled out (although it could have been the electrical fire). They never asked me why I made the choices I did. I strongly suspect that I was never in consideration anyway. I think they were checking off a "interviewed old guy, and he sucked" checkbox on an EEOC form. I handle stress quite well, indeed. A quick glance at my CV tells you that I worked for one of the top Japanese firms in the world, for 27 years. They refine and mass-produce stress in Tokyo. If you last 27 years at a joint like that, you can handle stress. The folks that actually get around to working with me, seem to enjoy the experience. Funny how that happens.
efwfwef 6y ago
My latest system design interview was: design this app. Dude was like "I'm expecting you to lead the conversation". I was like, bitch there's a million things to talk about, at some point you have to tell me where you want the convo to go. Needless to say I failed, when I'm probably overqualified for the job, but heh...
- MattGaiser 6y ago
  I had this for my current job. I was asked to design Facebook and it just ended up being nearly entirely about how to design a notification microservice. Got the job so it evidently worked, but I was just going on in a direction until told to stop.
- deleted 6y ago
  [deleted]
- xyzelement 6y ago
  I don't think you're overqualified. I'm not a systems designer by any means but here's how I'd lead this conversation: "Ok cool. Before we get started, let me understand some non functional requirements. How many concurrent users do we expect? Is brief downtime acceptable?" Then you listen to the answer. Then you say "ok cool. So given what we just estsblished, it's making me lean towards X type of architecture. Here are trade-offs A,B, and C that are involved. Does this make sense to you so far?" You listen to their answer. They will 100% of the time indicate to you if you are on the right track or missing something. Then you go "ok cool. So on the high level we are going to have (for example) a client, a server, a proxy, a database, etc . Given what we discussed, I think the most complex part of the system will be X so let me begin by thinking through that and then we can pop back up to other parts. Does that sound good to you?" Etc. I am not looking to make you feel bad but being able to have this conversation in this way is a baseline skill of a senior engineer. Compare that to what you actually did and ask yourself if it's you or the "bitch" that has room to improve here.
  - deleted 6y ago
    [deleted]
- 6y ago
taurath 6y ago
The interview process is somewhat akin to the “stress” gatekeeping in other industries, where people in charge say “I’ve gone through this, so it makes sense that others would have to also or else we risk getting worse people”. While not really in the same ballpark it feels somewhat like the medical residency system, something that everyone knows is drastically unhealthy but it keeps getting propagated because people are afraid of what happens when you change it. As someone with clinically diagnosed and treated anxiety the whiteboard interview is an absolute personal hell. I have extremely good performance ratings everywhere I’ve landed a job even in very high stakes actual work environments, but my opinion on the average interview process is it’s extremely discriminatory against people who deal with practically any mental health issue (which, let’s face it is probably quite prevalent in this industry).
- potta_coffee 6y ago
  A similar rationale was used to justify hazing and other stupid things when I was in the Marines. "We do this the most utterly painful and stupid way because that was what was done to me as a Private, and now it's my turn."
boredpandas777 6y ago
There is no other field where the candidate has to pass an exam ever time they switch jobs. I suggest getting software engineers licensed by an independent party just once (maybe renew every 10 years.)
- asdff 6y ago
  Some might say the degree is just that
- deleted 6y ago
  [deleted]
- EvilEy3 6y ago
  If everyone has the license it is useless and you're back to square one.
GongOfFour 6y ago
The only time I've ever had a panic attack was when I was white boarding some basic stuff in an interview while I was hungover. Obviously being hungover is non-ideal and one wouldn't expect the interview experience to account for that but nonetheless I have always found it interesting that it took that combination to put me over the edge. I remember leaving as quickly as I could and thinking that I had to find a place to sit down as soon as possible. I made it to Washington Square Park and just sat at a bench for about 30 minutes feeling like I was slowly coming out of a tunnel. Never happened before nor since.
bmitc 6y ago
Absolutely. I've had it happen. I have even undergone tests that shows I experience performance anxiety, and that it actually goes up the easier a question is. I've had interview instances where my face felt flushed and my mind completely blank on a question I knew how to do but struggled on. And then 30 minutes after the interview finished, the answer is clear as day in my head because it was indeed something I knew and the anxiety was gone. I feel there is a fair amount of discrimination against those of us who experience anxiety like this, but in a way, I don't care all that much because places that interview certain ways aren't places I want to work anyway.
ram_rar 6y ago
This is one of the reasons, I prefer small take home assignments which can be later discussed with the interviewer about the pros and cons of the decisions made during coding. As an interviewer, you get more insight into what the candidate was thinking and whether they were able to actually implement something end to end. Leetcode style problems rarely give you that signal from candidates.
- Trasmatta 6y ago
  I've done those before, and found the employer always underestimates how much time they should take. I think it's a good idea, but I've done 3 where the employer said "this should take 2-3 hours", but I actually had to spend an entire weekend on it instead. Felt like a waste of time, especially for the ones I didn't even make it to next phase in.
  - deleted 6y ago
    [deleted]
  - garbene 6y ago
    Absolutely, every job states it should take a few hours but it NEVER takes a few hours if you want to deliver something that’s remotely competitive. My current role’s take home was suppose to take “a few hours”, took the entire weekend. But hey, it worked out.
- bitbuilder 6y ago
  As a hiring manager who was once a developer who was absolutely terrified of the typical whiteboard/algorithm interview, take home assignments are my preferred method to judge tech ability. However, I often feel it's not fair to ask the candidate to invest their personal time in the interview process. So I'm glad to see some affirmation that it's the preferred method for at least some. When I have done take homes, I've tried to keep the time requirements minimal and the project itself simple and hopefully even a bit fun. It might even be as simple as "here's a REST endpoint, pick the JS SPA framework of your choice and do something interesting with that endpoint." Even just a simple form with clean code that builds will be enough to get you hired if you're otherwise a good candidate. Because in the real world, most of of us are just building CRUD apps. All that said, I only bother with this if the candidate is incredibly green. For anyone with decent experience I generally assume they're not lying on their resume, and an hour long chat about their projects and technology interests will tell you more than any technical assessment.
posharma 6y ago
We discuss interviewing a lot here. Despite all the proposals things don't really change. FA(A)NG style interviews are just that - whiteboarding solutions to N algorithmic problems in M minutes (or minor variants of this). Either we accept this and move on with solid preparation or modify our priorities such that we're fine with not wanting the money and/or the quality of work (debatable) offered by these companies. At the end of the day, it's a choice and it's still in our hands.
- deleted 6y ago
  [deleted]
daniel-thompson 6y ago
Speaking from personal experience, it absolutely does; both on a mental level (the feeling of being watched and judged, leading to constant second-guessing) and on a physical level, where it invokes the fight-or-flight response. I'm sure the latter in particular was super useful to our ancient ancestors, for whom a shot of adrenaline during stressful situations could be the difference between survival and death. But the physical symptoms - elevated heart rate, dry mouth, tunnel vision, shaking, excess perspiration - are actively harmful to performance in a software engineering interview. I have friends who, before upcoming situations they know will be stressful - interviews for engineers, performances for musicians, surgery for doctors - have obtained prescriptions for beta-blockers, which mitigate the physical symptoms of the fight-or-flight response for a few hours at a time. * https://en.wikipedia.org/wiki/Beta_blocker#Anxiety https://en.wikipedia.org/wiki/Beta_blocker#Anxiety
gkoberger 6y ago
We have a rule for technical interviews that we want to "enable each candidate to show the best version of themselves". It seems simple and obvious, but I'm amazed by how happy and excited people are by our interview process. I constantly get emails saying it was such a relief or that they loved it. It just seems so crazy that we pressure technical candidates the way we do, and expect decent results.
- ugh123 6y ago
  That sounds great, but how do you achieve that?
  - gkoberger 6y ago
    Here's an HN comment about I made about it a while ago: https://news.ycombinator.com/item?id=24457170 https://news.ycombinator.com/item?id=24457170
5cott0 6y ago
A kooky theory I harbor is that corporate tech continues to employ algo whiteboard hazing ritual interviews because if they were to evaluate SWEs with tasks more closely resembling the expected day-to-day they would have to pay them an hourly rate under CA labor law or face liability exposure.
jmfldn 6y ago
It absolutely does. One of my first programming interviews, post MSc graduation, was essentially a 3hr whiteboard interrogation. I should have walked out after an hour as, in retrospect I was stuck like a rabbit in the headlights, unable to think straight as this unpleasant interviewer essentially tried to dismantle every angle of my understanding of a subject. I was gripped with anxiety and lost confidence. He claimed he was just trying to find out the edges of what I knew but boy did he do it in an unpleasant way. Interrupting, making slightly snide comments, being plain rude at times. I think he was either an actual jerk or maybe just not very good with people. Anyway, other than that interview where I was super stressed and tanked it, all my others have been nothing like it. Just goes to show how stress and a loss of confidence can wreck your chances.
- garbene 6y ago
  I interviewed at a company last month that reminded me of your experience. The interviewer was extremely rude and it was more of an interrogation than anything. After berating one of my responses, I told him this wasn’t going to work out, that this was easily the most off putting interview I’d ever experienced, but thanks for the consideration. The dude was stunned, apologized profusely, and we somehow salvaged the last 20 minutes which we spent nerding out on tech and hobbies. I ended up taking my name out of consideration, but it was a cool experience realizing I can (respectfully but assertively) tell people to fuck off if I need to.
  - jmfldn 6y ago
    Well done for doing that. It's empowering to bear in mind that you are interviewing them too. If your interviewer is highly unpleasant then why on earth would you want to work for them / a company that thinks its OK to be represented by them. No thanks!
11thEarlOfMar 6y ago
We use a 3 step process, sans whiteboard: 1. First interview, the candidate interviews us. What market we serve, what our development processes are, what our technology stack is. If they express interest by being prepared and asking good questions, we send them 2. a programming task. Choose 1 of 5 tasks. The tasks are not abstract problems or puzzles, but come out of the designs we've implemented. We ask for 100 lines of code and not more than 2 hours. They take as much calendar time in days as they need, most have full time jobs, and let us know when you're done. The candidate then comes back, typically in 2 to 7 days and hosts a code review in front of our team. I give strict instructions before hand: The purpose of the review is to learn how they thought through the problem and how they solved it, not to criticize their style and approach. 3. If we decide we like them to this point, the third interview is with managers from other functions. How well does the candidate communicate, come across to non-developers, express interest in the company and role, etc. It's a check point to look for concerning weaknesses, as well as get buy in from the broader organization. We like this approach because it allows the candidate to code in a much more natural environment. They can take the time they need and comfortably solve. No one spends their professional career coding on a white board. This approach so far has yielded excellent results. We ask developers how much time they took in the programming task and why they chose the one they solved. The programming task is telling, not in the quality of their code and how long they took, but how much did they get into it? We have had the range from candidates who did not complete it at all and opted out, to candidates who stumped 40 year veterans with elegant code. In every case, we learn how well they can express their thoughts, and importantly, their level of love for the discipline. This is as important to me as any other attribute.
- maxwelldone 6y ago
  I love this approach. As someone who generally asks a lot of questions, your first round would give one a good idea if it's worth proceeding to other rounds or not. BTW, are you guys hiring?
- ammanley 6y ago
  Sounds like a place worth working for
- 6y ago
itsdrewmiller 6y ago
Kind of disappointing that all the top rated comments here are about people's personal opinions about interviewing, and don't engage with the pdf at all. What I found most interesting was Table 1 - it shows that in the unsupervised interview the average score of women was 3/3, while in the supervised version it was less than 1/3. Small sample sizes, but still seems suggestive. (I CTRL-Fed but didn't find much discussion of it in the paper.)
dmtroyer 6y ago
Yes.
dang 6y ago
Related from last year: Does Stress Impact Technical Interview Performance? - https://news.ycombinator.com/item?id=23844367 https://news.ycombinator.com/item?id=23844367 - July 2020 (26 comments)
lubesGordi 6y ago
It'd be interesting to see the impact wrt age.
nullifidian 6y ago
Didn't "stereotype threat" fail replication? Why do researchers cite it?
neilv 6y ago
The paper is easy to read, and a bunch of questions that came to mind were addressed, or at least acknowledged, in one place or another of the paper. I appreciated the following tidbit, which I didn't know (since I figured prep for this particular test was now widespread there, by convention, but not officially stated): > students at Stanford take courses for passing technical interviews, CS9: Problem-Solving for the CS Technical Interview. And because of things like that, the current interview convention seem even less useful as a predictor, for purposes other than "Did you go to Stanford, or are of similar socioeconomic status, to have been coached on beating the current metric, specifically?"
- coldcode 6y ago
  If you are expected to study to pass an interview, is the result really indicative of how you will succeed in the job, or a measure of your skill is studying to pass an interview?
worker767424 6y ago
I can't say I enjoy interviews, but they're really not that bad. Reading the comments here, you'd think everyone gets paralyzing anxiety during interviews. They're a chance to show off how smart you are.
- kevingadd 6y ago
  I've probably done at least a few hundred tech interviews in my career, and the vast majority of them are simply not an opportunity to show anything off. You're putting up with various interviewers in various moods of varying levels of competence, some of them actively hostile or working on some sort of weird agenda. You're bouncing between interviewers and getting asked the same question multiple times and having to explain the same thing over and over. Even if they don't make you anxious (that started to subside after the first hundred times or so) they're still not a good time. I've had senior staff running interviews at big companies do things like harass me over data structure choices, insist that I was lying when I described a textbook algorithm, accuse me of industrial espionage, and all other sorts of absurd or unpleasant things. In some cases even after that I still got an offer, because the interviewers weren't out to give someone a hard time... they were just bad at running interviews. One of my all-time worst interviews involved the design director (?) at a billion-dollar game studio calling me (unscheduled!) at 7am and demanding that I come up with 50 unique uses for a paperclip. Really. The gap between an average or bad interview and a truly good one is enormous, and the number of well-conducted interviews I've had is incredibly small. I can only imagine how much worse it is for someone with additional obstacles, like how is someone in a wheelchair supposed to stand up and write C++ on a whiteboard for 30 minutes? What does the whiteboard interview look like for a blind woman?
aphextron 6y ago
Literally just reading that headline gave me cold sweats and a wave of anxiety.
confidantlake 6y ago
I am surprised that there are not more "traditional tests". Give problem, give person 45 minutes, leave the room, come back and discuss it. I have strong social anxiety. It is extremely difficult to solve a problem with someone watching me. I get panic attacks in at least 1/3 of my interviews. I have never had to code with someone watching me during my day to day job. I am not not interviewing for the job of coding performance artist. Why make me audition like I am an actor or a musician?
say_it_as_it_is 6y ago
Tech wouldn't be where it is today without antiquated interviews and an industry of human resource management built on faith and tradition rather than science. Talented people go elsewhere, create new innovation, and prosper while the large tech companies copy or acquire the innovation. You are successful not only because of what you accomplish in life but because of what you don't accomplish. You choose not to spend hundreds of hours training to answer coding problems, prioritizing other activities to advance yourself.

109 points by kevbin 6y ago

> A technical interview has an uncanny resemblance to the trier social stress test [39], a procedure used for decades by psychologists and is the best known “gold standard” procedure [1] for the sole purpose of reliably inducing stress. > The trier social stress test involves having a subject prepare and then deliver an interview-style presentation and perform mental arithmetic, all in front of an audience. > … rather than avoiding unwanted stress, technical interviews may be inadvertently designed with the sole purpose of inducing it. 39: https://www.karger.com/Article/Abstract/119004 https://www.karger.com/Article/Abstract/119004

madhadron 6y ago
I mean, that's what the interview format was designed to do. Schockley decided that how he should hire engineers was to stress them and ask them brain teasers. He had no basis but his own opinion, but he had to hire fast because none of his colleagues from Bell Labs were willing to work with him again. Then a bunch of his engineers all quit because he was such a terrible boss and founded Fairchild Semiconductor. We've at least abolished the brain teaser part in most cases.
- gfxgirl 6y ago
  We really haven't though. I've been going through the leetcode questions and I'd say > 30% are basically brain teasers and either you've seen the answer or related answer before or you haven't. If you're lucky and have some magical insight then maybe you solve it. Otherwise you look up the answer, possibly implement it yourself to try to cement it in your mind, and now if you're lucky it's added to your internal list of known solutions. To give just one example, I don't think I'd have ever solved the "find the largest rectangle" in a bitmap. Certainly not in a 45 minute interview. The brute force solution is O(N^3 * M^3). Me, I couldn't think of any spatial structures that would make the problem all that much smaller. For any structural solution it seems to be the degenerate case still ends up using too much space and too much time. But, there's an optimal "magic" solution is O(N*M). If you happen to know it or have seen then then you bang it out. If you haven't it's arguably just luck that you'll have the insight during your interview. I've seen several others of the same vane. Given 4-24 hours and a debugger to help my thoughts and directly my mistaking assumptions I might solve some of them but in 45 minutes on a whiteboard, it's just a lucky role of the dice that I've seen something similar before. The Jan 2021 google foobar was similar. I didn't figure it out nor did a friend of mine. We both spent 3-4 days. I bought it up with another friend. His first solution was wrong. When I pointed out why he was like "Oh yea, this the XYZ-algo" (meaning he didn't solve it, he just had read the solution before). The wikipedia page for the arguably obscure algo basically showed the exact same example as the foobar one.

nabla9 6y ago

It seems that technical interview is a great way to do psychological evaluation. How does this person manage stress and deal with it?

kevbin 6y ago
The authors address this: the induced stress is not necessarily similar to the stress experienced on the job they're applying for.
- planet-and-halo 6y ago
  This has been my experience. Time is crunched in a rather unrealistic way. Not that time crunches don't happen, but it's never once been the case in my real job that I was approaching an unpracticed academic algorithm problem which had a 30 minute time cap. Recent example: I was asked to do a density map with about ~20 minutes remaining in an interview. I could immediately see the performance issue of the naive solution (re-summing the same positions), but I needed some time to think through an efficient solution. But because of the time limit and the fact that I had someone watching my every step (i.e. there was more performance than problem solving going on), we agreed that I would take the naive approach and simply sum all of the positions as quickly as I could. When I wasn't asked back for a second round, it was obviously in part because I had resorted to such an ugly and inefficient solution. I don't blame the interviewer—he was actually one of the best I've ever interacted with—and I actually think the problems selected fit the role very well. But at the end of the day I don't feel that format gave a very good snapshot of my abilities. (Case in point, I looked up solutions afterwards and an intuitive approach I had thought of was very close to the optimal solution, but I didn't feel I had time to go down that route.) I don't think there is a perfect interview format. They all have downsides. But this is definitely a tradeoff you're making with timed algos.
hinkley 6y ago
Why do you want all of your employees to be the ones most resilient to stress? Are you hiring for a fire department? The Navy Seals? Engineering section on a nuclear submarine? Maybe, just maybe, things would improve if you had a few people who said 'no way' when someone tries to crank up the drama on your team instead of just grinning and bearing it. Capitalists don't reward the quiet workers. They exploit them. Billionaires and wannabees do it remorselessly.

tchalla 6y ago

Previous Discussion 8 months ago : https://news.ycombinator.com/item?id=23848039 https://news.ycombinator.com/item?id=23848039

louisstow 6y ago

My least favourite interview technique has to be "talk me through a complex problem on a whiteboard". It's not so much the stress but the fact that you can't have a train of thought longer than 5 seconds before someone finds the silence awkward or interprets it as you're lost and breaks the silence and train of thought. It's a test for solving complex problems in 5 second blocks while being constantly interrupted.

Germanika 6y ago
Working through a problem out loud is definitely a bit of a different skill from just solving a complex problem on your own. I do find that saying something along the lines of "OK now I just need a minute to think through this next part" goes over well enough and can give you a minute to relax and think though.
- planet-and-halo 6y ago
  I think the issue is "a minute." You still feel pressured to get moving and keep talking, even if it buys you a little time. My favorite part of the job is whiteboarding with colleagues, but in an interview there is something that feels inherently adversarial about it. This isn't necessarily the fault of the interviewer either, it's just sort of baked into the format. There's a natural power discrepancy: the interviewer makes the final call, and they selected the problem to begin with, which means you can't possibly be on equal footing. That imbalance means you spend the whole time on the back foot. I'm not sure there's a better format available, but that's definitely an unnatural environment and can lead people including me to perform much differently than they do in a normal work environment. Unfortunately it may be the best we can do.
  - bmitc 6y ago
    I think the better format is to have the interviewee talk through a complex problem that they have already solved, whether it's a side project, a work project they can talk about, or whatever. It balances out the power imbalance, and it showcases how well the interviewee can explain and talk through problems, design decisions, what they learned, what worked, what didn't, etc. through a problem they are already familiar with. These whiteboarding interviews where the interviewers pose some random question does nothing for either party.

_greim_ 6y ago

So, it's a long PDF. Can someone summarize the recommendation from the study?

tchalla 6y ago
The abstract of a paper is an executive summary. In case, that is not enough - you can look into the press release [0]. [0] https://news.ncsu.edu/2020/07/tech-job-interviews-anxiety/ https://news.ncsu.edu/2020/07/tech-job-interviews-anxiety/

heroHACK17 6y ago

Or, an alternative title: Is Water Wet?

ChrisMarshallNY 6y ago
You do realize that the correct answer to any headline that ends with a question mark, is "No." So I guess water is not wet. I will say that I am comfortable with people not wanting to work with me, because I don't fit their "cultural criteria." Take-home tests are a good idea, but you still have the issue that so many folks here have mentioned: They don't just want you to solve the problem. They want you to solve it the way they want. Orthogonal approaches are generally not met with support. I once had a take-home, where I was asked to produce an iOS app that used a certain dependency (that I had never used), and was asked to do so, using MVVM. In four hours, I had it done. It was a localized, ship-ready app. I didn't use MVVM, because UIKit is designed specifically for MVC, and using other models actually makes the app larger and more prone to problems. It was a tiny app, and I didn't want to add any code that wasn't 100% necessary (basic Quality 101). I also encapsulated the dependency (Quality 101, again). Did I mention that I had never even looked at the dependency API, before starting that project? During the process, my home had an electrical emergency, and almost burned down, so I guess there was a bit of stress. I could smell the burning rubber, as they peeled out (although it could have been the electrical fire). They never asked me why I made the choices I did. I strongly suspect that I was never in consideration anyway. I think they were checking off a "interviewed old guy, and he sucked" checkbox on an EEOC form. I handle stress quite well, indeed. A quick glance at my CV tells you that I worked for one of the top Japanese firms in the world, for 27 years. They refine and mass-produce stress in Tokyo. If you last 27 years at a joint like that, you can handle stress. The folks that actually get around to working with me, seem to enjoy the experience. Funny how that happens.

efwfwef 6y ago

My latest system design interview was: design this app. Dude was like "I'm expecting you to lead the conversation". I was like, bitch there's a million things to talk about, at some point you have to tell me where you want the convo to go. Needless to say I failed, when I'm probably overqualified for the job, but heh...

MattGaiser 6y ago
I had this for my current job. I was asked to design Facebook and it just ended up being nearly entirely about how to design a notification microservice. Got the job so it evidently worked, but I was just going on in a direction until told to stop.
deleted 6y ago
[deleted]
xyzelement 6y ago
I don't think you're overqualified. I'm not a systems designer by any means but here's how I'd lead this conversation: "Ok cool. Before we get started, let me understand some non functional requirements. How many concurrent users do we expect? Is brief downtime acceptable?" Then you listen to the answer. Then you say "ok cool. So given what we just estsblished, it's making me lean towards X type of architecture. Here are trade-offs A,B, and C that are involved. Does this make sense to you so far?" You listen to their answer. They will 100% of the time indicate to you if you are on the right track or missing something. Then you go "ok cool. So on the high level we are going to have (for example) a client, a server, a proxy, a database, etc . Given what we discussed, I think the most complex part of the system will be X so let me begin by thinking through that and then we can pop back up to other parts. Does that sound good to you?" Etc. I am not looking to make you feel bad but being able to have this conversation in this way is a baseline skill of a senior engineer. Compare that to what you actually did and ask yourself if it's you or the "bitch" that has room to improve here.
- deleted 6y ago
  [deleted]
6y ago

taurath 6y ago

The interview process is somewhat akin to the “stress” gatekeeping in other industries, where people in charge say “I’ve gone through this, so it makes sense that others would have to also or else we risk getting worse people”. While not really in the same ballpark it feels somewhat like the medical residency system, something that everyone knows is drastically unhealthy but it keeps getting propagated because people are afraid of what happens when you change it. As someone with clinically diagnosed and treated anxiety the whiteboard interview is an absolute personal hell. I have extremely good performance ratings everywhere I’ve landed a job even in very high stakes actual work environments, but my opinion on the average interview process is it’s extremely discriminatory against people who deal with practically any mental health issue (which, let’s face it is probably quite prevalent in this industry).

potta_coffee 6y ago
A similar rationale was used to justify hazing and other stupid things when I was in the Marines. "We do this the most utterly painful and stupid way because that was what was done to me as a Private, and now it's my turn."

boredpandas777 6y ago

There is no other field where the candidate has to pass an exam ever time they switch jobs. I suggest getting software engineers licensed by an independent party just once (maybe renew every 10 years.)

asdff 6y ago
Some might say the degree is just that
deleted 6y ago
[deleted]
EvilEy3 6y ago
If everyone has the license it is useless and you're back to square one.

GongOfFour 6y ago

The only time I've ever had a panic attack was when I was white boarding some basic stuff in an interview while I was hungover. Obviously being hungover is non-ideal and one wouldn't expect the interview experience to account for that but nonetheless I have always found it interesting that it took that combination to put me over the edge. I remember leaving as quickly as I could and thinking that I had to find a place to sit down as soon as possible. I made it to Washington Square Park and just sat at a bench for about 30 minutes feeling like I was slowly coming out of a tunnel. Never happened before nor since.

bmitc 6y ago

Absolutely. I've had it happen. I have even undergone tests that shows I experience performance anxiety, and that it actually goes up the easier a question is. I've had interview instances where my face felt flushed and my mind completely blank on a question I knew how to do but struggled on. And then 30 minutes after the interview finished, the answer is clear as day in my head because it was indeed something I knew and the anxiety was gone. I feel there is a fair amount of discrimination against those of us who experience anxiety like this, but in a way, I don't care all that much because places that interview certain ways aren't places I want to work anyway.

ram_rar 6y ago

This is one of the reasons, I prefer small take home assignments which can be later discussed with the interviewer about the pros and cons of the decisions made during coding. As an interviewer, you get more insight into what the candidate was thinking and whether they were able to actually implement something end to end. Leetcode style problems rarely give you that signal from candidates.

Trasmatta 6y ago
I've done those before, and found the employer always underestimates how much time they should take. I think it's a good idea, but I've done 3 where the employer said "this should take 2-3 hours", but I actually had to spend an entire weekend on it instead. Felt like a waste of time, especially for the ones I didn't even make it to next phase in.
- deleted 6y ago
  [deleted]
- garbene 6y ago
  Absolutely, every job states it should take a few hours but it NEVER takes a few hours if you want to deliver something that’s remotely competitive. My current role’s take home was suppose to take “a few hours”, took the entire weekend. But hey, it worked out.
bitbuilder 6y ago
As a hiring manager who was once a developer who was absolutely terrified of the typical whiteboard/algorithm interview, take home assignments are my preferred method to judge tech ability. However, I often feel it's not fair to ask the candidate to invest their personal time in the interview process. So I'm glad to see some affirmation that it's the preferred method for at least some. When I have done take homes, I've tried to keep the time requirements minimal and the project itself simple and hopefully even a bit fun. It might even be as simple as "here's a REST endpoint, pick the JS SPA framework of your choice and do something interesting with that endpoint." Even just a simple form with clean code that builds will be enough to get you hired if you're otherwise a good candidate. Because in the real world, most of of us are just building CRUD apps. All that said, I only bother with this if the candidate is incredibly green. For anyone with decent experience I generally assume they're not lying on their resume, and an hour long chat about their projects and technology interests will tell you more than any technical assessment.

posharma 6y ago

We discuss interviewing a lot here. Despite all the proposals things don't really change. FA(A)NG style interviews are just that - whiteboarding solutions to N algorithmic problems in M minutes (or minor variants of this). Either we accept this and move on with solid preparation or modify our priorities such that we're fine with not wanting the money and/or the quality of work (debatable) offered by these companies. At the end of the day, it's a choice and it's still in our hands.

deleted 6y ago
[deleted]

daniel-thompson 6y ago

Speaking from personal experience, it absolutely does; both on a mental level (the feeling of being watched and judged, leading to constant second-guessing) and on a physical level, where it invokes the fight-or-flight response. I'm sure the latter in particular was super useful to our ancient ancestors, for whom a shot of adrenaline during stressful situations could be the difference between survival and death. But the physical symptoms - elevated heart rate, dry mouth, tunnel vision, shaking, excess perspiration - are actively harmful to performance in a software engineering interview. I have friends who, before upcoming situations they know will be stressful - interviews for engineers, performances for musicians, surgery for doctors - have obtained prescriptions for beta-blockers, which mitigate the physical symptoms of the fight-or-flight response for a few hours at a time. * https://en.wikipedia.org/wiki/Beta_blocker#Anxiety https://en.wikipedia.org/wiki/Beta_blocker#Anxiety

gkoberger 6y ago

We have a rule for technical interviews that we want to "enable each candidate to show the best version of themselves". It seems simple and obvious, but I'm amazed by how happy and excited people are by our interview process. I constantly get emails saying it was such a relief or that they loved it. It just seems so crazy that we pressure technical candidates the way we do, and expect decent results.

ugh123 6y ago
That sounds great, but how do you achieve that?
- gkoberger 6y ago
  Here's an HN comment about I made about it a while ago: https://news.ycombinator.com/item?id=24457170 https://news.ycombinator.com/item?id=24457170

5cott0 6y ago

A kooky theory I harbor is that corporate tech continues to employ algo whiteboard hazing ritual interviews because if they were to evaluate SWEs with tasks more closely resembling the expected day-to-day they would have to pay them an hourly rate under CA labor law or face liability exposure.

jmfldn 6y ago

It absolutely does. One of my first programming interviews, post MSc graduation, was essentially a 3hr whiteboard interrogation. I should have walked out after an hour as, in retrospect I was stuck like a rabbit in the headlights, unable to think straight as this unpleasant interviewer essentially tried to dismantle every angle of my understanding of a subject. I was gripped with anxiety and lost confidence. He claimed he was just trying to find out the edges of what I knew but boy did he do it in an unpleasant way. Interrupting, making slightly snide comments, being plain rude at times. I think he was either an actual jerk or maybe just not very good with people. Anyway, other than that interview where I was super stressed and tanked it, all my others have been nothing like it. Just goes to show how stress and a loss of confidence can wreck your chances.

garbene 6y ago
I interviewed at a company last month that reminded me of your experience. The interviewer was extremely rude and it was more of an interrogation than anything. After berating one of my responses, I told him this wasn’t going to work out, that this was easily the most off putting interview I’d ever experienced, but thanks for the consideration. The dude was stunned, apologized profusely, and we somehow salvaged the last 20 minutes which we spent nerding out on tech and hobbies. I ended up taking my name out of consideration, but it was a cool experience realizing I can (respectfully but assertively) tell people to fuck off if I need to.
- jmfldn 6y ago
  Well done for doing that. It's empowering to bear in mind that you are interviewing them too. If your interviewer is highly unpleasant then why on earth would you want to work for them / a company that thinks its OK to be represented by them. No thanks!

11thEarlOfMar 6y ago

We use a 3 step process, sans whiteboard: 1. First interview, the candidate interviews us. What market we serve, what our development processes are, what our technology stack is. If they express interest by being prepared and asking good questions, we send them 2. a programming task. Choose 1 of 5 tasks. The tasks are not abstract problems or puzzles, but come out of the designs we've implemented. We ask for 100 lines of code and not more than 2 hours. They take as much calendar time in days as they need, most have full time jobs, and let us know when you're done. The candidate then comes back, typically in 2 to 7 days and hosts a code review in front of our team. I give strict instructions before hand: The purpose of the review is to learn how they thought through the problem and how they solved it, not to criticize their style and approach. 3. If we decide we like them to this point, the third interview is with managers from other functions. How well does the candidate communicate, come across to non-developers, express interest in the company and role, etc. It's a check point to look for concerning weaknesses, as well as get buy in from the broader organization. We like this approach because it allows the candidate to code in a much more natural environment. They can take the time they need and comfortably solve. No one spends their professional career coding on a white board. This approach so far has yielded excellent results. We ask developers how much time they took in the programming task and why they chose the one they solved. The programming task is telling, not in the quality of their code and how long they took, but how much did they get into it? We have had the range from candidates who did not complete it at all and opted out, to candidates who stumped 40 year veterans with elegant code. In every case, we learn how well they can express their thoughts, and importantly, their level of love for the discipline. This is as important to me as any other attribute.

maxwelldone 6y ago
I love this approach. As someone who generally asks a lot of questions, your first round would give one a good idea if it's worth proceeding to other rounds or not. BTW, are you guys hiring?
ammanley 6y ago
Sounds like a place worth working for
6y ago

itsdrewmiller 6y ago

Kind of disappointing that all the top rated comments here are about people's personal opinions about interviewing, and don't engage with the pdf at all. What I found most interesting was Table 1 - it shows that in the unsupervised interview the average score of women was 3/3, while in the supervised version it was less than 1/3. Small sample sizes, but still seems suggestive. (I CTRL-Fed but didn't find much discussion of it in the paper.)

dmtroyer 6y ago

Yes.

dang 6y ago

Related from last year: Does Stress Impact Technical Interview Performance? - https://news.ycombinator.com/item?id=23844367 https://news.ycombinator.com/item?id=23844367 - July 2020 (26 comments)

lubesGordi 6y ago

It'd be interesting to see the impact wrt age.

nullifidian 6y ago

Didn't "stereotype threat" fail replication? Why do researchers cite it?

neilv 6y ago

The paper is easy to read, and a bunch of questions that came to mind were addressed, or at least acknowledged, in one place or another of the paper. I appreciated the following tidbit, which I didn't know (since I figured prep for this particular test was now widespread there, by convention, but not officially stated): > students at Stanford take courses for passing technical interviews, CS9: Problem-Solving for the CS Technical Interview. And because of things like that, the current interview convention seem even less useful as a predictor, for purposes other than "Did you go to Stanford, or are of similar socioeconomic status, to have been coached on beating the current metric, specifically?"

coldcode 6y ago
If you are expected to study to pass an interview, is the result really indicative of how you will succeed in the job, or a measure of your skill is studying to pass an interview?

worker767424 6y ago

I can't say I enjoy interviews, but they're really not that bad. Reading the comments here, you'd think everyone gets paralyzing anxiety during interviews. They're a chance to show off how smart you are.

kevingadd 6y ago
I've probably done at least a few hundred tech interviews in my career, and the vast majority of them are simply not an opportunity to show anything off. You're putting up with various interviewers in various moods of varying levels of competence, some of them actively hostile or working on some sort of weird agenda. You're bouncing between interviewers and getting asked the same question multiple times and having to explain the same thing over and over. Even if they don't make you anxious (that started to subside after the first hundred times or so) they're still not a good time. I've had senior staff running interviews at big companies do things like harass me over data structure choices, insist that I was lying when I described a textbook algorithm, accuse me of industrial espionage, and all other sorts of absurd or unpleasant things. In some cases even after that I still got an offer, because the interviewers weren't out to give someone a hard time... they were just bad at running interviews. One of my all-time worst interviews involved the design director (?) at a billion-dollar game studio calling me (unscheduled!) at 7am and demanding that I come up with 50 unique uses for a paperclip. Really. The gap between an average or bad interview and a truly good one is enormous, and the number of well-conducted interviews I've had is incredibly small. I can only imagine how much worse it is for someone with additional obstacles, like how is someone in a wheelchair supposed to stand up and write C++ on a whiteboard for 30 minutes? What does the whiteboard interview look like for a blind woman?

aphextron 6y ago

Literally just reading that headline gave me cold sweats and a wave of anxiety.

confidantlake 6y ago

I am surprised that there are not more "traditional tests". Give problem, give person 45 minutes, leave the room, come back and discuss it. I have strong social anxiety. It is extremely difficult to solve a problem with someone watching me. I get panic attacks in at least 1/3 of my interviews. I have never had to code with someone watching me during my day to day job. I am not not interviewing for the job of coding performance artist. Why make me audition like I am an actor or a musician?

say_it_as_it_is 6y ago

Tech wouldn't be where it is today without antiquated interviews and an industry of human resource management built on faith and tradition rather than science. Talented people go elsewhere, create new innovation, and prosper while the large tech companies copy or acquire the innovation. You are successful not only because of what you accomplish in life but because of what you don't accomplish. You choose not to spend hundreds of hours training to answer coding problems, prioritizing other activities to advance yourself.