5 ms·

Technical Interviews Reject the Wrong Engineers

57 points by fagnerbrack 4mo ago

techblueberry 4mo ago
> The deeper issue is tacit knowledge. Most of what a skilled engineer knows is not something they can articulate on demand. Why is it that everyone says that soft skills can be more important than hard skills, that engineers talk to people they don’t sit in rooms turning requirements into code, but then, it seems like one of the criticisms about the interview process is “well, engineers can come up with good solutions when alone in a room” That’s not the job. Articulating technical details when in conversation with your colleagues is.
- dijksterhuis 4mo ago
  eh, my take is that you’re kind of going off in a specific direction when reality is actually behind you little here. i cannot explain why it’s not possible for us to fling the flange if i don’t understand how both the flange flanges and the fling flings. being able to talk about it is a downstream effect of knowing about it.
- BugsJustFindMe 4mo ago
  Articulation is not the issue; the reasoning and consideration process is. If you step back a bit from the words on that page and squint, what you might see is something like "Most of what a skilled engineer does is recognize, sense problems, and feel things that may not be obvious." The discovery of the right path forward for the goals of the organization comes after that and takes time and planning.
  - jjav 4mo ago
    > "Most of what a skilled engineer does is recognize, sense problems, and feel things that may not be obvious." This is a great way of putting it, love it. At the end of the day, that's it. A lot of people are very good at executing the tasks, delivering what needs to be delivered. But the special people are the ones who maintain a constant sense of awareness of what might be off and digging deep into it.
- mixmastamyk 4mo ago
  One returns in a week with a fully articulated solution, not at point blank. Famously, Archimedes came up with his in the bathtub.
- watwut 4mo ago
  It totally is the job. What kind of unreasonable process you have that people dont have time to think? First you think, then you think about how to say it, then you talk with others. Then you think again and maybe talk again. But, it is not like we were designing everything in quick on the spot debates without research.
  - jjav 4mo ago
    > What kind of unreasonable process you have that people dont have time to think? AI is making this much worse. Executives expect you to be managing a herd of AI agents working on a dozen things at once, 24/7. Good luck trying to find time to think. This will backfire.
- fagnerbrack 4mo ago
  Tacit knowledge is not soft skill, there's quite a comprehensive field of neuroscience research around it: https://en.wikipedia.org/wiki/Tacit_knowledge https://en.wikipedia.org/wiki/Tacit_knowledge
a-dub 4mo ago
here's an idea for an "experienced" technical interview structure. "we care about x, y and z. you have forty five minutes to convince us that you will meaningful help us achieve our goals. we then will take 30 minutes to push on technical details as we see fit. you will be judged based on technical content, choices, taste and your overall approach and strategy for moving us forward and convincing us that you're the right person to do it. good luck!"
- paulluuk 4mo ago
  I wonder how many great engineers with stage fright you would lose this way, though.
- BugsJustFindMe 4mo ago
  > we care about x, y and z. you have forty five minutes to convince us that you will meaningful help us achieve our goals. we then will take 30 minutes to push on technical details This part is good. > as we see fit. you will be judged based on technical content, choices, taste and your overall approach and strategy for moving us forward and convincing us that you're the right person to do it. good luck! This part is confrontational and will lead to worse outcomes. "As we see fit" signals capriciousness. "You will be judged" signals hostility. "you're the right person" engineers conflict; there is no "the right" person, only "a good" person. And telling someone "good luck!" in this context is like telling them to try not to die while standing on a narrow plank over a pit of sharp spikes. No matter how you think you mean it, it will come across as callous to many.
  - FireBeyond 4mo ago
    I do agree with your assessment, on both parts, but I think it could survive with minor tweaking. "We are going to push on the technical details, both those we agree with and disagree with. This is where the rubber meets the road. You've painted a picture, how does it stand up to scrutiny and challenge. In the end, while there may not be perfect alignment, we're looking to see how your ideas go through the process of validation and how we get there, communication, exploration, and collaboration-wise."
zipy124 4mo ago
The main problem is good engineers have no need to sit through your 12 step process. It actively selects only for the most desperate or money driven people (if you pay very well).
- dangus 4mo ago
  This point gets repeated a lot as if we are supposed to coddle engineers by making interviews wildly easy. At some point as an employer you do want someone who is motivated enough to take some time out of their day to prepare for an interview. Do you really want an employee who gives so little of a shit that they refuse to use their brain to get a job? This isn’t exactly a hot labor market in tech. Companies have a good selection of quality talent available right now.
  - newtonianrules 4mo ago
    I don’t want to put my future coworker through six rounds of interviews. If it takes more than three rounds + a phone screen to figure out if someone is a good fit then the process is broken.
    - dangus 4mo ago
      Depends how long the rounds are. 6 rounds of 20 minutes is only 2 hours. If you think that’s unreasonable, please go ahead and add a few fire sauce packets to the bag for me.
      - newtonianrules 4mo ago
        How many interviews have you been on that a round is 20 minutes?
      - angiolillo 4mo ago
        Whether it's reasonable depends on the distribution, not just the duration. A 2 hour onsite with the candidate being rapid-fire interviewed by six different different teams and a 20 minute call every couple weeks for three months are very different (and select for very different types of candidates) despite having the same overall duration.
IshKebab 4mo ago
Of course. This isn't a secret. But nobody has come up with anything better. Doesn't seem like this guy is proposing anything either.
kadhirvelm 4mo ago
We’ve added collaboration and communication as big facets in our hiring loop rubrics, and reduced the complexity of our questions. We also tell candidates this ahead of time - been honestly working pretty well so far, it indexes more on those soft skills and gives us just enough insight into the hard skills to make a call. We’ve also been increasingly finding very few candidates know how to solve a complex question these days without LLM support lol. How are other folks changing their loops these days?
- root-parent 4mo ago
  >> few candidates know how to solve a complex question these days without LLM support lol Please tell me you are joking.
  - myself248 4mo ago
    If you're hiring from the town where Kool-Aid is headquartered, don't be surprised if most of the candidates drink a lot of it.
  - kadhirvelm 4mo ago
    I wish, but maybe it’s more fair to say how people think has changed. It’s like for these complex questions, they can break the problem down quickly into smaller functions, but implementing the functions is slow? Sometimes a non-starter? Idk, it’s different. Our old ways of interviewing aren’t working the same
mathisfun123 4mo ago
don't you people get tired of reposting this take? don't you realize it's exactly like "attractive women reject the wrong suitors" ???
- bryanrasmussen 4mo ago
  I think I find someone sexy and fun is obviously more a personal feeling on matters while I am looking for someone who will help me meet our goals for Q4 with a high degree of technical excellence seems something that should be measurable and not left up to a feeling at the moment. So I guess I don't realize it's exactly like, I personally I feel it is significantly different.
dkarl 4mo ago
This article repeats what we've long known about how technical interviews aren't great at evaluating technical skills and inadvertently filter for things that aren't important. But it doesn't offer a better way of evaluating technical skills. It talks about how to evaluate other things that do matter but aren't substitutes or proxies for technical skills. Also, this argument is some grade school smarty pants "I'm too smart to show my work" bullshit: > And because the interviewer can’t distinguish “skipped steps due to incompetence” from “skipped steps due to operating at a higher cognitive level,” they default to the interpretation that protects their ego. I thought the days of hiring toxic "so smart I can't communicate" superstars was over?
- jghn 4mo ago
  I know this is an outlier, unpopular opinion by my experience has been that the percentage of people who can really talk in depth through technical details but can't write decent code is quite small. But the percentage of people who can talk the talk but can't write code or solve puzzles on the spot in an interview environment is much higher. Everyone has their one favorite anecdote about the one person who slipped through the cracks and singlehandedly brought their company to its knees. But I contend that people spend far more energy defending against this use case than the reality warrants.
- seanmcdirmid 4mo ago
  There are many steps so you can’t show them all (each step has sub steps and so on). You prioritize steps, but when you are showing your work you focus on steps that are key for you and think would be key for your interviewer. It isn’t crazy that you would guess wrong about the latter given you don’t really know them. A reasonable candidate can definitely make the mistake of not showing steps that they think aren’t important (and they can also go the other way and show way too many unimportant steps).
- JohnMakin 4mo ago
  Someone like this may be skipping steps because to them the steps may be so obvious as to not need to be explained, and they are giving more benefit of the doubt to the interviewer than they should. your framing makes it seem like arrogance when in the vast majority of the time I’ve seen this, it’s the candidate making the assumption the person on the other side is their peer.
pelagicAustral 4mo ago
I don't understand why can't someone just come up with an interview where they propose an issue that is truly something that can be expected in the role, give the applicant a few hours to come up with an action plan and a solution architecture, put it down in writing, sketch some graphs, and then present it... Why is that so groundbraking? wouldn't that immediately tell you a lot about the applicant, regardless of the final implementation being a 10 or not?
- bradlys 4mo ago
  That’s a system design interview. Those exist already and are about 45 minutes in length.
  - jghn 4mo ago
    This style of panel is great. The one thing interviewers need to stay aware of though is that they will have immense insight into the problem whereas the candidate likely has never thought through the problem space before. And this can lead to unfair judgements. I know I've fallen into this trap before, where I'd run the same session with so many candidates that I had a thorough big picture view of every possible approach. I'd see a candidate stumbling a bit and think, "Why can't they see the obvious answer in front of them?" when in fact I wouldn't have seen it on my first go either. I just had seen dozens of candidates go through the same exercise to the point where it seemed like old hat.
- jjmarr 4mo ago
  These are called take-home interviews. They exist and applicants HATE THEM for being too long/"free work".
  - pelagicAustral 4mo ago
    Well, yeah I know about take-homes, I mean 2 3 hours tops. In-office.
  - Aurornis 4mo ago
    In my experience most candidates actually love them. They prefer the lower pressure and better relevance to the job. If you give candidates the choice between a short take home or a 1-hour on-site technical round, almost everyone picks the take home. We’ve tried it. It’s only online that you get the rants about “free work”, but the people making those complaints usually hate on-site technical interviews equally.
andrewstuart 4mo ago
Even harder with AI. How do you assess developers when AI makes it possible to create software without even knowing the language?
- robofanatic 4mo ago
  Give an abstract requirement and access to your AI tool. Ask the candidate to create a working solution and review the AI generated code. requirement analysis and code review are now the primary skills for developers.
  - andrewstuart 4mo ago
    Interesting. I don’t think code review of AI is important at all - specially given the developer might not even know the language, in which case it’s irrelevant. I think ability to build checking, verification, linting, testing and cross LLM code review mechanisms is important, which ensure that fast changing or unpredictably changing ai code has consistent behavior and is security checked.
HarHarVeryFunny 4mo ago
The main goal of hiring someone should be to assess how well they can do the job you are hiring for, and for anyone with job experience (i.e. not fresh out of college) the best indicator of that is what have they previously achieved (especially in more recent years). How well someone can solve a whiteboard challenge or brainteaser is irrelevant unless you are hiring someone to solve 10min whiteboard challenges. Of course the difficulty is how do you assess what the candidate has honestly achieved in prior jobs - what was there personal contribution, and do they seem to have approached things in a way that suggests transferable/general skills that can be applied to the position you are hiring for. The graphic at the beginning of the article is certainly one problem - the interviewer needs to themselves have the experience to assess the candidate. There is no point having someone with 5 years of experience interview someone with 20 years, assuming you are hiring them for that level of experience. I think the best interviewing technique is just to go over the candidates experience, from most recent to as far back as you think is relevant, and get them to talk about each project. What was their role, what were their contributions, why did they make the decisions they did, what might they have done differently, etc. etc. Ask them to summarize and deep dive into the architecture, draw it on a whiteboard perhaps.
- tptacek 4mo ago
  Experience-based interviews are a fantastic way to select for candidates who have "failed up" through a long series of jobs. The underlying dynamic is that it usually takes more than a year to sever a technical employee; you can faceplant in a role and still wind up with a resume improvement.
  - skydhash 4mo ago
    Experience as in numbers of years or project participation is flawed. But experience as in contributions and knowledge of some specific domains is good IMO.
  - sanderjd 4mo ago
    I hear this a lot, but man, I just really don't think so. First of all, 1-year stints come off very poorly in this kind of discussion. (I would say that a bias against people with a bunch of short stints would be a failure mode of "experience-based interviews" rather than the failure mode you're describing.) But also, I have had these discussions with (in my judgement) both "failed up" candidates and "tons of valuable experience" candidates, and I just don't have this experience that it's difficult to differentiate. I'm fully aware that according to the internet, there is an epidemic of bullshit artists who can go deep on the architecture and tradeoffs and their contribution to the things they worked on, without having actually contributed to those things, but I dunno, the narrative just doesn't jibe with my anecdotal experience. My only uncertainty here is that I do think I have been very fortunate in the people I have worked with in my career, so I might just be getting lucky. But I have truly never worked with someone who is hired largely on the basis of a strong resume, but genuinely can't grok fizzbuzz, or whatever the contemporary equivalent of that is. I also recognize that you ran a real company doing real work on this, so I'm generally inclined to defer to your wisdom... You can tell that I'm very torn on this, because the conventional wisdom is so strongly against my perspective on it, and I generally put a good deal of weight on conventional wisdom. But man, I dunno, it all really feels like an inertia thing that I increasingly question the original foundations of. Sometimes the emperor actually doesn't have any clothes on...
sanderjd 4mo ago
I was really hoping for a conclusion section with pragmatic suggestions for how to apply these insights to a real hiring process. I'm left thinking "ok this all makes sense, but what do I do with it?".
tptacek 4mo ago
This post is 2/3rds about accuracy problems with interviews (and, yes, I think the weight of the evidence is that interviews are effectively random functions), and then it culminates in a "FATE" metric that is mostly not about accuracy. Giving candidates feedback is good, but it has nothing to do with the likelihood or cost of a bad hire. Making timely decisions is good ceteris paribus, but it's of secondary importance compared to selecting the right candidate. Same with "effort".
m_m_carvalho 4mo ago
As a former IT instructor, I've seen students who performed brilliantly in exams and interviews but struggled when faced with real-world ambiguity. I've also seen quieter people who were average in interviews become excellent builders once they had a real problem to solve. Interviews are useful, but the ability to ship, maintain and improve real projects should probably carry more weight.
- dust-jacket 4mo ago
  > Interviews are useful, but the ability to ship, maintain and improve real projects should probably carry more weight. But how do you assess this? Maybe we should get them to write a document that details what they've done, and then invite them to a conversation to discuss it. Oh wait...
TheGRS 4mo ago
I'm thinking of a technical screen I did recently where I didn't move forward. The time to do the screen was 30 minutes, and it was where they had a full frontend/backend and I needed to navigate around to fix a pretty arbitrary issue. I'd say this is preferable to a leetcode problem for sure, but also, I do tend to take my time to understand the system a bit before committing to changes, I mean this is sight unseen. I'm wondering if this felt too slow to the interviewer. They sent me a summary document before the meeting, but I couldn't see the code until the interview. I felt like I identified the issue and where to make changes rather quickly, like 10 minutes of looking around and talking through how all the components and APIs fit together. Then the interviewer asked me to implement a datetime solution, which in this time-boxed window my mind raced around to multiple solutions that I talked through out loud: I could write it myself which would definitely take some time for me to remember all of the syntax involved and reason through the problem, I could download an existing library which would also take some time to read documentation, I could google around for existing solutions in somewhere like Stack Overflow which is pretty hit and miss, or I could prompt an AI agent to write a solution for me. I talked through all of these, they wanted to know how I'd do it by hand at first, which I talked through for a bit but admitted I wasn't sure if it was a good way to go about it. Then I said given the time constraints the AI prompt route would probably make the most sense. By the time we arrived at that and tried it for a bit our time was basically up. And I got the impression suggesting AI to help code didn't impress the interviewer at all. If others are able to stand out in this scenario then I guess I'll just admit I'm not the top candidate. My brain just doesn't work that quickly. I like to spend time gathering context and tinkering before really getting into the solution, and that probably doesn't come across well in these situations.
- glaslong 4mo ago
  It often selects for a confident first shot, which is why we see these orgs drift towards lots of engineers who can blast out code but cannot maintain or evolve any existing systems proficiently. On rare occasion that is even the hiring goal!
  - lubujackson 4mo ago
    This is the exact problem. Some people can flip between one shotting for interviews and going deep for real work, but they are extremely rare. As a senior, I would much prefer a candidate who can discuss options more than write code - the writing itself is secondary, especially with AI. I want to see someone grapple with tradeoffs, clarify what they know and what breadcrumbs they want to follow before committing to a solution.
dzonga 4mo ago
let's say you make a saas for automatic birdfeeders. if you can scope out your problem to a simplified version & have a pen & paper / whiteboard discussion with a candidate. the candidate starts from a blank slate. they create their own constraints, validate their own assumptions etc. if they can't design something that works - u can prove this since you already have a working product in production. that way interview takes place in less than 1:30hr. saving time for u & candidate. you cover: communication, critical thinking, technical chops. 3 birds with one stone. but unfortunately most people don't want to do that - because 1. their products r fugazi (fueled by vc money), they're on ego trips (we gonna scale) & lastly want to make interviewing a hazing|humiliation ritual.
- stormking 4mo ago
  Someone else here already pointed out the problem with that approach: After the second or third candidate, the interviewer is very familiar with the problem and may become unfairly judgemental with later candidates if they don't immediately see "the obvious". Which wasn't obvious at all to him as well, two weeks ago.
pcan77 4mo ago
I love how people in this thread are ABSOLUTELY BAFFLED at "how do we do this a better way." How about the same way doctors, lawyers, and literally everyone else does it? Look at credentials, schooling, past experience, references and personality interviews rather than 99% leetcode? I have lawyer, doctor, etc friends (aka high up professionals who get paid what we do or more) and they think it's absolute insanity what SOFTWARE engineers have to do to get a freaking job. They think it's asinine, quite frankly and all go "well what did you even go to school for? Don't they know you worked at a Fortune 500 company for over a decade? Why are they having you do trick questions from CS101 courses?" Y'all, it's not that difficult. You can just pretend we're like everyone else because gasp our profession simply isn't that special like we all think it is for some stupid reason.
- eudamoniac 4mo ago
  This doesn't work when there are terrible devs somehow stealing paychecks from big companies for years, not even mentioning interview fraud. I've interviewed credentialed people who can't write fizzbuzz. Until tenure and credentials indicate skill, they can't be used! Some companies I think do indicate skill, like a L5 at Google can do fizzbuzz I'm sure, but somewhere like Cisco or random small startup might as well be nowhere. As for education, no CS program in the country is a guarantee.
  - jghn 4mo ago
    This is such a tired canard. Do these people exist? Yes. Assuming you can hold an in depth, detailed technical conversation with them I'd contend the percentage of these people is astonishingly low. We spend way too much mental energy defending against them. The real problem is not enough interviewers can discuss in depth technical bits themselves, and thus they can't get signal when talking to a candidate.
    - eudamoniac 4mo ago
      You can contend whatever you want, but I interview people all the time who have passed a phone screen, and your contention is wrong.
SatvikBeri 4mo ago
Every interview method has some glaring flaws, and I find you mostly have a choice of which flaws you pick. On my current team, I care a lot about the ability to write fast code. The most important part of my process is a take-home designed to take 2 hours where the main goal is to solve a relatively easy problem as performantly as possible. Answers have varied from 0.2ms – 50ms. Take-homes have some obvious disadvantages, but overall I find they're better at finding the people we're looking for than just about every other method. But I'm at a small company, hiring for a fairly specialized team. If the situation was different (e.g. I needed to hire 50 people/year) I'd use a much more standard process.
- JamesSwift 4mo ago
  Hmm thats a pretty interesting variant. I dont think it would apply directly to our team, but I like the potential playground it gives for both coming up with solutions as well as being a kickstarter for convos on alternatives.
mattgrice 4mo ago
I think there is a misconception that FAANG type coding interviews are trying to find stars. They aren't, otherwise they would not cap the difficulty and have a bank of approved questions which can be crammed via leetcode. They are testing for diligence just like exams in Confucian based systems.
type0 4mo ago
> Some companies try to add science to the process with personality assessments Soon they'll add phrenology assessments and will call it science
readthenotes1 4mo ago
"They found that avoiding a toxic worker generates roughly twice the return of hiring a star performer." This is very true and one of the easiest ways to find that out is to put people into a pressure situation and see how they respond. And of course, you need to find out if someone has a famous clue of how to go about solving problems.
relaxing 4mo ago
[dead]

57 points by fagnerbrack 4mo ago

> The deeper issue is tacit knowledge. Most of what a skilled engineer knows is not something they can articulate on demand. Why is it that everyone says that soft skills can be more important than hard skills, that engineers talk to people they don’t sit in rooms turning requirements into code, but then, it seems like one of the criticisms about the interview process is “well, engineers can come up with good solutions when alone in a room” That’s not the job. Articulating technical details when in conversation with your colleagues is.

dijksterhuis 4mo ago
eh, my take is that you’re kind of going off in a specific direction when reality is actually behind you little here. i cannot explain why it’s not possible for us to fling the flange if i don’t understand how both the flange flanges and the fling flings. being able to talk about it is a downstream effect of knowing about it.
BugsJustFindMe 4mo ago
Articulation is not the issue; the reasoning and consideration process is. If you step back a bit from the words on that page and squint, what you might see is something like "Most of what a skilled engineer does is recognize, sense problems, and feel things that may not be obvious." The discovery of the right path forward for the goals of the organization comes after that and takes time and planning.
- jjav 4mo ago
  > "Most of what a skilled engineer does is recognize, sense problems, and feel things that may not be obvious." This is a great way of putting it, love it. At the end of the day, that's it. A lot of people are very good at executing the tasks, delivering what needs to be delivered. But the special people are the ones who maintain a constant sense of awareness of what might be off and digging deep into it.
mixmastamyk 4mo ago
One returns in a week with a fully articulated solution, not at point blank. Famously, Archimedes came up with his in the bathtub.
watwut 4mo ago
It totally is the job. What kind of unreasonable process you have that people dont have time to think? First you think, then you think about how to say it, then you talk with others. Then you think again and maybe talk again. But, it is not like we were designing everything in quick on the spot debates without research.
- jjav 4mo ago
  > What kind of unreasonable process you have that people dont have time to think? AI is making this much worse. Executives expect you to be managing a herd of AI agents working on a dozen things at once, 24/7. Good luck trying to find time to think. This will backfire.
fagnerbrack 4mo ago
Tacit knowledge is not soft skill, there's quite a comprehensive field of neuroscience research around it: https://en.wikipedia.org/wiki/Tacit_knowledge https://en.wikipedia.org/wiki/Tacit_knowledge

a-dub 4mo ago

here's an idea for an "experienced" technical interview structure. "we care about x, y and z. you have forty five minutes to convince us that you will meaningful help us achieve our goals. we then will take 30 minutes to push on technical details as we see fit. you will be judged based on technical content, choices, taste and your overall approach and strategy for moving us forward and convincing us that you're the right person to do it. good luck!"

paulluuk 4mo ago
I wonder how many great engineers with stage fright you would lose this way, though.
BugsJustFindMe 4mo ago
> we care about x, y and z. you have forty five minutes to convince us that you will meaningful help us achieve our goals. we then will take 30 minutes to push on technical details This part is good. > as we see fit. you will be judged based on technical content, choices, taste and your overall approach and strategy for moving us forward and convincing us that you're the right person to do it. good luck! This part is confrontational and will lead to worse outcomes. "As we see fit" signals capriciousness. "You will be judged" signals hostility. "you're the right person" engineers conflict; there is no "the right" person, only "a good" person. And telling someone "good luck!" in this context is like telling them to try not to die while standing on a narrow plank over a pit of sharp spikes. No matter how you think you mean it, it will come across as callous to many.
- FireBeyond 4mo ago
  I do agree with your assessment, on both parts, but I think it could survive with minor tweaking. "We are going to push on the technical details, both those we agree with and disagree with. This is where the rubber meets the road. You've painted a picture, how does it stand up to scrutiny and challenge. In the end, while there may not be perfect alignment, we're looking to see how your ideas go through the process of validation and how we get there, communication, exploration, and collaboration-wise."

zipy124 4mo ago

The main problem is good engineers have no need to sit through your 12 step process. It actively selects only for the most desperate or money driven people (if you pay very well).

dangus 4mo ago
This point gets repeated a lot as if we are supposed to coddle engineers by making interviews wildly easy. At some point as an employer you do want someone who is motivated enough to take some time out of their day to prepare for an interview. Do you really want an employee who gives so little of a shit that they refuse to use their brain to get a job? This isn’t exactly a hot labor market in tech. Companies have a good selection of quality talent available right now.
- newtonianrules 4mo ago
  I don’t want to put my future coworker through six rounds of interviews. If it takes more than three rounds + a phone screen to figure out if someone is a good fit then the process is broken.
  - dangus 4mo ago
    Depends how long the rounds are. 6 rounds of 20 minutes is only 2 hours. If you think that’s unreasonable, please go ahead and add a few fire sauce packets to the bag for me.
    - newtonianrules 4mo ago
      How many interviews have you been on that a round is 20 minutes?
    - angiolillo 4mo ago
      Whether it's reasonable depends on the distribution, not just the duration. A 2 hour onsite with the candidate being rapid-fire interviewed by six different different teams and a 20 minute call every couple weeks for three months are very different (and select for very different types of candidates) despite having the same overall duration.

IshKebab 4mo ago

Of course. This isn't a secret. But nobody has come up with anything better. Doesn't seem like this guy is proposing anything either.

kadhirvelm 4mo ago

We’ve added collaboration and communication as big facets in our hiring loop rubrics, and reduced the complexity of our questions. We also tell candidates this ahead of time - been honestly working pretty well so far, it indexes more on those soft skills and gives us just enough insight into the hard skills to make a call. We’ve also been increasingly finding very few candidates know how to solve a complex question these days without LLM support lol. How are other folks changing their loops these days?

root-parent 4mo ago
>> few candidates know how to solve a complex question these days without LLM support lol Please tell me you are joking.
- myself248 4mo ago
  If you're hiring from the town where Kool-Aid is headquartered, don't be surprised if most of the candidates drink a lot of it.
- kadhirvelm 4mo ago
  I wish, but maybe it’s more fair to say how people think has changed. It’s like for these complex questions, they can break the problem down quickly into smaller functions, but implementing the functions is slow? Sometimes a non-starter? Idk, it’s different. Our old ways of interviewing aren’t working the same

mathisfun123 4mo ago

don't you people get tired of reposting this take? don't you realize it's exactly like "attractive women reject the wrong suitors" ???

bryanrasmussen 4mo ago
I think I find someone sexy and fun is obviously more a personal feeling on matters while I am looking for someone who will help me meet our goals for Q4 with a high degree of technical excellence seems something that should be measurable and not left up to a feeling at the moment. So I guess I don't realize it's exactly like, I personally I feel it is significantly different.

dkarl 4mo ago

This article repeats what we've long known about how technical interviews aren't great at evaluating technical skills and inadvertently filter for things that aren't important. But it doesn't offer a better way of evaluating technical skills. It talks about how to evaluate other things that do matter but aren't substitutes or proxies for technical skills. Also, this argument is some grade school smarty pants "I'm too smart to show my work" bullshit: > And because the interviewer can’t distinguish “skipped steps due to incompetence” from “skipped steps due to operating at a higher cognitive level,” they default to the interpretation that protects their ego. I thought the days of hiring toxic "so smart I can't communicate" superstars was over?

jghn 4mo ago
I know this is an outlier, unpopular opinion by my experience has been that the percentage of people who can really talk in depth through technical details but can't write decent code is quite small. But the percentage of people who can talk the talk but can't write code or solve puzzles on the spot in an interview environment is much higher. Everyone has their one favorite anecdote about the one person who slipped through the cracks and singlehandedly brought their company to its knees. But I contend that people spend far more energy defending against this use case than the reality warrants.
seanmcdirmid 4mo ago
There are many steps so you can’t show them all (each step has sub steps and so on). You prioritize steps, but when you are showing your work you focus on steps that are key for you and think would be key for your interviewer. It isn’t crazy that you would guess wrong about the latter given you don’t really know them. A reasonable candidate can definitely make the mistake of not showing steps that they think aren’t important (and they can also go the other way and show way too many unimportant steps).
JohnMakin 4mo ago
Someone like this may be skipping steps because to them the steps may be so obvious as to not need to be explained, and they are giving more benefit of the doubt to the interviewer than they should. your framing makes it seem like arrogance when in the vast majority of the time I’ve seen this, it’s the candidate making the assumption the person on the other side is their peer.

pelagicAustral 4mo ago

I don't understand why can't someone just come up with an interview where they propose an issue that is truly something that can be expected in the role, give the applicant a few hours to come up with an action plan and a solution architecture, put it down in writing, sketch some graphs, and then present it... Why is that so groundbraking? wouldn't that immediately tell you a lot about the applicant, regardless of the final implementation being a 10 or not?

bradlys 4mo ago
That’s a system design interview. Those exist already and are about 45 minutes in length.
- jghn 4mo ago
  This style of panel is great. The one thing interviewers need to stay aware of though is that they will have immense insight into the problem whereas the candidate likely has never thought through the problem space before. And this can lead to unfair judgements. I know I've fallen into this trap before, where I'd run the same session with so many candidates that I had a thorough big picture view of every possible approach. I'd see a candidate stumbling a bit and think, "Why can't they see the obvious answer in front of them?" when in fact I wouldn't have seen it on my first go either. I just had seen dozens of candidates go through the same exercise to the point where it seemed like old hat.
jjmarr 4mo ago
These are called take-home interviews. They exist and applicants HATE THEM for being too long/"free work".
- pelagicAustral 4mo ago
  Well, yeah I know about take-homes, I mean 2 3 hours tops. In-office.
- Aurornis 4mo ago
  In my experience most candidates actually love them. They prefer the lower pressure and better relevance to the job. If you give candidates the choice between a short take home or a 1-hour on-site technical round, almost everyone picks the take home. We’ve tried it. It’s only online that you get the rants about “free work”, but the people making those complaints usually hate on-site technical interviews equally.

andrewstuart 4mo ago

Even harder with AI. How do you assess developers when AI makes it possible to create software without even knowing the language?

robofanatic 4mo ago
Give an abstract requirement and access to your AI tool. Ask the candidate to create a working solution and review the AI generated code. requirement analysis and code review are now the primary skills for developers.
- andrewstuart 4mo ago
  Interesting. I don’t think code review of AI is important at all - specially given the developer might not even know the language, in which case it’s irrelevant. I think ability to build checking, verification, linting, testing and cross LLM code review mechanisms is important, which ensure that fast changing or unpredictably changing ai code has consistent behavior and is security checked.

HarHarVeryFunny 4mo ago

The main goal of hiring someone should be to assess how well they can do the job you are hiring for, and for anyone with job experience (i.e. not fresh out of college) the best indicator of that is what have they previously achieved (especially in more recent years). How well someone can solve a whiteboard challenge or brainteaser is irrelevant unless you are hiring someone to solve 10min whiteboard challenges. Of course the difficulty is how do you assess what the candidate has honestly achieved in prior jobs - what was there personal contribution, and do they seem to have approached things in a way that suggests transferable/general skills that can be applied to the position you are hiring for. The graphic at the beginning of the article is certainly one problem - the interviewer needs to themselves have the experience to assess the candidate. There is no point having someone with 5 years of experience interview someone with 20 years, assuming you are hiring them for that level of experience. I think the best interviewing technique is just to go over the candidates experience, from most recent to as far back as you think is relevant, and get them to talk about each project. What was their role, what were their contributions, why did they make the decisions they did, what might they have done differently, etc. etc. Ask them to summarize and deep dive into the architecture, draw it on a whiteboard perhaps.

tptacek 4mo ago
Experience-based interviews are a fantastic way to select for candidates who have "failed up" through a long series of jobs. The underlying dynamic is that it usually takes more than a year to sever a technical employee; you can faceplant in a role and still wind up with a resume improvement.
- skydhash 4mo ago
  Experience as in numbers of years or project participation is flawed. But experience as in contributions and knowledge of some specific domains is good IMO.
- sanderjd 4mo ago
  I hear this a lot, but man, I just really don't think so. First of all, 1-year stints come off very poorly in this kind of discussion. (I would say that a bias against people with a bunch of short stints would be a failure mode of "experience-based interviews" rather than the failure mode you're describing.) But also, I have had these discussions with (in my judgement) both "failed up" candidates and "tons of valuable experience" candidates, and I just don't have this experience that it's difficult to differentiate. I'm fully aware that according to the internet, there is an epidemic of bullshit artists who can go deep on the architecture and tradeoffs and their contribution to the things they worked on, without having actually contributed to those things, but I dunno, the narrative just doesn't jibe with my anecdotal experience. My only uncertainty here is that I do think I have been very fortunate in the people I have worked with in my career, so I might just be getting lucky. But I have truly never worked with someone who is hired largely on the basis of a strong resume, but genuinely can't grok fizzbuzz, or whatever the contemporary equivalent of that is. I also recognize that you ran a real company doing real work on this, so I'm generally inclined to defer to your wisdom... You can tell that I'm very torn on this, because the conventional wisdom is so strongly against my perspective on it, and I generally put a good deal of weight on conventional wisdom. But man, I dunno, it all really feels like an inertia thing that I increasingly question the original foundations of. Sometimes the emperor actually doesn't have any clothes on...

sanderjd 4mo ago

I was really hoping for a conclusion section with pragmatic suggestions for how to apply these insights to a real hiring process. I'm left thinking "ok this all makes sense, but what do I do with it?".

tptacek 4mo ago

This post is 2/3rds about accuracy problems with interviews (and, yes, I think the weight of the evidence is that interviews are effectively random functions), and then it culminates in a "FATE" metric that is mostly not about accuracy. Giving candidates feedback is good, but it has nothing to do with the likelihood or cost of a bad hire. Making timely decisions is good ceteris paribus, but it's of secondary importance compared to selecting the right candidate. Same with "effort".

m_m_carvalho 4mo ago

As a former IT instructor, I've seen students who performed brilliantly in exams and interviews but struggled when faced with real-world ambiguity. I've also seen quieter people who were average in interviews become excellent builders once they had a real problem to solve. Interviews are useful, but the ability to ship, maintain and improve real projects should probably carry more weight.

dust-jacket 4mo ago
> Interviews are useful, but the ability to ship, maintain and improve real projects should probably carry more weight. But how do you assess this? Maybe we should get them to write a document that details what they've done, and then invite them to a conversation to discuss it. Oh wait...

TheGRS 4mo ago

I'm thinking of a technical screen I did recently where I didn't move forward. The time to do the screen was 30 minutes, and it was where they had a full frontend/backend and I needed to navigate around to fix a pretty arbitrary issue. I'd say this is preferable to a leetcode problem for sure, but also, I do tend to take my time to understand the system a bit before committing to changes, I mean this is sight unseen. I'm wondering if this felt too slow to the interviewer. They sent me a summary document before the meeting, but I couldn't see the code until the interview. I felt like I identified the issue and where to make changes rather quickly, like 10 minutes of looking around and talking through how all the components and APIs fit together. Then the interviewer asked me to implement a datetime solution, which in this time-boxed window my mind raced around to multiple solutions that I talked through out loud: I could write it myself which would definitely take some time for me to remember all of the syntax involved and reason through the problem, I could download an existing library which would also take some time to read documentation, I could google around for existing solutions in somewhere like Stack Overflow which is pretty hit and miss, or I could prompt an AI agent to write a solution for me. I talked through all of these, they wanted to know how I'd do it by hand at first, which I talked through for a bit but admitted I wasn't sure if it was a good way to go about it. Then I said given the time constraints the AI prompt route would probably make the most sense. By the time we arrived at that and tried it for a bit our time was basically up. And I got the impression suggesting AI to help code didn't impress the interviewer at all. If others are able to stand out in this scenario then I guess I'll just admit I'm not the top candidate. My brain just doesn't work that quickly. I like to spend time gathering context and tinkering before really getting into the solution, and that probably doesn't come across well in these situations.

glaslong 4mo ago
It often selects for a confident first shot, which is why we see these orgs drift towards lots of engineers who can blast out code but cannot maintain or evolve any existing systems proficiently. On rare occasion that is even the hiring goal!
- lubujackson 4mo ago
  This is the exact problem. Some people can flip between one shotting for interviews and going deep for real work, but they are extremely rare. As a senior, I would much prefer a candidate who can discuss options more than write code - the writing itself is secondary, especially with AI. I want to see someone grapple with tradeoffs, clarify what they know and what breadcrumbs they want to follow before committing to a solution.

dzonga 4mo ago

let's say you make a saas for automatic birdfeeders. if you can scope out your problem to a simplified version & have a pen & paper / whiteboard discussion with a candidate. the candidate starts from a blank slate. they create their own constraints, validate their own assumptions etc. if they can't design something that works - u can prove this since you already have a working product in production. that way interview takes place in less than 1:30hr. saving time for u & candidate. you cover: communication, critical thinking, technical chops. 3 birds with one stone. but unfortunately most people don't want to do that - because 1. their products r fugazi (fueled by vc money), they're on ego trips (we gonna scale) & lastly want to make interviewing a hazing|humiliation ritual.

stormking 4mo ago
Someone else here already pointed out the problem with that approach: After the second or third candidate, the interviewer is very familiar with the problem and may become unfairly judgemental with later candidates if they don't immediately see "the obvious". Which wasn't obvious at all to him as well, two weeks ago.

pcan77 4mo ago

I love how people in this thread are ABSOLUTELY BAFFLED at "how do we do this a better way." How about the same way doctors, lawyers, and literally everyone else does it? Look at credentials, schooling, past experience, references and personality interviews rather than 99% leetcode? I have lawyer, doctor, etc friends (aka high up professionals who get paid what we do or more) and they think it's absolute insanity what SOFTWARE engineers have to do to get a freaking job. They think it's asinine, quite frankly and all go "well what did you even go to school for? Don't they know you worked at a Fortune 500 company for over a decade? Why are they having you do trick questions from CS101 courses?" Y'all, it's not that difficult. You can just pretend we're like everyone else because gasp our profession simply isn't that special like we all think it is for some stupid reason.

eudamoniac 4mo ago
This doesn't work when there are terrible devs somehow stealing paychecks from big companies for years, not even mentioning interview fraud. I've interviewed credentialed people who can't write fizzbuzz. Until tenure and credentials indicate skill, they can't be used! Some companies I think do indicate skill, like a L5 at Google can do fizzbuzz I'm sure, but somewhere like Cisco or random small startup might as well be nowhere. As for education, no CS program in the country is a guarantee.
- jghn 4mo ago
  This is such a tired canard. Do these people exist? Yes. Assuming you can hold an in depth, detailed technical conversation with them I'd contend the percentage of these people is astonishingly low. We spend way too much mental energy defending against them. The real problem is not enough interviewers can discuss in depth technical bits themselves, and thus they can't get signal when talking to a candidate.
  - eudamoniac 4mo ago
    You can contend whatever you want, but I interview people all the time who have passed a phone screen, and your contention is wrong.

SatvikBeri 4mo ago

Every interview method has some glaring flaws, and I find you mostly have a choice of which flaws you pick. On my current team, I care a lot about the ability to write fast code. The most important part of my process is a take-home designed to take 2 hours where the main goal is to solve a relatively easy problem as performantly as possible. Answers have varied from 0.2ms – 50ms. Take-homes have some obvious disadvantages, but overall I find they're better at finding the people we're looking for than just about every other method. But I'm at a small company, hiring for a fairly specialized team. If the situation was different (e.g. I needed to hire 50 people/year) I'd use a much more standard process.

JamesSwift 4mo ago
Hmm thats a pretty interesting variant. I dont think it would apply directly to our team, but I like the potential playground it gives for both coming up with solutions as well as being a kickstarter for convos on alternatives.

mattgrice 4mo ago

I think there is a misconception that FAANG type coding interviews are trying to find stars. They aren't, otherwise they would not cap the difficulty and have a bank of approved questions which can be crammed via leetcode. They are testing for diligence just like exams in Confucian based systems.

type0 4mo ago

> Some companies try to add science to the process with personality assessments Soon they'll add phrenology assessments and will call it science

readthenotes1 4mo ago

"They found that avoiding a toxic worker generates roughly twice the return of hiring a star performer." This is very true and one of the easiest ways to find that out is to put people into a pressure situation and see how they respond. And of course, you need to find out if someone has a famous clue of how to go about solving problems.

relaxing 4mo ago

[dead]