5 ms·

Why are interviews harder than the job?

56 points by mooreds 1y ago

billy99k 1y ago
I prefer take home, and then having a conversation about what is done, reasoning, etc.
- galdor 1y ago
  It feels like the age of AI has made take home coding exercises obsolete. I still have not found a good replacement.
  - deleted 1y ago
    [deleted]
  - Gigachad 1y ago
    In office whiteboard interview
  - Jeremy1026 1y ago
    If a person can use AI to complete an interview task, why don't you think they'd be able to use it to complete a work task? I think this thinking is flawed. 15 years ago the argument was, "they'll just use StackOverflow during their take home test." But then once we all got the job we'd check StackOverflow to help solve problems we come across. I don't think AI should be treated any differently, it's a tool to complete the job.
  - corytheboyd 1y ago
    > […] and then having a conversation about what is done, reasoning, etc. Isn’t this where it would likely unravel? The interviewer will know what the interesting parts of the exercise are, and ask the deep questions about them. Observe some more: do they know how to use an IDE, run their own program, cut through code to the parts that matter. Basically, can they do the things someone who wrote the code should trivially be able to do? Since it was mentioned in a sibling comment: Even if the candidate used an LLM to write the code at home, I don’t care, so long as they ace the explanation part of the interview.
    - mooreds 1y ago
      Agreed. It's one thing to ask the AI to solve the problem; it's another thing to be able to explain the way the problem was solved in real-time. (Though you have to watch out for folks that are using the AI to answer your questions.) In fact, I'm okay with people using AI to solve coding problems, as long as that is acceptable behavior at work as well. That should all be spelled out in the interview expectations.
      - corytheboyd 1y ago
        > Though you have to watch out for folks that are using the AI to answer your questions. Heh I do think that happened once (that I was aware of), but it was on a topic I knew a lot about, and it fell apart after layer one. Still, pretty lame, I’d much prefer a “I don’t know,” which I usually say if they start guessing.
  - palebluedot 1y ago
    We were worried about that as well. But we have found that most people are not doing well on our take home. If we get to the point that most people are crushing it, then we may need to think more about AI and take homes (maybe tweak the it with the explicit expectation that they may use AI, etc.) They also need to be able to reason well about why they made the choices they did. Something useful when talking to them can be asking questions like "If X changed, how would that impact your design?". If they were reliant on AI for vibing (rather than just using it as a tool), then those can be more difficult questions to answer well.
  - skeeter2020 1y ago
    the point of the take-home is not to assess the answer, but use it as an anchor to discuss the how / why / what else? type questions. If someone used AI for just the results this is obvious. If they used AI to get the answers AND learn / understand what wa produced, that's probably the new reality
Simulacra 1y ago
I think there's also another element that the author is missing and that is likability. I've never experienced this, but I've heard about people being put through grueling interview processes merely to weed them out because they were deemed "not a good culture fit". I think at least half of all interviews are a popularity contest, and the other half are your qualifications.
- skeeter2020 1y ago
  I don't think "popularity contest" is accurate, but cultural fit is both a real - and important - component AND a risky opportunity to apply massive personal bias. Lots of variability here; I have a coworker who asked the exact questions to all candidates to avoid this, while I can't stop going off script. I know my risk is higher but I also find better candidates (while very likely passing on very good candidates for the wrong reasons).
- ryandrake 1y ago
  Not a fan on hiring based on popularity, but I do wish companies spent more time on actual culture fit and whether or not the person they are hiring is a raging asshole. I've worked with my share of "brilliant jerks" and they're just awful to be around. Even if they are John Carmack level of coding, it's not worth it--they are a net negative because of things like poor attitude, lack of communication skill, arrogance, my-way-or-the-highway stance on everything... I wish that filtering these people out was 50% of my company's interviews!
  - wiether 1y ago
    And I can tell from experience that it's quite easy to get rid of "brilliant jerks" during the recruitement process. The key is to make them stop the process, not you; so you won't have to deal with HR or management telling you that they are brilliant and you should not reject them just because "you" don't like their behavior. Scaring jerks out is quite easy: just be excessively nice during the interview. Tell them how much time and energy everybody take caring for each other's well-being and feelings. Tell them that in the team you have lunch together every single day. And you do stuff together after works. And during the weekends. They'll just run. Sure, you're lying because you're working in a regular team where people are simply respectful professionnals, but they don't need to know that. You'll do everybody's a favour. The jerk included.
epolanski 1y ago
Jm2c but interviews tell you absolutely none, nothing, about what kind of a professional the candidate is. I have no clue whether he'll care and help or pretend to work and drag everybody else down. There's a huge number of incredibly capable developers who could pass any interview but then spend days playing video games and sabotaging projects and teams. I really don't believe in technical interviews, I'd rather base the relationship on trust, if you tell me you're good/experienced at X I trust you to be. If it was bs you'll be shown the door with ease. Instead many companies make it insanely hard to get you hired, but also incredibly hard to cut you out even if you're impact is a very net negative.
- qsort 1y ago
  This works very well for contractors, less so for full-time employees. You can't just fire somebody on a whim, at least not for free, not even in the US, let alone in most of Europe. To be clear, I'm not saying worker protections are bad, just that if firing is much more expensive than hiring, you can't really afford to hire any warm body that walks through the door. These days everyone and their mother is in CS, there are many more talented people than ever, but also more duds than ever.
  - skeeter2020 1y ago
    IME: the only valuable signal comes from direct personal referals. If you have someone who you think is good, and they recommend someone who they say is good from a previous engagement, odds are it will work out. There's a transitive, holistic measure in play; they're not going to destroy their reputation by recommending a weak player or even a strong jerk. The problem here is scale (you quickly milk direct networks dry) but nothing else seems to work well.
  - surgical_fire 1y ago
    > firing is much more expensive than hiring It's the reason why most jobs, even in Europe, have a probation period. During this oeriod firing is inexpensive. In my current employment the probation period was 6 months. If in 6 months you still can't figure out if a hire was good or not, interviewing won't save you.
TrackerFF 1y ago
Cargo culting, and the fear of bad hires. Everyone wants "the best engineers". Probably 1% of tech firms work with deep tech, where boundaries are pushed, and every percentage of improvement or degradation can result in non-trivial gains or losses. The rest can do just fine with normal engineers, and do interviews like the rest of the professional world.
em500 1y ago
Valid points, but I think the most obvious reason is that (at least the recognizable) employers are swamped with applications that all look great on paper, which likely got much worse with the rise of good LLMs. Compared with similar high paying carreers, like medicine (multi-year residency) or high finance (solving math and probability problems on the spot), the hiring process for software engineers isn't especially gruesome.
- deleted 1y ago
  [deleted]
- silvestrov 1y ago
  "multi-year residency" in medicine is part of the education. We don't have that for software development. I'd say that the problem is that the diplomas (degree certificate, ...) are useless when hiring software developers. A doctor who has a diploma is much more likely to be a useful/good doctor than a person with a "computer science" diploma will be a good developer.
  - zdragnar 1y ago
    Back when I started, I think maybe half of the people I worked with had a relevant degree at most. Even that might be an overestimate. Back then, CS degree programs didn't even teach higher level languages than C/++ as they were thought to change too quickly. Whatever you learned during your four years wouldn't apply after you graduated. Instead, the programs focused on the low level implementation details with the theory that was where the engineering and science were. Now, the same school I went to has courses and tracks for web technologies, so who knows.
  - gwbas1c 1y ago
    > A doctor who has a diploma is much more likely to be a useful/good doctor than a person with a "computer science" diploma will be a good developer. A few things to consider: What we practice is "software engineering." Computer science is closely related to software engineering; but not the same thing. (It's like the difference between a degree in physics vs mechanical engineering.) Doctors still have to be board certified, which requires self-study of topics that aren't taught in class. Some people do get their medical diploma and fail their board certifications, in which case they can't practice.
alephnerd 1y ago
A major issue the author missed is it is much harder to fire a non-performer in the tech industry today. It takes 2 quarters (ie. 6 months) to go from recognizing a problem employee to firing said employee. This makes the risk of hiring the wrong candidate significant as a hiring manager, because a bad hire reflects badly on you and eats up your budget thus preventing a backfill. On top of that, firing individual employees can lead to litigation risk (even if frivolous), thus requiring hiring managers to go through the extreme song and dance of the PIP process and documented reprimands in order to provide counsel if litigated.
- baka367 1y ago
  Yet I have met very few truly bad engineers in my life. Most of the "bad" ones were not bad in skills, but a bad match due to their willingness to die on one hill or another and complete refusal to work with others. Yet, most of the interviews put way too little focus on the soft skills and way too much focus on the hard skills.
  - skeeter2020 1y ago
    hard skills are difficult enough to try and assess; soft skills even harder. Most try with behavioural questions which have very little signal IMO. I'm a senior manager / director now so most interviews focus on softer skills. My strategy is to give very concrete examples to the "tell me about a time" style questions. Every other answer is easily forgettable.
- bradlys 1y ago
  In what companies are you speaking of? In Silicon Valley, we fire people very quickly as soon as notice it’s not a correctable problem. Hell, sometimes even when it is correctable.
higeorge13 1y ago
Companies keep ignoring the historical interview point. I have been to a few occasions when companies needed the exact thing i built in the past (e.g. migration to clickhouse), but chose to put me into a random take home related to a different technology (e.g. some bigquery assignment) and eventually reject me. Go off script and ask me details about the project which might solve your hands, why do i need to talk about something else?
- skeeter2020 1y ago
  quick answer: this takes time & energy; the interviewing company is willing to miss you to save this effort. Plus the interviewers are not likely to be good at these tasks, even if they're solid developers; it's a skill they have relatively little practice and training.
  - higeorge13 1y ago
    I understand this for a dry candidate pool. I have been there as a hiring manager, you need some signal with take home or live interviewing. But on the rare opportunity that you find someone who is willing to talk in details about a similar project you want to do, you skip the pipeline and if he indeed built it, you hire him.
MontyCarloHall 1y ago
Why? Because the vast majority of job interviews are with terrible candidates, even if the majority of candidates are excellent. This apparent paradox has a simple explanation: excellent candidates selectively apply to a few companies and get interviews/offers at almost all of them. On the other hand, terrible candidates are rejected at every step of the hiring process, and have to constantly reenter the interview pool. Suppose 90% of candidates are excellent and 10% are terrible. If the excellent 90% only need to interview at one company, whereas the bad 10% need to interview at 20 companies, then only 0.9/(0.1*20+0.9)=31% of interviews will be with qualified candidates. To retierate: almost 70% of interviews will be with terrible candidates, even though 90% of people applying for jobs are excellent. Because the cost of a bad hire is so consequential, the interview process is not designed to efficiently handle a minority of qualified candidates, but rather efficiently weed out a majority of horrible candidates. It is therefore a terrible process for the people actually qualified to pass it.
- ranger207 1y ago
  Nowadays it's more like the excellent 90% need to interview at 20 companies and the terrible 10% need to interview at yet more
- jurebb 1y ago
  best explanation i’ve read so far!
  - the_af 1y ago
    If I remember correctly, there's an ancient article by Joel Spolsky about this. That great candidates are not out there doing blind interviews, only those of us who are average are interviewing and going through hoops. Engineers below average are almost constantly interviewing.
    - MontyCarloHall 1y ago
      Yup, back in 2005 [0]! It's far from a new idea. [0] https://www.joelonsoftware.com/2005/01/ https://www.joelonsoftware.com/2005/01/
StopDisinfo910 1y ago
In my experience, technical interviews are not really useful past the very basic "Can this candidate actually write a conditional and has the slightest clue about programming?". Ability to solve hard leetcode-like problems under time pressure in a stressful environment doesn't meaningfully translate to "will be a great contributor to the team work on the kind of problem we have". Our best hires are nearly always coming from the network of a team member or people we contracted with and decided to hire full time. Most of my time in interview nowadays is spent understanding what the candidate has done before, explaining to them what we do and asking open questions to see how they would approach our issues and how they link them to their experience. If it seems to fit, we hire. My country standard contract offers a fairly long probation period for new hire and we don't hesite about parting with people when it's not working after a quarter. We are very explicit about this policy.
- mooreds 1y ago
  > Our best hires are nearly always coming from the network of a team member or people we contracted with and decided to hire full time. I love this approach. However, at least in the USA, there are substantial costs to candidates for this approach: - no health care while contracting (unless the candidate pays for it) - being a contractor is a different level of risk than moving from FTE to FTE - if the employer decides it isn't a fit, candidate has to find another contractor Do you have any great candidates who approach you and then, finding out your approach, pass?
  - StopDisinfo910 1y ago
    > Do you have any great candidates who approach you and then, finding out your approach, pass? We never generalised the contractors to employee thing. It's just that we hire contractors fairly often to fill in temporary needs and we generally extend offer for full time employment to the ones we would like to keep with us when we can. They generally say no because they like being contractors. For candidate applying, I have yet to see one explicitely refuse an offer because they know we don't always keep going after the probation period. Then again, the standard probation period for engineers in my country is 3 months which can be renewed once so they would get the same offer anywhere. But I have never seen someone being surprised when we parted after the 3 months. If someone told me they were during our offboarding meeting, I would take it as us having done something seriously wrong during our onboarding. To be honest, we don't have that many hirings which end up not working and of them I can only remember one being due to an actual lack of skill from someone who clearly padded their resume yet managed to go through the interview. Some people needed a bit more time than others to reach the level of delivery we expect but that's ok. We don't always need rock stars, just professional who can deliver consistently and are open to learning new things. When we need specialists and don't have someone sufficently skilled internally, we contract. Working together with a specialist tends to raise the level of the team as a whole. Most what I consider our true hiring failures have come from a mismatch between what the person expected and what the job actually is. That's why I now take time to ensure the person I'm interviewing actually has a good idea of what we are doing.
jerf 1y ago
The alternative I prefer isn't any of the given ones, but a scaled interview. I start with something that you can solve with a Python one-liner, if you know what you're doing, but if you solve that instantly I've got a series of questions that will scale up until we're eventually doing the task using untrusted user input, outputting into a security-sensitive environment, under heavy load that we want to be cheap, in a highly-available environment, etc. etc. I also tell the applicant up front that this is the plan, because if you don't do that, it feels like you're jerking them around by constantly changing the requirements. I've also got a number of directions it can go depending on the applicant, and depending on what they tell me about their past experience. The other nice thing about this is your interviewee generally doesn't leave feeling like a failure; it's not like I have three questions and you can get them all wrong. Unfortunately there are some people who end up spinning on the very easiest question for the entire session, and, uh, well... I can only do so much, if you really don't know anything about programming at all. This is at least the exception, though. I have not had to do an interview in the age of practical AI yet, though. In person I don't think I'd have to change much, I've always interviewed with a policy of "I'm not worried about whether the string split command takes its parameters in this or that order, I just want to know you know it exists" and I can basically serve as an AI in the same way I was already being the API reference. Remotely, I'm not sure what I'd do yet.
- skeeter2020 1y ago
  I do like your approach more than many alternatives but there are still lots of challenges. Two big issues: 1. this is not typically how even a confident, prepared, solid developer approaches coding problems, so they can be poorly prepared for this style of interview. The result is you don't get an accurate representation of the individual. 2. it takes an awful lot of the interviewer's time to prepare, even if you can scale to multiple candidates with much of the same work. You need a very strong technical resource to deliver this style of interview, and they could probably be doing higher value work. I've hired a lot of the past ~5 years and my take away: it sucks for everyone - on both sides - except the early HR-led functions, and they don't add any value or much help. They may actually make most of the challenges much worse.
  - 1y ago
michaelteter 1y ago
Interviews allow just a little time for strangers to attempt to sync up and get a sense of each other. Actual jobs offer much more time for people to learn to sync and communicate. Also, on both sides of the interview table, people have varying strengths when it comes to short and long term communication skills. Plenty of interviewers are not good at interviewing, just as plenty of candidates are not good at their side of the process. In short, interviews are a very poor approach to choosing who belongs long term. Ultimately, regardless of the interview process used, every job is ultimately a long term “try and see” interview. You only know how someone fits by trying them for a while.
deleted 1y ago
[deleted]
jgeada 1y ago
I know it is hearsay, but it seems many times interviews try to get free work from candidates. Here, have a look at this real problem, see if you can solve it in whatever the interview time frame is. When there are too many candidates and not so many jobs, and absolutely no regulation, employers are free to exploit applicants without consequences. Employers will do this even when there is no actual job being offered, so they can use the availability of applicants to apply downward pressure on the salaries of the workers they already have.
- skeeter2020 1y ago
  I have never seen this from either side. Think about it from the hiring side: how good of a solution could be produced by someone with zero context and experience in the business, domain and internal environment? This would be like less efficient "organic" vibe coding.
- sigwinch 1y ago
  We should establish a convention that # noqa cannot be removed from the top of my code sample.
rvz 1y ago
This is going to be the new reality of an AGI future.
bluGill 1y ago
Interviews have been extensively studied by phd, figuring out what works and what doesn't. I have yet to see anyone write a blog or comment that shows even an awareness that this research exists, much less what it in it. (I know this research exists, but I'll admit to not knowing how to find it. My company insists the interview process they make me follow is based on it)
agentultra 1y ago
Invert a binary tree, find the optimal path in an n-dimensional space of real numbers given by a function, design a horizontally scalable build system, get grilled by at least two people on your leadership skills... then, if you're extremely lucky, get hired to configure the button to be blue and have round corners. And get laid off in 9 months. (Not saying this happened to me but it's a common story I've heard in the last few years) I agree with the author that it is hard to assess someone's skill if you have a list of 100 people to interview and you know nothing about them. The bigger the stack of applications the easier it is to treat them like data and not people. I know plenty of software developers who write, maintain, or contribute to OSS libraries; they write blog posts, give talks at conferences/meetups, and make videos in their spare time (or sometimes as part of their work). I've rarely walked into an interview where the hiring manager or the technical interviewer hadn't just read my name off my resume as I joined the meeting. Get to know the candidate's work before assuming they know nothing! Maybe people involved in the hiring process should be given more time to properly research a candidate. Relying on ATS' and putting the burden on applicant's to do the work of proving themselves is causing a lot of folks to burnout just trying to get a job.
ChrisMarshallNY 1y ago
Good post. Thanks for that. I tended to do the "Historical Interview" one. In my case, it worked well. I think take-home tests are probably damn near worthless, these days. People will just feed the assignment into an LLM, and return the results. I am not a fan of LeetCode, because I don't want to waste time, studying stuff that won't have any relevance to what needs to get done. I like to keep my dance card full, and wasting precious time to make someone else happy, isn't my idea of a good time. That said, I understand why they are there. I just feel that they aren't really something that I'd use to judge senior-level talent; which was what I used to hire.
anself 1y ago
The 100m / 10k analogy is fantastic, but the reality is even worse. Let’s say you expect a candidate to stay for average five years. That’s about 1200 working days. If you get to interview them for 1 day, then that day is 8.3m to the 10k. You get to watch them run for 8.3m and decide how well they can run the 10k.
- agentultra 1y ago
  You're not even watching them for the skills required on the job either. Most jobs aren't even about running at all. Basic, boring, line-of-business web application shops are asking candidates to implement Boyer-Moore and k-d trees. When the applicant is going to be hired to mostly update JSX templates and figure out why the customer is frustrated. It's like every business needs Olympic athletes when they're running a hot-dog stand. It's one thing if your business is writing a RTOS for a custom platform you develop. Asking a candidate to implement a slab allocator might be a fine exercise. It fits the work being done. Businesses need to get off their high horse and look in the mirror. Most of them are doing mediocre things that need average, good-natured people who are willing to work.
gwbas1c 1y ago
> Asking a candidate to solve some problem which lets a candidate work in a slightly less high stakes environment. but requires candidates to do extra work, taking longer than just the interview. I refuse a lot of these. I've put a lot of time into many of them, just to be ghosted. Typically, when the take-home comes with a laundry list of frameworks and 3rd party libraries, I walk away. Coming up to speed with all of them is too time consuming for the probability of getting ghosted. It takes a lot of work on the interviewers side to come up with a good take-home. It requires discipline to reject the temptation to just throw in a laundry list of requirements.
singularity2001 1y ago
Is there any other industry with such a disconnect between education and job where the candidates need to do another exam when they try to join the job?
hunkmuller0 1y ago
[dead]

56 points by mooreds 1y ago

billy99k 1y ago

I prefer take home, and then having a conversation about what is done, reasoning, etc.

galdor 1y ago
It feels like the age of AI has made take home coding exercises obsolete. I still have not found a good replacement.
- deleted 1y ago
  [deleted]
- Gigachad 1y ago
  In office whiteboard interview
- Jeremy1026 1y ago
  If a person can use AI to complete an interview task, why don't you think they'd be able to use it to complete a work task? I think this thinking is flawed. 15 years ago the argument was, "they'll just use StackOverflow during their take home test." But then once we all got the job we'd check StackOverflow to help solve problems we come across. I don't think AI should be treated any differently, it's a tool to complete the job.
- corytheboyd 1y ago
  > […] and then having a conversation about what is done, reasoning, etc. Isn’t this where it would likely unravel? The interviewer will know what the interesting parts of the exercise are, and ask the deep questions about them. Observe some more: do they know how to use an IDE, run their own program, cut through code to the parts that matter. Basically, can they do the things someone who wrote the code should trivially be able to do? Since it was mentioned in a sibling comment: Even if the candidate used an LLM to write the code at home, I don’t care, so long as they ace the explanation part of the interview.
  - mooreds 1y ago
    Agreed. It's one thing to ask the AI to solve the problem; it's another thing to be able to explain the way the problem was solved in real-time. (Though you have to watch out for folks that are using the AI to answer your questions.) In fact, I'm okay with people using AI to solve coding problems, as long as that is acceptable behavior at work as well. That should all be spelled out in the interview expectations.
    - corytheboyd 1y ago
      > Though you have to watch out for folks that are using the AI to answer your questions. Heh I do think that happened once (that I was aware of), but it was on a topic I knew a lot about, and it fell apart after layer one. Still, pretty lame, I’d much prefer a “I don’t know,” which I usually say if they start guessing.
- palebluedot 1y ago
  We were worried about that as well. But we have found that most people are not doing well on our take home. If we get to the point that most people are crushing it, then we may need to think more about AI and take homes (maybe tweak the it with the explicit expectation that they may use AI, etc.) They also need to be able to reason well about why they made the choices they did. Something useful when talking to them can be asking questions like "If X changed, how would that impact your design?". If they were reliant on AI for vibing (rather than just using it as a tool), then those can be more difficult questions to answer well.
- skeeter2020 1y ago
  the point of the take-home is not to assess the answer, but use it as an anchor to discuss the how / why / what else? type questions. If someone used AI for just the results this is obvious. If they used AI to get the answers AND learn / understand what wa produced, that's probably the new reality

Simulacra 1y ago

I think there's also another element that the author is missing and that is likability. I've never experienced this, but I've heard about people being put through grueling interview processes merely to weed them out because they were deemed "not a good culture fit". I think at least half of all interviews are a popularity contest, and the other half are your qualifications.

skeeter2020 1y ago
I don't think "popularity contest" is accurate, but cultural fit is both a real - and important - component AND a risky opportunity to apply massive personal bias. Lots of variability here; I have a coworker who asked the exact questions to all candidates to avoid this, while I can't stop going off script. I know my risk is higher but I also find better candidates (while very likely passing on very good candidates for the wrong reasons).
ryandrake 1y ago
Not a fan on hiring based on popularity, but I do wish companies spent more time on actual culture fit and whether or not the person they are hiring is a raging asshole. I've worked with my share of "brilliant jerks" and they're just awful to be around. Even if they are John Carmack level of coding, it's not worth it--they are a net negative because of things like poor attitude, lack of communication skill, arrogance, my-way-or-the-highway stance on everything... I wish that filtering these people out was 50% of my company's interviews!
- wiether 1y ago
  And I can tell from experience that it's quite easy to get rid of "brilliant jerks" during the recruitement process. The key is to make them stop the process, not you; so you won't have to deal with HR or management telling you that they are brilliant and you should not reject them just because "you" don't like their behavior. Scaring jerks out is quite easy: just be excessively nice during the interview. Tell them how much time and energy everybody take caring for each other's well-being and feelings. Tell them that in the team you have lunch together every single day. And you do stuff together after works. And during the weekends. They'll just run. Sure, you're lying because you're working in a regular team where people are simply respectful professionnals, but they don't need to know that. You'll do everybody's a favour. The jerk included.

epolanski 1y ago

Jm2c but interviews tell you absolutely none, nothing, about what kind of a professional the candidate is. I have no clue whether he'll care and help or pretend to work and drag everybody else down. There's a huge number of incredibly capable developers who could pass any interview but then spend days playing video games and sabotaging projects and teams. I really don't believe in technical interviews, I'd rather base the relationship on trust, if you tell me you're good/experienced at X I trust you to be. If it was bs you'll be shown the door with ease. Instead many companies make it insanely hard to get you hired, but also incredibly hard to cut you out even if you're impact is a very net negative.

qsort 1y ago
This works very well for contractors, less so for full-time employees. You can't just fire somebody on a whim, at least not for free, not even in the US, let alone in most of Europe. To be clear, I'm not saying worker protections are bad, just that if firing is much more expensive than hiring, you can't really afford to hire any warm body that walks through the door. These days everyone and their mother is in CS, there are many more talented people than ever, but also more duds than ever.
- skeeter2020 1y ago
  IME: the only valuable signal comes from direct personal referals. If you have someone who you think is good, and they recommend someone who they say is good from a previous engagement, odds are it will work out. There's a transitive, holistic measure in play; they're not going to destroy their reputation by recommending a weak player or even a strong jerk. The problem here is scale (you quickly milk direct networks dry) but nothing else seems to work well.
- surgical_fire 1y ago
  > firing is much more expensive than hiring It's the reason why most jobs, even in Europe, have a probation period. During this oeriod firing is inexpensive. In my current employment the probation period was 6 months. If in 6 months you still can't figure out if a hire was good or not, interviewing won't save you.

TrackerFF 1y ago

Cargo culting, and the fear of bad hires. Everyone wants "the best engineers". Probably 1% of tech firms work with deep tech, where boundaries are pushed, and every percentage of improvement or degradation can result in non-trivial gains or losses. The rest can do just fine with normal engineers, and do interviews like the rest of the professional world.

em500 1y ago

Valid points, but I think the most obvious reason is that (at least the recognizable) employers are swamped with applications that all look great on paper, which likely got much worse with the rise of good LLMs. Compared with similar high paying carreers, like medicine (multi-year residency) or high finance (solving math and probability problems on the spot), the hiring process for software engineers isn't especially gruesome.

deleted 1y ago
[deleted]
silvestrov 1y ago
"multi-year residency" in medicine is part of the education. We don't have that for software development. I'd say that the problem is that the diplomas (degree certificate, ...) are useless when hiring software developers. A doctor who has a diploma is much more likely to be a useful/good doctor than a person with a "computer science" diploma will be a good developer.
- zdragnar 1y ago
  Back when I started, I think maybe half of the people I worked with had a relevant degree at most. Even that might be an overestimate. Back then, CS degree programs didn't even teach higher level languages than C/++ as they were thought to change too quickly. Whatever you learned during your four years wouldn't apply after you graduated. Instead, the programs focused on the low level implementation details with the theory that was where the engineering and science were. Now, the same school I went to has courses and tracks for web technologies, so who knows.
- gwbas1c 1y ago
  > A doctor who has a diploma is much more likely to be a useful/good doctor than a person with a "computer science" diploma will be a good developer. A few things to consider: What we practice is "software engineering." Computer science is closely related to software engineering; but not the same thing. (It's like the difference between a degree in physics vs mechanical engineering.) Doctors still have to be board certified, which requires self-study of topics that aren't taught in class. Some people do get their medical diploma and fail their board certifications, in which case they can't practice.

alephnerd 1y ago

A major issue the author missed is it is much harder to fire a non-performer in the tech industry today. It takes 2 quarters (ie. 6 months) to go from recognizing a problem employee to firing said employee. This makes the risk of hiring the wrong candidate significant as a hiring manager, because a bad hire reflects badly on you and eats up your budget thus preventing a backfill. On top of that, firing individual employees can lead to litigation risk (even if frivolous), thus requiring hiring managers to go through the extreme song and dance of the PIP process and documented reprimands in order to provide counsel if litigated.

baka367 1y ago
Yet I have met very few truly bad engineers in my life. Most of the "bad" ones were not bad in skills, but a bad match due to their willingness to die on one hill or another and complete refusal to work with others. Yet, most of the interviews put way too little focus on the soft skills and way too much focus on the hard skills.
- skeeter2020 1y ago
  hard skills are difficult enough to try and assess; soft skills even harder. Most try with behavioural questions which have very little signal IMO. I'm a senior manager / director now so most interviews focus on softer skills. My strategy is to give very concrete examples to the "tell me about a time" style questions. Every other answer is easily forgettable.
bradlys 1y ago
In what companies are you speaking of? In Silicon Valley, we fire people very quickly as soon as notice it’s not a correctable problem. Hell, sometimes even when it is correctable.

higeorge13 1y ago

Companies keep ignoring the historical interview point. I have been to a few occasions when companies needed the exact thing i built in the past (e.g. migration to clickhouse), but chose to put me into a random take home related to a different technology (e.g. some bigquery assignment) and eventually reject me. Go off script and ask me details about the project which might solve your hands, why do i need to talk about something else?

skeeter2020 1y ago
quick answer: this takes time & energy; the interviewing company is willing to miss you to save this effort. Plus the interviewers are not likely to be good at these tasks, even if they're solid developers; it's a skill they have relatively little practice and training.
- higeorge13 1y ago
  I understand this for a dry candidate pool. I have been there as a hiring manager, you need some signal with take home or live interviewing. But on the rare opportunity that you find someone who is willing to talk in details about a similar project you want to do, you skip the pipeline and if he indeed built it, you hire him.

MontyCarloHall 1y ago

Why? Because the vast majority of job interviews are with terrible candidates, even if the majority of candidates are excellent. This apparent paradox has a simple explanation: excellent candidates selectively apply to a few companies and get interviews/offers at almost all of them. On the other hand, terrible candidates are rejected at every step of the hiring process, and have to constantly reenter the interview pool. Suppose 90% of candidates are excellent and 10% are terrible. If the excellent 90% only need to interview at one company, whereas the bad 10% need to interview at 20 companies, then only 0.9/(0.1*20+0.9)=31% of interviews will be with qualified candidates. To retierate: almost 70% of interviews will be with terrible candidates, even though 90% of people applying for jobs are excellent. Because the cost of a bad hire is so consequential, the interview process is not designed to efficiently handle a minority of qualified candidates, but rather efficiently weed out a majority of horrible candidates. It is therefore a terrible process for the people actually qualified to pass it.

ranger207 1y ago
Nowadays it's more like the excellent 90% need to interview at 20 companies and the terrible 10% need to interview at yet more
jurebb 1y ago
best explanation i’ve read so far!
- the_af 1y ago
  If I remember correctly, there's an ancient article by Joel Spolsky about this. That great candidates are not out there doing blind interviews, only those of us who are average are interviewing and going through hoops. Engineers below average are almost constantly interviewing.
  - MontyCarloHall 1y ago
    Yup, back in 2005 [0]! It's far from a new idea. [0] https://www.joelonsoftware.com/2005/01/ https://www.joelonsoftware.com/2005/01/

StopDisinfo910 1y ago

In my experience, technical interviews are not really useful past the very basic "Can this candidate actually write a conditional and has the slightest clue about programming?". Ability to solve hard leetcode-like problems under time pressure in a stressful environment doesn't meaningfully translate to "will be a great contributor to the team work on the kind of problem we have". Our best hires are nearly always coming from the network of a team member or people we contracted with and decided to hire full time. Most of my time in interview nowadays is spent understanding what the candidate has done before, explaining to them what we do and asking open questions to see how they would approach our issues and how they link them to their experience. If it seems to fit, we hire. My country standard contract offers a fairly long probation period for new hire and we don't hesite about parting with people when it's not working after a quarter. We are very explicit about this policy.

mooreds 1y ago
> Our best hires are nearly always coming from the network of a team member or people we contracted with and decided to hire full time. I love this approach. However, at least in the USA, there are substantial costs to candidates for this approach: - no health care while contracting (unless the candidate pays for it) - being a contractor is a different level of risk than moving from FTE to FTE - if the employer decides it isn't a fit, candidate has to find another contractor Do you have any great candidates who approach you and then, finding out your approach, pass?
- StopDisinfo910 1y ago
  > Do you have any great candidates who approach you and then, finding out your approach, pass? We never generalised the contractors to employee thing. It's just that we hire contractors fairly often to fill in temporary needs and we generally extend offer for full time employment to the ones we would like to keep with us when we can. They generally say no because they like being contractors. For candidate applying, I have yet to see one explicitely refuse an offer because they know we don't always keep going after the probation period. Then again, the standard probation period for engineers in my country is 3 months which can be renewed once so they would get the same offer anywhere. But I have never seen someone being surprised when we parted after the 3 months. If someone told me they were during our offboarding meeting, I would take it as us having done something seriously wrong during our onboarding. To be honest, we don't have that many hirings which end up not working and of them I can only remember one being due to an actual lack of skill from someone who clearly padded their resume yet managed to go through the interview. Some people needed a bit more time than others to reach the level of delivery we expect but that's ok. We don't always need rock stars, just professional who can deliver consistently and are open to learning new things. When we need specialists and don't have someone sufficently skilled internally, we contract. Working together with a specialist tends to raise the level of the team as a whole. Most what I consider our true hiring failures have come from a mismatch between what the person expected and what the job actually is. That's why I now take time to ensure the person I'm interviewing actually has a good idea of what we are doing.

jerf 1y ago

The alternative I prefer isn't any of the given ones, but a scaled interview. I start with something that you can solve with a Python one-liner, if you know what you're doing, but if you solve that instantly I've got a series of questions that will scale up until we're eventually doing the task using untrusted user input, outputting into a security-sensitive environment, under heavy load that we want to be cheap, in a highly-available environment, etc. etc. I also tell the applicant up front that this is the plan, because if you don't do that, it feels like you're jerking them around by constantly changing the requirements. I've also got a number of directions it can go depending on the applicant, and depending on what they tell me about their past experience. The other nice thing about this is your interviewee generally doesn't leave feeling like a failure; it's not like I have three questions and you can get them all wrong. Unfortunately there are some people who end up spinning on the very easiest question for the entire session, and, uh, well... I can only do so much, if you really don't know anything about programming at all. This is at least the exception, though. I have not had to do an interview in the age of practical AI yet, though. In person I don't think I'd have to change much, I've always interviewed with a policy of "I'm not worried about whether the string split command takes its parameters in this or that order, I just want to know you know it exists" and I can basically serve as an AI in the same way I was already being the API reference. Remotely, I'm not sure what I'd do yet.

skeeter2020 1y ago
I do like your approach more than many alternatives but there are still lots of challenges. Two big issues: 1. this is not typically how even a confident, prepared, solid developer approaches coding problems, so they can be poorly prepared for this style of interview. The result is you don't get an accurate representation of the individual. 2. it takes an awful lot of the interviewer's time to prepare, even if you can scale to multiple candidates with much of the same work. You need a very strong technical resource to deliver this style of interview, and they could probably be doing higher value work. I've hired a lot of the past ~5 years and my take away: it sucks for everyone - on both sides - except the early HR-led functions, and they don't add any value or much help. They may actually make most of the challenges much worse.
- 1y ago

michaelteter 1y ago

Interviews allow just a little time for strangers to attempt to sync up and get a sense of each other. Actual jobs offer much more time for people to learn to sync and communicate. Also, on both sides of the interview table, people have varying strengths when it comes to short and long term communication skills. Plenty of interviewers are not good at interviewing, just as plenty of candidates are not good at their side of the process. In short, interviews are a very poor approach to choosing who belongs long term. Ultimately, regardless of the interview process used, every job is ultimately a long term “try and see” interview. You only know how someone fits by trying them for a while.

deleted 1y ago

[deleted]

jgeada 1y ago

I know it is hearsay, but it seems many times interviews try to get free work from candidates. Here, have a look at this real problem, see if you can solve it in whatever the interview time frame is. When there are too many candidates and not so many jobs, and absolutely no regulation, employers are free to exploit applicants without consequences. Employers will do this even when there is no actual job being offered, so they can use the availability of applicants to apply downward pressure on the salaries of the workers they already have.

skeeter2020 1y ago
I have never seen this from either side. Think about it from the hiring side: how good of a solution could be produced by someone with zero context and experience in the business, domain and internal environment? This would be like less efficient "organic" vibe coding.
sigwinch 1y ago
We should establish a convention that # noqa cannot be removed from the top of my code sample.

rvz 1y ago

This is going to be the new reality of an AGI future.

bluGill 1y ago

Interviews have been extensively studied by phd, figuring out what works and what doesn't. I have yet to see anyone write a blog or comment that shows even an awareness that this research exists, much less what it in it. (I know this research exists, but I'll admit to not knowing how to find it. My company insists the interview process they make me follow is based on it)

agentultra 1y ago

Invert a binary tree, find the optimal path in an n-dimensional space of real numbers given by a function, design a horizontally scalable build system, get grilled by at least two people on your leadership skills... then, if you're extremely lucky, get hired to configure the button to be blue and have round corners. And get laid off in 9 months. (Not saying this happened to me but it's a common story I've heard in the last few years) I agree with the author that it is hard to assess someone's skill if you have a list of 100 people to interview and you know nothing about them. The bigger the stack of applications the easier it is to treat them like data and not people. I know plenty of software developers who write, maintain, or contribute to OSS libraries; they write blog posts, give talks at conferences/meetups, and make videos in their spare time (or sometimes as part of their work). I've rarely walked into an interview where the hiring manager or the technical interviewer hadn't just read my name off my resume as I joined the meeting. Get to know the candidate's work before assuming they know nothing! Maybe people involved in the hiring process should be given more time to properly research a candidate. Relying on ATS' and putting the burden on applicant's to do the work of proving themselves is causing a lot of folks to burnout just trying to get a job.

ChrisMarshallNY 1y ago

Good post. Thanks for that. I tended to do the "Historical Interview" one. In my case, it worked well. I think take-home tests are probably damn near worthless, these days. People will just feed the assignment into an LLM, and return the results. I am not a fan of LeetCode, because I don't want to waste time, studying stuff that won't have any relevance to what needs to get done. I like to keep my dance card full, and wasting precious time to make someone else happy, isn't my idea of a good time. That said, I understand why they are there. I just feel that they aren't really something that I'd use to judge senior-level talent; which was what I used to hire.

anself 1y ago

The 100m / 10k analogy is fantastic, but the reality is even worse. Let’s say you expect a candidate to stay for average five years. That’s about 1200 working days. If you get to interview them for 1 day, then that day is 8.3m to the 10k. You get to watch them run for 8.3m and decide how well they can run the 10k.

agentultra 1y ago
You're not even watching them for the skills required on the job either. Most jobs aren't even about running at all. Basic, boring, line-of-business web application shops are asking candidates to implement Boyer-Moore and k-d trees. When the applicant is going to be hired to mostly update JSX templates and figure out why the customer is frustrated. It's like every business needs Olympic athletes when they're running a hot-dog stand. It's one thing if your business is writing a RTOS for a custom platform you develop. Asking a candidate to implement a slab allocator might be a fine exercise. It fits the work being done. Businesses need to get off their high horse and look in the mirror. Most of them are doing mediocre things that need average, good-natured people who are willing to work.

gwbas1c 1y ago

> Asking a candidate to solve some problem which lets a candidate work in a slightly less high stakes environment. but requires candidates to do extra work, taking longer than just the interview. I refuse a lot of these. I've put a lot of time into many of them, just to be ghosted. Typically, when the take-home comes with a laundry list of frameworks and 3rd party libraries, I walk away. Coming up to speed with all of them is too time consuming for the probability of getting ghosted. It takes a lot of work on the interviewers side to come up with a good take-home. It requires discipline to reject the temptation to just throw in a laundry list of requirements.

singularity2001 1y ago

Is there any other industry with such a disconnect between education and job where the candidates need to do another exam when they try to join the job?

hunkmuller0 1y ago

[dead]