26 ms·

Having been an interviewer at a FAANG for many years, I can explain some of the logic behind it. I'm not saying this logic is valid, but it's how we got here, i

by mabbo 5y ago

Having been an interviewer at a FAANG for many years, I can explain some of the logic behind it. I'm not saying this logic is valid, but it's how we got here, imho. First: we no longer trust the hiring manager alone, because probably they aren't a strong developer. We instead trust strong developers that are well trained at evaluating good devs. At the same time, we don't want to thrust a dev onto a hiring manager, so they also need to interview you too and have a say. Second: Is it really fair to have just one or two developers evaluate you? When I first was an interviewer, I liked everybody! I would have hired them all. So getting multiple data points matters. Best to have at least a couple dev interviews. Then there's the whole problem of needing to evaluate you on multiple dimensions. Can one interview really tell if you're good problem solving, coding, algorithms/data structures, and any specialization the role has? What about the soft skills aspect? We're going to need to have at least 3 or 4 interviews to cover all these aspects. These roles pay a huge sum of money, so there's a lot of worry that someone will be hired who doesn't really meet the bar, you know? But now we have a bigger problem: if we're going to invest 4+ people to spend an hour of time with you each, we'd better have some data points that you're worth that investment. So maybe we need one or two initial interviews ahead of time to weed out any obviously unlikely candidates. After that, it's every other company going "Oh shit, Amazon does 6 interviews? We should do that too!".

mitchdoogle 5y ago
> "Oh shit, Amazon does 6 interviews? We should do that too!" Therein lies the problem. Doing things because someone bigger and better is doing it is not the right decision making process
dleslie 5y ago
What I'm reading here is that even as an experienced interviewer you struggle to make the format effective and equitable. Sounds to me like the format isn't worth keeping; time to hire more, and fire fast?
- lbotos 5y ago
  1. Often, legally, firing people is harder than hiring. 2. Hire more, fire fast sounds good on paper but will destroy morale as the team identity will be constantly influx. 3. Engineers will spend less time onboarding newer engineers to the detriment of everyone because well, they might not just last.
  - dleslie 5y ago
    Basically everywhere in NA has allowances for probationary periods with at-will employment. Moreover, prospective hires could start on contract. Morale is improved when team members feel like they have some control over their membership. It seems like you're assuming that a great many people would be fired; why so? In my experience, having worked for several such companies, firing is rare because even minimal filters are effective. Resume check, reference check, and meet the team. Done.
    - ratww 5y ago
      > Basically everywhere in NA has allowances for probationary periods with at-will employment. Yep. Lots of places with even stricter pro-employee laws, including Europe, also have it.
    - StupidUser123 5y ago
      I’m not saying you’re categorically wrong, but this doesn’t match with my experience, at all. If you’re an HM and this works for you and your company, great, but the HMs responding here don’t agree. The processes you are questioning scale to thousands of people with varying backgrounds. It’s not an accident all of these companies do this. Your process places extreme trust in one group of people. It’s just so risky. Also, resumes are mostly bullshit, IMO. Reference checks are complete bullshit, IMO. I’ve worked with my buddies at startups (who get lofty titles) and just have them give me reference checks. Other references have asked me for call scripts. Also keep in mind that most of these big tech jobs are highly competitive. Your solution eventually requires a coin toss if you have limited spots. There’s only so much data you can collect from the process you propose. The natural thing is to then assess each candidate a bit more until you’re confident you’ve picked the right one. And then you’re here.
- edmundsauto 5y ago
  Hire fast and fire requires even more effort and commitment from a candidate, I’m not sure how that would be an improvement. Firing fast would be like 3 months of interviews! I’d also like to add that having multiple interviewers, if they have diverse backgrounds, is more likely to be equitable than a single interviewer.
  - dleslie 5y ago
    It's not so much effort or commitment if the team is fully remote.
    - StupidUser123 5y ago
      You’re asking someone to quit their job to maybe get this new job. And you won’t know for months so all your other offers are gone. So if it doesn’t work out, you’re unemployed with no prospects. Sounds terrible. I’d take 15 interviews before taking that risk.
      - dleslie 5y ago
        If it takes months to determine that someone is a bad fit for a team then the team has internal issues that need to be resolved. Moreover, if someone has the skills on paper, and the references to support that, then any tech screening is unnecessary. It just tests for ability to pass tech screens.
        kyawzazaw 5y ago
        That's what internships are in a way. Do you think that's a bad idea?
        TheCoelacanth 5y ago
        Internships are probably the best way to hire, but they only work for people at that specific stage of life. University students already have a three-month gap in their schedule where they aren't doing anything, so they are willing to take a temporary role to fill that gap. Experienced people who already have a job aren't going to drop that to take on a temporary role unless they have an unusually high risk-tolerance or unless they are desperate.
- marssaxman 5y ago
  "Hire more, fire fast" does not sound like a company I'd want to work for.
- kjeetgill 5y ago
  I never understand answers of this format: "status quo has flaw of some known, finite impact" time to "completely ridiculously overcompensate, overreact, and make arbitrary move?" I can't even fathom in what ways you think "time to hire more, and fire fast?" is more "effective and equitable"? You just shove even more risk onto individuals who might be getting their footing still. You're just turning a 8-9 hours commitment to an interview into a 6 month commitment. Sure, you get paid but you also have to deal with churn and burn. I mean, did you think stack ranking was a good idea?
  - dleslie 5y ago
    The response is to the needless growth in complexity of the hiring process; advocating for reducing that complexity is appropriate. You're tacitly expressing a concern that a fast hiring commitment would be riskier; but that's not necessarily so. A minimal set of filters are meaningfully effective: resume, references, and meet the team. Anything beyond that doesn't have just diminishing returns, it can negatively impact the company by narrowing the field of viable candidates too far.
    - prepend 5y ago
      > expressing a concern that a fast hiring commitment would be riskier; but that's not necessarily so. I think it is necessarily so. Hiring fast is effectively hiring randomly and that is way riskier. Would you read random books? Watch random movies? Maybe, but probably not. Would you buy a random car? Of course not. Would you buy a random home? Etc etc You can logistically reduce the number of days by maybe doing multiple interviews into a single day, but that’s still hours. I’ve worked back in the dot com days where you were hired on the spot and that was fun until they fired on the spot. Having a well-designed process that takes a while is not perfect, but is better than the other options. “Slow to hire, slow to fire” is a good maxim.
      - dleslie 5y ago
        I'm not advocating for hiring literally anyone. If they pass your credential requirements, reference checks, and team meet then they're already passing through significant filtering.
- TameAntelope 5y ago
  Have you ever been fired before? It sucks. I’d much rather not get hired then get fired “fast”.
- sokoloff 5y ago
  To be honest, it sounds like "the format with only 3-4 interviews wasn't effective and equitable, so we adjusted it to make it more so, resulting in the current 5-7 format..."
  - dleslie 5y ago
    Yah, and I expect they'll find that insufficient and find themselves considering 8-9 interviews! ;)
- dclowd9901 5y ago
  I’m not sure that’s feasible. Not sure what the roadblocks are, but every company I’ve ever worked for seems to have real reservations, usually at the HR level, with simply getting rid of bad eggs.
- vsareto 5y ago
  Fire fast is a harmful policy to employees in the US due to health insurance being tied to employment.
- analyst74 5y ago
  Amazon follows the hire fast PIP fast strategy. Look what that did to their reputation and company culture. Netflix is a little more successful at this, but they have a much smaller eng footprint and much higher compensation in order to attract and retain top performers.
  - dleslie 5y ago
    Where I am, in BC, Amazon is recognized as a challenging but rewarding employer; one that compensates better than most and expects consistently strong results. I wouldn't consider that negative, really. Daunting, perhaps.
angarg12 5y ago
I was going to say this. To add more: a single person (HM) might have biases for or against certain candidates. More people in the loop brings in diversity and helps to keep interviewers honest and consistent. Also these roles are usually high stakes and the cost of a bad hire is too high.
- 8ytecoder 5y ago
  Most orgs use a veto model. If not everyone is a strong hire, the candidate is rejected. The bias is still there, just not a positive bias now. It’s a strong negative bias. And obviously most people don’t want to come across as “too easy”. Essential you’re positively reinforcing to reject more and more candidates and make the whole process a nightmare.
  - igetspam 5y ago
    I will accept a veto but only if the interviewer can explain it sufficiently. I think it's important to trust but it's also important prevent abuse from bias. (Someone who abuses interview vetoes is likely a poison pill themselves and that's something I have to deal with as a manager.)
  - mabbo 5y ago
    Amazon had an interesting system for this: one of the interviewers, the 'bar raiser' gets the final decision, ideally based on what everyone else says (but not necessarily). That BR has done 100+ interviews, took a lot of additional training, and is empowered to ask a lot of questions to dive into the feedback people are giving about the candidate. So if someone is being too hard, or has a weird bias, the BR can override them. It's a fascinating system and seems to work okay.
- mabbo 5y ago
  Yeah this also is a big issue. It was nice having the "debrief" meeting afterwards. This meant that people had to back up what they had written. "Why do you think this person 'isnt a good culture fit'? Oh because the team you run are all 20-something male brogrammers and this candidate is a 38 year old woman with kids? Hey cool, that's illegal discrimination."
StupidUser123 5y ago
I’m a HM at a big tech company with this format as well. Honestly, I really like it. I don’t want to hire the wrong person, it’s expensive and it makes my job awful for a while. It’s great getting data points on several programming interviews, system design, etc. That makes my sell interview so much easier because I can trust the process to assess their technical skills. You need multiple people because you’re constantly training up new people and need to calibrate everyone fairly. Scaling any process to thousands of people is always hard. Is it annoying for candidates? Yeah, but we pay you a lot of money, and it does actually work for us. Is it perfect? No, but it does work. “Hire and Fire fast” doesn’t actually work well if you want to give people a chance to succeed. Also people tend to keep low performers around too long. Best to avoid this as much as possible.
- alexashka 5y ago
  > Is it perfect? No, but it does work. How do you know it works? > I don’t want to hire the wrong person, it’s expensive and it makes my job awful for a while. There we are. It is all about you. No need to say anything else - your life is all about you, so of course if it works for you, 'it works'.
  - deleted 5y ago
    [deleted]
  - JohnHaugeland 5y ago
    > How do you know it works? Because the hallways aren't empty.
    - mylons 5y ago
      the hallways aren’t empty due to the total compensation and perceived prestige working at a FAANG or similar company offers.
      - dwater 5y ago
        I had a startup give me 2 interviews and a paid coding exercise before making an offer, but the compensation was below market with some long shot equity comp. I had a large consulting company give me one recruiter screen and a series of 3 20 minute interviews, one tech one manager and one in between before making an offer at market rate. Sounds like generally interview time scales with total compensation, which seems fine to me. I’m a data scientist and I’ve never had a live coding interview but I also have no desire to apply to a top tier tech co.
logifail 5y ago
> Can one interview really tell if you're good problem solving, coding, algorithms/data structures, and any specialization the role has? Sorry, but if it can't, aren't you doing the interview wrong?
- azinman2 5y ago
  They all are doing the interview wrong and everyone knows it. Unfortunately there aren’t a lot of great alternatives that don’t take up even more time.
  - logifail 5y ago
    > They all are doing the interview wrong and everyone knows it Most countries manage to devise the testing for a driving licence based on - at most - one theory test and one practical test. The practical test typically includes many different situations and manoeuvres in order to assess the candidate's suitabilty for being allowed to drive. You don't sit a practical driving test in which you consider one single aspect of driving. The test attempts to cover everything. In one test. Why would one not attempt to devise an interview along the same lines?
igetspam 5y ago
Don't forget the need to filter for bias. Many many moons ago, when Google's ATS was young, they didn't weight scores based on the interviewer. After some incredibly confusing low scores and some obviously bad hires, it was determined that there was a need to account for internal bias. That lead to applying weights to the interviewer's score because you still needed them to interview but you also knew that you couldn't trust them outright. More interviewers, more data points and better selection even with accounting for bias.
pqwEfkvjs 5y ago
It is somewhat easy to cheat through these N+1 interviews by grinding tons of practice problems before. At least that was my strategy. Sure I can implement DFS or a linked list, but I am a pretty suboptimal developer, especially when working with huuge codebases. Maybe they should ask to implement a feature or fix a bug in some huge OSS project. That would evaluate skills relevant for a job in FAANG more closely imho.
- danielmarkbruce 5y ago
  Everyone I've ever met who said this was a better engineer than they thought. They seemed to look at the very best engineers, see a difference between themself and that person, and think they weren't that good. I think there is also some kind of survivorship bias - you don't hear from the people who tried it and failed.
- JohnBooty 5y ago
  but I am a pretty suboptimal developer, especially when working with huuge codebases. I'm in the same boat when it comes to huge codebases. I see people being effective on them and I often simply do not know how they do it. But it is for sure, a completely different skill from leetcode style stuff.
gcheong 5y ago
Why not extend this same criterion to the work itself? I mean you can’t really trust that any developer is writing the best code at any given point in time so it would seem at least a minimum of 5 or 6 developers should weigh in on any given pull request. Maybe that is the way it works there idk and probably never will.
- khazhoux 5y ago
  It's extremely common to have that many +1's required for a PR that touches several parts of the codebase. And fixing a bad commit doesn't require 6 months of emotionally draining work.
benreesman 5y ago
> First: we no longer trust the hiring manager alone, because probably they aren't a strong developer This is a huge problem IMHO. I topped out at L7 on the FAANG EM track, but that’s dozens of ICs and a few EMs in your org and I still think you need to be able to build the software and review diffs and write serious ones now and again. Clearly this is now only part of your job, not the focus of it, but it’s very difficult to manage a process that you don’t understand with some sophistication. In everything from law to management consulting to steel fabrication: the person in charge is the most knowledgeable person. Carmack is the best hacker, Mike Krieger wrote code. Hell pg wrote this site and the language it’s written in while building the most successful early-stage investment firm in the world. Obviously directors and VPs and CEOs are delegating the details at same point, but this idea that an L6 manager shouldn’t need to seriously understand the subject matter seems wrong to me both in principle and based on watching it go to hell countless times.
- joshuamorton 5y ago
  An L6 manager and an L6 IC do different work though. And it's rarely (read: never) the case that the person in charge is most knowledgeable about all the details. If an L6 manager could keep all of the details of all of the things all their reports are working on organized, they aren't handling enough scope. Imo that's the difference between 5 and 6. You can no longer track all the details in one person's head.
  - ltbarcly3 5y ago
    > If an L6 manager could keep all of the details of all of the things all their reports are working... That's not what they said, is it? There is a huge difference between what they said and how you interpreted it. They said "you need to be able to build the software and review diffs and write serious ones now and again" and "it’s very difficult to manage a process that you don’t understand with some sophistication" (emphasis mine). How do you get 'know every little detail about everything' from that? It seems obviously true to me that any manager who can't "build the software", "review diffs", and "write serious [diffs] from time to time" is useless. Anyone who disagrees with this is probably dead wood, spends their time fighting political wars about issues they don't understand, and their team is more likely than not constantly fire fighting and in serious trouble (in my opinion). If I had a dollar for every time some clueless manager lectured me about how to write software, and I rolled my eyes and ignored them, and they never found out (because they had no idea what was going on to begin with, and were just posturing based on something someone said) I would have at least $100. I obviously have a chip on my shoulder here, but managers who barely know what is going on still tend to want to 'contribute', which of course they can't do in reality, so they end up playing keyword matching and 'helping' the team avoid 'duplicate effort'. For example, they will see one team building something, and another team building something, and notice some of the words are the same, and then make a big show of 'avoiding duplicate work', but in reality the use cases are extremely different and nothing was being duplicated. Once they have alienated enough of the team, people just start telling them nonsense and they have no way to know it. Productive, smart people start to leave or lose motivation. A fast pace is replaced with constant excuses and a team that is basically no longer showing up for work. The manager can't tell the difference. This is what managers who aren't in the details are like, pretty much without exception.
zerr 5y ago
What do you think about missing on people who are not desperate enough to go through such processes?
bryanrasmussen 5y ago
Ok well, that makes sense on the other hand I went through a 7 interview round and I ended up taking another job before the last call to tell me you got the job! That said it was mainly because they said after several interviews oh you just have one more to go but then the next interview said you have one more to go and they would also say you will be hearing from us in a week but it turned out I heard from them in two weeks. But in the end, actually, both jobs paid the same, but one of them seemed to me to value my time more.
- graderjs 5y ago
  Completely relate to this!
georgeburdell 5y ago
>Second: Is it really fair to have just one or two developers evaluate you? When I first was an interviewer, I liked everybody! I would have hired them all. So getting multiple data points matters. Best to have at least a couple dev interviews. Except FAANG interviews require excellence on almost all of the interviews, so the extra people only represent further opportunities to be denied.
- cableshaft 5y ago
  Exactly. Many of these interviews it only takes one person to feel 'eh' or negative on the person to tank the candidate, so the more people you interview that person, the more likely you're going to have that impression on one of those people (especially if it's an all day series of interviews, you'll probably start to get mentally exhausted near the end of it), and the more likely any given candidate will be rejected.
- heavyset_go 5y ago
  It's the same principle behind using large numbers of disks in certain RAID configurations. If the pool can't tolerate a disk or two failing, adding more disks increases the chances of one or more failures occurring and tanking the pool.
hinkley 5y ago
You’ve already identified several ways to improve this. You don’t want to waste the devs’ time so you screen with low tech questions, which are also the only ones the HM is qualified to ask. (Also, nothing is as expensive to a dev as a manager with too much free time. Spending a little on HR may be a QoL improvement even if we never hire) I am also exceptionally fond of being a silent party in an interview. Nobody wants to walk into a room with four people who are all asking questions, but I can learn a lot about people just by watching their interactions. Indeed I learn more about secondary characteristics than the interviewer because they’re wrapped up in the answer, not side comments the candidate makes about development philosophy. Additionally, it takes less investment in an interview to ride shotgun like this. So jumping from 1 to 2 is not twice as expensive, and it protects you from hiring decisions being made with only one employee having been privy to the conversation. Especially with all of the EEO concerns that come from he said she said situations like this. Three is the most I would put in a room, and only if two are asking questions and the third only answers them. But you can still easily get seven people in front of the candidate with only 3 sessions in this way.
karaterobot 5y ago
That's probably true, but I think at least part of it is just smaller companies doing what bigger companies do, in a kind of cargo cult ritual. The last company I worked at had 8 interviews, which I thought was a lot. My current company (about 100 people) had no less than 11 scheduled interviews! Of those, only the first 2 were with people in my department — my manager first, then my manager and immediate coworkers. There was another group call with people in a related team I might conceivably be expected to liason with eventually. Fair enough. The remaining 8 calls were with leadership in every other division in the company, most of which I would never work with. People with jobs where I don't even know what they do, and I'm sure they had no idea what I do. I just politely made conversation with them and answered their (very general) questions. Now that I've worked there a while, I have never spoken with these people again, or worked in any way with the teams they oversee. I've also participated in a few interviews at this company now, from the other end of the Zoom call, and I know how it works: literally every branch of the org chart gets a meeting for every interviewee, regardless of what they're applying for. Everyone has a chance to say "no", but in practice nobody outside the relevant teams is going to exercise that veto, because they are well aware that they are unqualified to judge the candidate's skills, and will never have to work with them anyway. It's just a big waste of time for everyone involved.
jeffybefffy519 5y ago
Why not just do 1-2 interviews with all relevant parties.
emrah 5y ago
> Is it really fair to have just one or two developers evaluate you? Output of multiple interviewers are combined by AND'ing, so when any one interviewer has even the slightest hesitation, you are out. Because of this, more is not necessarily better
soheil 5y ago
> These roles pay a huge sum of money, so there's a lot of worry that someone will be hired who doesn't really meet the bar, you know? No, I don't. Making the wrong hire is costly not because of the total annual compensation, but because of the upfront cost which is realized the moment the employee signs the offer letter. All the admin stuff. By your logic you shouldn't fire someone either (at least not that easily) and maybe that's why so many FAANG engineers end up "cruising" because the company is so scared of hiring new talent.
ajsnigrutin 5y ago
If we're talking about a one phonecall + one in person session with different teams, lasting 2, 3 hours max, sure. If you expect me to take time off work 4, 5, 6 times and get inteviewed for several hours each time, you're either paying me for the lost time, or I'm not going through the process the moment i find out the length.
imapeopleperson 5y ago
All great points - you obviously condense this into a single day, right?
ozzythecat 5y ago
> First: we no longer trust the hiring manager alone, because probably they aren't a strong developer. We instead trust strong developers that are well trained at evaluating good devs. At the same time, we don't want to thrust a dev onto a hiring manager, so they also need to interview you too and have a say. We decided to ignore this protocol and hired a senior lever manager from IBM. After the next two years, we had an IBM fiefdom. This person hired his IBM buddies, who hired their own IBM talent. None of these people accepted our culture. It reflected in the organization’s output. Turns out, when you bring in people from these legacy companies where people seemingly coast by and don’t actually produce out, and you have little oversight on what they actually output, it just turns into an inefficient and ineffective mess. This went on for a while until the entire org was gutted with over half the people fired.
politelemon 5y ago
> When I first was an interviewer, I liked everybody! I've been interviewing for a long time and I _still_ really like everybody. I want everyone to join and even if they're not a fit I want them to join and want us to spend time on bringing them up to speed and nurturing them. Of course I'm aware that an org isn't always able to do that.
babyshake 5y ago
I'm convinced that the best interview is to give someone an app (react or node app for example) and do exactly what occurs all the time in the real world. Give vague indication that a feature appears to sometimes not work correctly. In the app code there should be one or two very obvious bugs and easy optimizations to make, and then put more subtle and challenging to fix issues there as well. And ideally make it something where a really sharp and experienced developer would identify some high-level architectural flaw and they would know how to rearchitect it. Everyone knows that the leetcode-style questions are contrived and don't usually reflect real world work but we continue to do it anyways.
- JohnBooty 5y ago
  I've done a variation of this exercise with some folks I helped to interview. We showed them our website and asked how they'd investigate/troubleshoot a complaint that the site was "slow." Then we kind of role-played the troubleshooting process. Ideally I wanted them to determine if the site was slow for everybody, vs. just the person that reported the issue. If it was only slow for a single person, was it their account, or many just their internet connection? How would they determine that? If we determined the issue was happening for everybody, how would you determine which part of the stack was slow? Etc.
zeagle 5y ago
Interesting. In medicine they tried to solve this issue by having multiple assessors in the Multi Mini Interview (MMI) [1]. Applicants spend 8-12 minutes with different interviewers in short succession and have a couple minutes to prepare between stations. Each station and assessor has a different question or goal (e.g. critical thinking, communication skills) which are unlinked and it is supposed to remove cumulative biases or the risk of the interview panel having deciding a negative outcome in the first five minutes. It ends up being a lot of behavioural interviewing but seems to work out. [1] https://students-residents.aamc.org/applying-medical-school/what-it-s-participate-multiple-mini-interviews-mmis https://students-residents.aamc.org/applying-medical-school/...
akhmatova 5y ago
These roles pay a huge sum of money, so there's a lot of worry that someone will be hired who doesn't really meet the bar, you know? Except that now of course the practice has been by companies offering roles that don't pay huge sums (and at the end of the day are more or less standard CRUD roles). For the reason you cite at the end of your post
mym1990 5y ago
What you just outlined should be a poster definition for 'a slippery slope'.
miraculixx 5y ago
Well if you want me to invest 5-7 hours of my time you'd better give me some assurances as to why I should do that.
weatherlite 5y ago
> These roles pay a huge sum of money, so there's a lot of worry that someone will be hired who doesn't really meet the bar, you know? So do roles in medicine, finance etc. Do they also go through 7 interviews grilling them about material from their bachelor's or asking random logic questions? Why don't we test doctors "problem solving" skills by asking them how many cars are in Manhattan, don't doctors need good problem solving skills?
jbergens 5y ago
One main issue I see is that it is a huge difference between FAANG and other companies but everyone seems to do the same as FAANG. I don't live in the US but I think even different US companies has different needs. Our current problem is that it is very hard to find developers. A small or medium sized company may not get many candidates to interview, the problem then changes from finding the best in a large pool of candidates to finding anyone that can do the job. Scaring people off with 7 interviews would probably get us even fewer candidates.
codatory 5y ago
I feel like 7+ interviews is selecting for desperation more than any specific skill or fit. That's still likely to yield decent short/mid-term hires, and if the promotion structure keeps those hires around might be a successful strategy (we will leave the candidate's experience out of value judgement entirely) but it will eliminate an entire class of hires who aren't willing to submit to that amount of free time, energy and expertise for the possibility of future opportunities.
musicale 5y ago
> When I first was an interviewer, I liked everybody! I would have hired them all Probably would have worked out just as well as the current system if not better.
xyzzy21 5y ago
Sounds like an organizational flaw is behind all of this.

by mabbo 5y ago

mitchdoogle 5y ago

> "Oh shit, Amazon does 6 interviews? We should do that too!" Therein lies the problem. Doing things because someone bigger and better is doing it is not the right decision making process

dleslie 5y ago

What I'm reading here is that even as an experienced interviewer you struggle to make the format effective and equitable. Sounds to me like the format isn't worth keeping; time to hire more, and fire fast?

lbotos 5y ago
1. Often, legally, firing people is harder than hiring. 2. Hire more, fire fast sounds good on paper but will destroy morale as the team identity will be constantly influx. 3. Engineers will spend less time onboarding newer engineers to the detriment of everyone because well, they might not just last.
- dleslie 5y ago
  Basically everywhere in NA has allowances for probationary periods with at-will employment. Moreover, prospective hires could start on contract. Morale is improved when team members feel like they have some control over their membership. It seems like you're assuming that a great many people would be fired; why so? In my experience, having worked for several such companies, firing is rare because even minimal filters are effective. Resume check, reference check, and meet the team. Done.
  - ratww 5y ago
    > Basically everywhere in NA has allowances for probationary periods with at-will employment. Yep. Lots of places with even stricter pro-employee laws, including Europe, also have it.
  - StupidUser123 5y ago
    I’m not saying you’re categorically wrong, but this doesn’t match with my experience, at all. If you’re an HM and this works for you and your company, great, but the HMs responding here don’t agree. The processes you are questioning scale to thousands of people with varying backgrounds. It’s not an accident all of these companies do this. Your process places extreme trust in one group of people. It’s just so risky. Also, resumes are mostly bullshit, IMO. Reference checks are complete bullshit, IMO. I’ve worked with my buddies at startups (who get lofty titles) and just have them give me reference checks. Other references have asked me for call scripts. Also keep in mind that most of these big tech jobs are highly competitive. Your solution eventually requires a coin toss if you have limited spots. There’s only so much data you can collect from the process you propose. The natural thing is to then assess each candidate a bit more until you’re confident you’ve picked the right one. And then you’re here.
edmundsauto 5y ago
Hire fast and fire requires even more effort and commitment from a candidate, I’m not sure how that would be an improvement. Firing fast would be like 3 months of interviews! I’d also like to add that having multiple interviewers, if they have diverse backgrounds, is more likely to be equitable than a single interviewer.
- dleslie 5y ago
  It's not so much effort or commitment if the team is fully remote.
  - StupidUser123 5y ago
    You’re asking someone to quit their job to maybe get this new job. And you won’t know for months so all your other offers are gone. So if it doesn’t work out, you’re unemployed with no prospects. Sounds terrible. I’d take 15 interviews before taking that risk.
    - dleslie 5y ago
      If it takes months to determine that someone is a bad fit for a team then the team has internal issues that need to be resolved. Moreover, if someone has the skills on paper, and the references to support that, then any tech screening is unnecessary. It just tests for ability to pass tech screens.
      - kyawzazaw 5y ago
        That's what internships are in a way. Do you think that's a bad idea?
        TheCoelacanth 5y ago
        Internships are probably the best way to hire, but they only work for people at that specific stage of life. University students already have a three-month gap in their schedule where they aren't doing anything, so they are willing to take a temporary role to fill that gap. Experienced people who already have a job aren't going to drop that to take on a temporary role unless they have an unusually high risk-tolerance or unless they are desperate.
marssaxman 5y ago
"Hire more, fire fast" does not sound like a company I'd want to work for.
kjeetgill 5y ago
I never understand answers of this format: "status quo has flaw of some known, finite impact" time to "completely ridiculously overcompensate, overreact, and make arbitrary move?" I can't even fathom in what ways you think "time to hire more, and fire fast?" is more "effective and equitable"? You just shove even more risk onto individuals who might be getting their footing still. You're just turning a 8-9 hours commitment to an interview into a 6 month commitment. Sure, you get paid but you also have to deal with churn and burn. I mean, did you think stack ranking was a good idea?
- dleslie 5y ago
  The response is to the needless growth in complexity of the hiring process; advocating for reducing that complexity is appropriate. You're tacitly expressing a concern that a fast hiring commitment would be riskier; but that's not necessarily so. A minimal set of filters are meaningfully effective: resume, references, and meet the team. Anything beyond that doesn't have just diminishing returns, it can negatively impact the company by narrowing the field of viable candidates too far.
  - prepend 5y ago
    > expressing a concern that a fast hiring commitment would be riskier; but that's not necessarily so. I think it is necessarily so. Hiring fast is effectively hiring randomly and that is way riskier. Would you read random books? Watch random movies? Maybe, but probably not. Would you buy a random car? Of course not. Would you buy a random home? Etc etc You can logistically reduce the number of days by maybe doing multiple interviews into a single day, but that’s still hours. I’ve worked back in the dot com days where you were hired on the spot and that was fun until they fired on the spot. Having a well-designed process that takes a while is not perfect, but is better than the other options. “Slow to hire, slow to fire” is a good maxim.
    - dleslie 5y ago
      I'm not advocating for hiring literally anyone. If they pass your credential requirements, reference checks, and team meet then they're already passing through significant filtering.
TameAntelope 5y ago
Have you ever been fired before? It sucks. I’d much rather not get hired then get fired “fast”.
sokoloff 5y ago
To be honest, it sounds like "the format with only 3-4 interviews wasn't effective and equitable, so we adjusted it to make it more so, resulting in the current 5-7 format..."
- dleslie 5y ago
  Yah, and I expect they'll find that insufficient and find themselves considering 8-9 interviews! ;)
dclowd9901 5y ago
I’m not sure that’s feasible. Not sure what the roadblocks are, but every company I’ve ever worked for seems to have real reservations, usually at the HR level, with simply getting rid of bad eggs.
vsareto 5y ago
Fire fast is a harmful policy to employees in the US due to health insurance being tied to employment.
analyst74 5y ago
Amazon follows the hire fast PIP fast strategy. Look what that did to their reputation and company culture. Netflix is a little more successful at this, but they have a much smaller eng footprint and much higher compensation in order to attract and retain top performers.
- dleslie 5y ago
  Where I am, in BC, Amazon is recognized as a challenging but rewarding employer; one that compensates better than most and expects consistently strong results. I wouldn't consider that negative, really. Daunting, perhaps.

angarg12 5y ago

I was going to say this. To add more: a single person (HM) might have biases for or against certain candidates. More people in the loop brings in diversity and helps to keep interviewers honest and consistent. Also these roles are usually high stakes and the cost of a bad hire is too high.

8ytecoder 5y ago
Most orgs use a veto model. If not everyone is a strong hire, the candidate is rejected. The bias is still there, just not a positive bias now. It’s a strong negative bias. And obviously most people don’t want to come across as “too easy”. Essential you’re positively reinforcing to reject more and more candidates and make the whole process a nightmare.
- igetspam 5y ago
  I will accept a veto but only if the interviewer can explain it sufficiently. I think it's important to trust but it's also important prevent abuse from bias. (Someone who abuses interview vetoes is likely a poison pill themselves and that's something I have to deal with as a manager.)
- mabbo 5y ago
  Amazon had an interesting system for this: one of the interviewers, the 'bar raiser' gets the final decision, ideally based on what everyone else says (but not necessarily). That BR has done 100+ interviews, took a lot of additional training, and is empowered to ask a lot of questions to dive into the feedback people are giving about the candidate. So if someone is being too hard, or has a weird bias, the BR can override them. It's a fascinating system and seems to work okay.
mabbo 5y ago
Yeah this also is a big issue. It was nice having the "debrief" meeting afterwards. This meant that people had to back up what they had written. "Why do you think this person 'isnt a good culture fit'? Oh because the team you run are all 20-something male brogrammers and this candidate is a 38 year old woman with kids? Hey cool, that's illegal discrimination."

StupidUser123 5y ago

I’m a HM at a big tech company with this format as well. Honestly, I really like it. I don’t want to hire the wrong person, it’s expensive and it makes my job awful for a while. It’s great getting data points on several programming interviews, system design, etc. That makes my sell interview so much easier because I can trust the process to assess their technical skills. You need multiple people because you’re constantly training up new people and need to calibrate everyone fairly. Scaling any process to thousands of people is always hard. Is it annoying for candidates? Yeah, but we pay you a lot of money, and it does actually work for us. Is it perfect? No, but it does work. “Hire and Fire fast” doesn’t actually work well if you want to give people a chance to succeed. Also people tend to keep low performers around too long. Best to avoid this as much as possible.

alexashka 5y ago
> Is it perfect? No, but it does work. How do you know it works? > I don’t want to hire the wrong person, it’s expensive and it makes my job awful for a while. There we are. It is all about you. No need to say anything else - your life is all about you, so of course if it works for you, 'it works'.
- deleted 5y ago
  [deleted]
- JohnHaugeland 5y ago
  > How do you know it works? Because the hallways aren't empty.
  - mylons 5y ago
    the hallways aren’t empty due to the total compensation and perceived prestige working at a FAANG or similar company offers.
    - dwater 5y ago
      I had a startup give me 2 interviews and a paid coding exercise before making an offer, but the compensation was below market with some long shot equity comp. I had a large consulting company give me one recruiter screen and a series of 3 20 minute interviews, one tech one manager and one in between before making an offer at market rate. Sounds like generally interview time scales with total compensation, which seems fine to me. I’m a data scientist and I’ve never had a live coding interview but I also have no desire to apply to a top tier tech co.

logifail 5y ago

> Can one interview really tell if you're good problem solving, coding, algorithms/data structures, and any specialization the role has? Sorry, but if it can't, aren't you doing the interview wrong?

azinman2 5y ago
They all are doing the interview wrong and everyone knows it. Unfortunately there aren’t a lot of great alternatives that don’t take up even more time.
- logifail 5y ago
  > They all are doing the interview wrong and everyone knows it Most countries manage to devise the testing for a driving licence based on - at most - one theory test and one practical test. The practical test typically includes many different situations and manoeuvres in order to assess the candidate's suitabilty for being allowed to drive. You don't sit a practical driving test in which you consider one single aspect of driving. The test attempts to cover everything. In one test. Why would one not attempt to devise an interview along the same lines?

igetspam 5y ago

Don't forget the need to filter for bias. Many many moons ago, when Google's ATS was young, they didn't weight scores based on the interviewer. After some incredibly confusing low scores and some obviously bad hires, it was determined that there was a need to account for internal bias. That lead to applying weights to the interviewer's score because you still needed them to interview but you also knew that you couldn't trust them outright. More interviewers, more data points and better selection even with accounting for bias.

pqwEfkvjs 5y ago

It is somewhat easy to cheat through these N+1 interviews by grinding tons of practice problems before. At least that was my strategy. Sure I can implement DFS or a linked list, but I am a pretty suboptimal developer, especially when working with huuge codebases. Maybe they should ask to implement a feature or fix a bug in some huge OSS project. That would evaluate skills relevant for a job in FAANG more closely imho.

danielmarkbruce 5y ago
Everyone I've ever met who said this was a better engineer than they thought. They seemed to look at the very best engineers, see a difference between themself and that person, and think they weren't that good. I think there is also some kind of survivorship bias - you don't hear from the people who tried it and failed.
JohnBooty 5y ago
but I am a pretty suboptimal developer, especially when working with huuge codebases. I'm in the same boat when it comes to huge codebases. I see people being effective on them and I often simply do not know how they do it. But it is for sure, a completely different skill from leetcode style stuff.

gcheong 5y ago

Why not extend this same criterion to the work itself? I mean you can’t really trust that any developer is writing the best code at any given point in time so it would seem at least a minimum of 5 or 6 developers should weigh in on any given pull request. Maybe that is the way it works there idk and probably never will.

khazhoux 5y ago
It's extremely common to have that many +1's required for a PR that touches several parts of the codebase. And fixing a bad commit doesn't require 6 months of emotionally draining work.

benreesman 5y ago

> First: we no longer trust the hiring manager alone, because probably they aren't a strong developer This is a huge problem IMHO. I topped out at L7 on the FAANG EM track, but that’s dozens of ICs and a few EMs in your org and I still think you need to be able to build the software and review diffs and write serious ones now and again. Clearly this is now only part of your job, not the focus of it, but it’s very difficult to manage a process that you don’t understand with some sophistication. In everything from law to management consulting to steel fabrication: the person in charge is the most knowledgeable person. Carmack is the best hacker, Mike Krieger wrote code. Hell pg wrote this site and the language it’s written in while building the most successful early-stage investment firm in the world. Obviously directors and VPs and CEOs are delegating the details at same point, but this idea that an L6 manager shouldn’t need to seriously understand the subject matter seems wrong to me both in principle and based on watching it go to hell countless times.

joshuamorton 5y ago
An L6 manager and an L6 IC do different work though. And it's rarely (read: never) the case that the person in charge is most knowledgeable about all the details. If an L6 manager could keep all of the details of all of the things all their reports are working on organized, they aren't handling enough scope. Imo that's the difference between 5 and 6. You can no longer track all the details in one person's head.
- ltbarcly3 5y ago
  > If an L6 manager could keep all of the details of all of the things all their reports are working... That's not what they said, is it? There is a huge difference between what they said and how you interpreted it. They said "you need to be able to build the software and review diffs and write serious ones now and again" and "it’s very difficult to manage a process that you don’t understand with some sophistication" (emphasis mine). How do you get 'know every little detail about everything' from that? It seems obviously true to me that any manager who can't "build the software", "review diffs", and "write serious [diffs] from time to time" is useless. Anyone who disagrees with this is probably dead wood, spends their time fighting political wars about issues they don't understand, and their team is more likely than not constantly fire fighting and in serious trouble (in my opinion). If I had a dollar for every time some clueless manager lectured me about how to write software, and I rolled my eyes and ignored them, and they never found out (because they had no idea what was going on to begin with, and were just posturing based on something someone said) I would have at least $100. I obviously have a chip on my shoulder here, but managers who barely know what is going on still tend to want to 'contribute', which of course they can't do in reality, so they end up playing keyword matching and 'helping' the team avoid 'duplicate effort'. For example, they will see one team building something, and another team building something, and notice some of the words are the same, and then make a big show of 'avoiding duplicate work', but in reality the use cases are extremely different and nothing was being duplicated. Once they have alienated enough of the team, people just start telling them nonsense and they have no way to know it. Productive, smart people start to leave or lose motivation. A fast pace is replaced with constant excuses and a team that is basically no longer showing up for work. The manager can't tell the difference. This is what managers who aren't in the details are like, pretty much without exception.

zerr 5y ago

What do you think about missing on people who are not desperate enough to go through such processes?

bryanrasmussen 5y ago

Ok well, that makes sense on the other hand I went through a 7 interview round and I ended up taking another job before the last call to tell me you got the job! That said it was mainly because they said after several interviews oh you just have one more to go but then the next interview said you have one more to go and they would also say you will be hearing from us in a week but it turned out I heard from them in two weeks. But in the end, actually, both jobs paid the same, but one of them seemed to me to value my time more.

graderjs 5y ago
Completely relate to this!

georgeburdell 5y ago

>Second: Is it really fair to have just one or two developers evaluate you? When I first was an interviewer, I liked everybody! I would have hired them all. So getting multiple data points matters. Best to have at least a couple dev interviews. Except FAANG interviews require excellence on almost all of the interviews, so the extra people only represent further opportunities to be denied.

cableshaft 5y ago
Exactly. Many of these interviews it only takes one person to feel 'eh' or negative on the person to tank the candidate, so the more people you interview that person, the more likely you're going to have that impression on one of those people (especially if it's an all day series of interviews, you'll probably start to get mentally exhausted near the end of it), and the more likely any given candidate will be rejected.
heavyset_go 5y ago
It's the same principle behind using large numbers of disks in certain RAID configurations. If the pool can't tolerate a disk or two failing, adding more disks increases the chances of one or more failures occurring and tanking the pool.

hinkley 5y ago

You’ve already identified several ways to improve this. You don’t want to waste the devs’ time so you screen with low tech questions, which are also the only ones the HM is qualified to ask. (Also, nothing is as expensive to a dev as a manager with too much free time. Spending a little on HR may be a QoL improvement even if we never hire) I am also exceptionally fond of being a silent party in an interview. Nobody wants to walk into a room with four people who are all asking questions, but I can learn a lot about people just by watching their interactions. Indeed I learn more about secondary characteristics than the interviewer because they’re wrapped up in the answer, not side comments the candidate makes about development philosophy. Additionally, it takes less investment in an interview to ride shotgun like this. So jumping from 1 to 2 is not twice as expensive, and it protects you from hiring decisions being made with only one employee having been privy to the conversation. Especially with all of the EEO concerns that come from he said she said situations like this. Three is the most I would put in a room, and only if two are asking questions and the third only answers them. But you can still easily get seven people in front of the candidate with only 3 sessions in this way.

karaterobot 5y ago

That's probably true, but I think at least part of it is just smaller companies doing what bigger companies do, in a kind of cargo cult ritual. The last company I worked at had 8 interviews, which I thought was a lot. My current company (about 100 people) had no less than 11 scheduled interviews! Of those, only the first 2 were with people in my department — my manager first, then my manager and immediate coworkers. There was another group call with people in a related team I might conceivably be expected to liason with eventually. Fair enough. The remaining 8 calls were with leadership in every other division in the company, most of which I would never work with. People with jobs where I don't even know what they do, and I'm sure they had no idea what I do. I just politely made conversation with them and answered their (very general) questions. Now that I've worked there a while, I have never spoken with these people again, or worked in any way with the teams they oversee. I've also participated in a few interviews at this company now, from the other end of the Zoom call, and I know how it works: literally every branch of the org chart gets a meeting for every interviewee, regardless of what they're applying for. Everyone has a chance to say "no", but in practice nobody outside the relevant teams is going to exercise that veto, because they are well aware that they are unqualified to judge the candidate's skills, and will never have to work with them anyway. It's just a big waste of time for everyone involved.

jeffybefffy519 5y ago

Why not just do 1-2 interviews with all relevant parties.

emrah 5y ago

> Is it really fair to have just one or two developers evaluate you? Output of multiple interviewers are combined by AND'ing, so when any one interviewer has even the slightest hesitation, you are out. Because of this, more is not necessarily better

soheil 5y ago

> These roles pay a huge sum of money, so there's a lot of worry that someone will be hired who doesn't really meet the bar, you know? No, I don't. Making the wrong hire is costly not because of the total annual compensation, but because of the upfront cost which is realized the moment the employee signs the offer letter. All the admin stuff. By your logic you shouldn't fire someone either (at least not that easily) and maybe that's why so many FAANG engineers end up "cruising" because the company is so scared of hiring new talent.

ajsnigrutin 5y ago

If we're talking about a one phonecall + one in person session with different teams, lasting 2, 3 hours max, sure. If you expect me to take time off work 4, 5, 6 times and get inteviewed for several hours each time, you're either paying me for the lost time, or I'm not going through the process the moment i find out the length.

imapeopleperson 5y ago

All great points - you obviously condense this into a single day, right?

ozzythecat 5y ago

> First: we no longer trust the hiring manager alone, because probably they aren't a strong developer. We instead trust strong developers that are well trained at evaluating good devs. At the same time, we don't want to thrust a dev onto a hiring manager, so they also need to interview you too and have a say. We decided to ignore this protocol and hired a senior lever manager from IBM. After the next two years, we had an IBM fiefdom. This person hired his IBM buddies, who hired their own IBM talent. None of these people accepted our culture. It reflected in the organization’s output. Turns out, when you bring in people from these legacy companies where people seemingly coast by and don’t actually produce out, and you have little oversight on what they actually output, it just turns into an inefficient and ineffective mess. This went on for a while until the entire org was gutted with over half the people fired.

politelemon 5y ago

> When I first was an interviewer, I liked everybody! I've been interviewing for a long time and I _still_ really like everybody. I want everyone to join and even if they're not a fit I want them to join and want us to spend time on bringing them up to speed and nurturing them. Of course I'm aware that an org isn't always able to do that.

babyshake 5y ago

I'm convinced that the best interview is to give someone an app (react or node app for example) and do exactly what occurs all the time in the real world. Give vague indication that a feature appears to sometimes not work correctly. In the app code there should be one or two very obvious bugs and easy optimizations to make, and then put more subtle and challenging to fix issues there as well. And ideally make it something where a really sharp and experienced developer would identify some high-level architectural flaw and they would know how to rearchitect it. Everyone knows that the leetcode-style questions are contrived and don't usually reflect real world work but we continue to do it anyways.

JohnBooty 5y ago
I've done a variation of this exercise with some folks I helped to interview. We showed them our website and asked how they'd investigate/troubleshoot a complaint that the site was "slow." Then we kind of role-played the troubleshooting process. Ideally I wanted them to determine if the site was slow for everybody, vs. just the person that reported the issue. If it was only slow for a single person, was it their account, or many just their internet connection? How would they determine that? If we determined the issue was happening for everybody, how would you determine which part of the stack was slow? Etc.

zeagle 5y ago

Interesting. In medicine they tried to solve this issue by having multiple assessors in the Multi Mini Interview (MMI) [1]. Applicants spend 8-12 minutes with different interviewers in short succession and have a couple minutes to prepare between stations. Each station and assessor has a different question or goal (e.g. critical thinking, communication skills) which are unlinked and it is supposed to remove cumulative biases or the risk of the interview panel having deciding a negative outcome in the first five minutes. It ends up being a lot of behavioural interviewing but seems to work out. [1] https://students-residents.aamc.org/applying-medical-school/what-it-s-participate-multiple-mini-interviews-mmis https://students-residents.aamc.org/applying-medical-school/...

akhmatova 5y ago

These roles pay a huge sum of money, so there's a lot of worry that someone will be hired who doesn't really meet the bar, you know? Except that now of course the practice has been by companies offering roles that don't pay huge sums (and at the end of the day are more or less standard CRUD roles). For the reason you cite at the end of your post

mym1990 5y ago

What you just outlined should be a poster definition for 'a slippery slope'.

miraculixx 5y ago

Well if you want me to invest 5-7 hours of my time you'd better give me some assurances as to why I should do that.

weatherlite 5y ago

> These roles pay a huge sum of money, so there's a lot of worry that someone will be hired who doesn't really meet the bar, you know? So do roles in medicine, finance etc. Do they also go through 7 interviews grilling them about material from their bachelor's or asking random logic questions? Why don't we test doctors "problem solving" skills by asking them how many cars are in Manhattan, don't doctors need good problem solving skills?

jbergens 5y ago

One main issue I see is that it is a huge difference between FAANG and other companies but everyone seems to do the same as FAANG. I don't live in the US but I think even different US companies has different needs. Our current problem is that it is very hard to find developers. A small or medium sized company may not get many candidates to interview, the problem then changes from finding the best in a large pool of candidates to finding anyone that can do the job. Scaring people off with 7 interviews would probably get us even fewer candidates.

codatory 5y ago

I feel like 7+ interviews is selecting for desperation more than any specific skill or fit. That's still likely to yield decent short/mid-term hires, and if the promotion structure keeps those hires around might be a successful strategy (we will leave the candidate's experience out of value judgement entirely) but it will eliminate an entire class of hires who aren't willing to submit to that amount of free time, energy and expertise for the possibility of future opportunities.

musicale 5y ago

> When I first was an interviewer, I liked everybody! I would have hired them all Probably would have worked out just as well as the current system if not better.

xyzzy21 5y ago

Sounds like an organizational flaw is behind all of this.