8 ms·

Being a hiring manager at a big company, I can tell you this is just as frustrating for me as it is for candidates. I hate “leet code” and frankly find algorith

Being a hiring manager at a big company, I can tell you this is just as frustrating for me as it is for candidates. I hate “leet code” and frankly find algorithmic interviews to be very low signal compared to more practical, open-ended, domain-specific problems. I will say though, the problem is one of “standardization” across an organization where it’s too big for everyone to fit in a room. Suppose you give each team high autonomy to hire whoever they like using whatever “good” process they come up with. 90% of the time this results in good hires. But as you grow, that ten percent of underperforming people becomes large in absolute numbers, and is very painful to deal with. It becomes a real problem when relatively lower performing people end up concentrated on a team, and then start being the hiring gatekeepers for that team, thus multiplying the number of lower performing hires. Later you start having institutional problems when everyone starts to perceive that the engineers in Department A are generally better than the engineers in Department B. Engineers in Department A are more likely to leave if they perceive the company is getting worse at engineering - it becomes a self-fulfilling prophecy. Then you get enormous pressure to come up with standardized testing - aka algorithms on the whiteboard, or some other academic inspired exercise - imposed by higher level leadership that wants to address a genuine problem (skill disparity across the org) but does not know any better way to do it. I think, as PG points out, there may be a real opportunity to innovate here, and probably a big financial opportunity if anyone can figure out how to productize and scale a solution. I struggle to see an easy answer, though. In a utopian universe (for a hiring manager) I’d do something like pay candidates to come on site and work for a week, then make a hire/no-hire decision based on that. But I think that is far too onerous for candidates (and a big company) to have legs.

lifeisstillgood 7y ago
My solution is smaller companies - that way the underperforming ones can be eliminated more easily.
- TeMPOraL 7y ago
  A federation. To be honest, if a large company has multiple departments that essentially govern themselves, perhaps they could just give them proper names, as if they were subsidiary companies? Because it seems to me that the mechanism behind the "best performers leave when they feel the average performance of workers in the company goes down" is really bounded by the company name. I.e. people in department of frobnification leave because of the people in department of foobing bars. But people in Frobnificator perhaps wouldn't leave because of people in FooBars.io, even if legally/administratively the two situations were equivalent.
  - scarejunba 7y ago
    No, it's not that. It's that often, to make a new product you have to have multiple parts of the business work together. Then you will find yourself working with the other department and they'll frustrate you. Some places attempt to solve this by saying that you can use whatever vendors you need to solve the problem but often the best place to solve it is the other department since the solution is infeasible without them. The problem is that often the solution is infeasible with them because the guys who made them good have left and now you only have the lameos. As to why they can't operate as independent business unit? Well, there's the Theory of the Firm to tell you why that doesn't work often.
- alien_ 7y ago
  At Amazon, perhaps one of the largest and most successful employers, the solution to this is that when interviewing, everyone's main goal is to hire people better than the current average of the team. To ensure this, someone experienced with interviewing from outside the team/organization, so called bar raiser, is always added to the interviewing team. This person is there to ensure high quality hires and has hiring veto power. The rest of the interviewers are from within the team, likely under pressure to compromise on quality when they're desperate enough to hire someone, but they can't do it unless the bar raiser agreed.
  - halflings 7y ago
    FWIW this wasn't my experience when interviewing with Amazon; I felt the bar was much lower than other companies I interviewed with, even when I wasn't that good at algorithmic problems. That's just one data point, and maybe what you're describing is for more senior hires?
    - kudokatz 7y ago
      I've found a generally higher bar than places like Microsoft but definitely a much lower bar than other FAANG companies, unicorns, and even tiny-but-potentially-elite startups run by people who know what they're doing.
      - halflings 7y ago
        Agreed, Microsoft was an even lower bar... but in a good way: I felt they just wanted someone with basic CS knowledge, whereas the Amazon interviewers seemed to think they were some sort of elite group of engineers.
  - kudokatz 7y ago
    > At Amazon, perhaps one of the largest and most successful employers, the solution to this is that when interviewing, everyone's main goal is to hire people better than the current average of the team. At this point it's Kool-aid. In practice bar raisers have so much more interview experience that you can very easily pick them out, but so much of their behavior results in simply weeding out the bad. I have received LOTS of offers from Amazon over more than a decade (only one accepted at one point), and my friend used to be a bar raiser. In his first interview at a new company he was both rude to an interviewee and flat-out wrong technically on multiple points along with it. You can't raise the bar if you can't pick the bar raisers.
blablabla123 7y ago
>But I think that is far too onerous for candidates (and a big company) to have legs. Actually I did exactly this as a candidate at one place for an afternoon. I left after 2.5 months. :-) In the end I felt a bit sorry for them since they put a lot of effort into the process, being a really small company. On the other hand I really didn't want to stay there. It was technically really interesting and I could contribute but the business was run really badly IMHO
OnlineGladiator 7y ago
> But as you grow, that ten percent of underperforming people becomes large in absolute numbers, and is very painful to deal with. I think you could reduce the 10% to 1% simply by actively firing bad employees, and the easiest way to do that is with a probationary hiring period (let's say 6 months, maybe a year). I am fully aware of how systemic rot can be, and how hard it can be to remove. The real problem is when you hire a bad manager, because they will not only let the rot fester but encourage it to grow. I think you could wildly relax hiring standards for ICs and be incredibly successful. What we need are better practices around hiring managers more than anything.
- johnm 7y ago
  Alas, for all of the focus on the interview processes to "prevent" false positive hires, those same companies suck as actually getting rid of the bad hires. Some is rational risk aversion to lawsuits and the like but that clearly bs when you see just how fast people that aren't liked by someone high enough up the food chain are shown the door. The effect on morale and real productivity of getting rid of the toxic and worst performers is amazing.
  - OnlineGladiator 7y ago
    The first job I had there was a probationary period - it was good and bad. It was good because you knew everybody there was extremely talented and you could rely on your coworkers. There was a bit of an understanding about the cut throat nature of the place though, and obviously that's not for everyone. But at the end of the day, when it came time to just get shit done, there was never any problem. I can't name that place without doxing myself, but it sounds like Netflix has a similar culture. Some people love it. Some people hate it. But I don't think many people would say they have an issue with employing less-than-stellar employees. Of course this is only "solving" (it's certainly not perfect) the problem of hiring for engineers, which most of us are on HN (at least I assume, but certainly more engineers than engineering managers). It does nothing to address the issue of hiring bad managers. I have no solution for that, other than to make sure the first manager is excellent. A players hire A players because they want to do the best work they can, and will go out of their way to hire people better than themselves. B players hire C players because they just want to make themselves look good. Once you have a bad manager, the assumption is everyone beneath him will be as bad or worse (obviously there are exceptions).
TeMPOraL 7y ago
> But as you grow, that ten percent of underperforming people becomes large in absolute numbers, and is very painful to deal with. I feel that beyond hiring, our entire society has a problem with tails of distributions. There is this strong movement behind standardization of everything everywhere to minimize the variance, but that not only eliminates both tails, it also lowers the mean. So e.g. here, in order to avoid exceptionally bad hires, companies not only forgo the chance of getting exceptionally good hires, they also lower the quality of a typical hire.
ChrisMarshallNY 7y ago
I was a hiring manager at a big company for years. We never did coding tests, and I like to think that I made good choices every time. I kept a high-functioning team together, under fairly humble pay, and stressful, sometimes demoralizing, conditions, for decades. I'm mediocre, at best, at these tests. I don't come from a traditional CS background (started as an EE). I tend to take unusual, hybrid approaches to solving problems; often incorporating elements of new-fangled tech with patterns that have been around for thirty years, and I've found that people get VERY uncomfortable with "thinking outside the box." I also tend to take some time arriving at the final release, going through iterations of improvement. My first bash is usually fairly naive and clumsy. It works, but not so well. I make each iteration improve on it, maintaining that working software throughout; so there's always something working and testable. Despite all the aspiration to "disruption," it seems that people don't like to step out of their comfort zones. Making software that SHIPS is a learned and earned skill, and one that I believe, can be most effectively demonstrated with portfolios. When a designer goes for a job, they bring with them a large black case. It's filled with their designs and working drafts. Much of the interview consists of the designer going through this portfolio with the hiring manager; discussing each example, and talking about why they did this, or why they didn't do that, etc. No design manager in their right mind would ignore that portfolio; instead, throwing a matchbook on the interview table, and asking the applicant to "Draw Spunky." I'm actually shocked that there is so little importance placed on software portfolios. A portfolio represents fairly substantial work product. That's why I find open-source contribution work to be so attractive.
- xvector 7y ago
  Unfortunately portfolios would make finding jobs very difficult for myself and my colleagues. We aren’t really allowed to talk about what we work on outside of what could easily fit in a vague, 1-sentence summary on LinkedIn. This is actually frustrating me quite a bit because as I look for new jobs people like to ask, in detail, what I worked on, and I can’t really tell them.
  - ChrisMarshallNY 7y ago
    I completely understand. That was a big reason that I learned to draw people out when they interviewed. I asked them to "tell me stories" about projects they worked on, without getting detailed about specifics. In my experience, they were always able to demonstrate plenty of enthusiasm and creativity without revealing the family jewels. A half-hour whiteboard test would have been completely useless. I was usually able to establish a fairly comfortable level of tech knowledge fairly quickly, and the bulk of the interview was really about how well I thought they'd fit the team (NOTE TO SELF: Don't go for homogeneity). However, this isn't a problem that's unique to our industry. My father was in the CIA, and I never found out until just a few years before his death. Lawyers have legal cases they can't discuss, doctors have medical cases they can't discuss in detail, without violation of HIPAA, etc. Somehow or another, these folks are able to interview for jobs that often have a far greater risk than a line programmer on a major initiative. True, some are screw-ups, and that doesn't become apparent until after they are hired, but the same goes for an engineer that can ace every problem on HackerRank, but goes all to pieces, when presented with 100 KLoC of spaghetti code, and told to fix a problem (exactly what happened to me, on one of my first programming jobs. 100 KLoC of 1970s-vintage FORTRAN -I fixed the bug, but had to hold my nose, while doing it. I later learned that this was what everyone did). What bothers me most, is that a chief reason that I'm given for these tests, is that companies want to find people that can come up with innovative and unique solutions, yet they seem to actually have the opposite effect; filtering for people that only come up with standard solutions. I remember once doing a "take home" test that asked me to apply a third-party library. I did what I always do with dependencies; I encapsulated it, and that seemed to totally freak out the interviewer. To this day, I have no idea why they lost their bottle so badly over it. My solution worked extremely well, and it also gave the problem tremendous "future-proofing." That's why I encapsulate dependencies.
AlwaysBCoding 7y ago
it's a really simple problem honestly. Make a standardized test, but don't put an hour time limit on it. If you gave the exact same algorithm test with a 24-hour time limit vs. a 1-hour time limit then had a 1-hour interview explaining the solution to the problem you would be testing for something closer to programming acumen than memorization. I have no idea why tech companies find this so challenging.
- halflings 7y ago
  Cheating? Even with phone screens, some candidates still cheat at these interviews (have some engineer with them answering questions, or looking up similar questions online). I agree a full blown project over 24 hours would be better, but it's more costly to create questions and score them (if you want to do it in a way where cheating is hard/impossible). I've seen startups do this, and it works well for them, but might not be scalable for companies that hire thousands of employees each year.
  - AlwaysBCoding 7y ago
    what even is cheating on an algorithm problem? looking up an answer online? asking someone in your social network to help you work through the problem? because that's what you actually do in the real world. as long as you can explain / defend your solution what does cheating even mean.
    - halflings 7y ago
      That's an idealistic way of looking at things. People would look-up solutions to that exact problem; it's very hard to come up with a unique problem that was never asked before, and questions leak pretty quickly. (especially at larger companies) The goal of a test is to evaluate whether you'd be a good employee; I agree algorithmic questions are not representative of day-to-day work, but evaluating your friend's ability to help you is out of scope. Companies want to hire someone that has decent programming skills; you can't rely on other people to solve all your problems, you need to have a minimum level of skill. [again, playing devil's advocate here; we all agree the process is suboptimal, but let's not ignore the negatives of some of the alternatives suggested here]
richmarr 7y ago
> I will say though, the problem is one of “standardization” across an organization where it’s too big for everyone to fit in a room. I think you've got a lot of this right (disclaimer: we've built the product I think you're describing) The don't think the most important problem is standardisation though, it's observability/instrumentation ie. if you don't measure what's working, you can't improve things. The very best tech companies measure quite a lot, and often look back at their hiring processes in the event of a mis-hire to figure out what went wrong and how they can avoid the same happening in future... but even then they only do that in exceptional cases because it's done fairly manually. That means they have low statistical significance and a stuttering cycle of learning. I believe they should be constantly looking at what's working well, for every hire. So that's what we built. Once your hiring pipeline is trivially visible, a lot of these questions go away. You can see what's working well and try new things in safety, you can optimise with your eyes wide open. One thing we did straight away was to deprioritise CVs and replace them with written scenario-based questions relevant to the job. If managed properly that takes your sift stage from a predictive power around r=0.3 to a performance we find typically above r=0.6. Far fewer early false negatives makes your hiring funnel (a) less leaky, (b) more open to pools of talent previously ruled out by clumsy CV sifting, and (c) potentially shorter as the improved sift accuracy allows companies to consider dropping their phone interview stage(s) Our NPS rating for HR teams is currently running at 85, and MRR churn is under 1% so there's clearly some value to the approach.
vinniejames 7y ago
"do something like pay candidates to come on site and work for a week" by doing this you've essentially cut your hiring pool down to 3 categories: new graduates; mediocre engineers who were fired from their last job; or terrible engineers who can't find a job.
- roland35 7y ago
  Or freelancers who probably aren't looking for full time work anyways! But yeah this is exactly why a weeklong trial won't work
- hnick 7y ago
  The other option is taking leave. In many countries people might have a week saved up, or maybe they work for one of those fancy companies with Unlimited Leave* * conditions apply.
marcosdumay 7y ago
If you can't stop bad people from accumulating, you will have a problem no matter how selective is your hiring process.
- mavhc 7y ago
  Can't you just unhire them?
  - johnm 7y ago
    Yes but there's all sorts of silliness that people and companies need to unlearn so they can do this effectively and efficiently.
  - WrtCdEvrydy 7y ago
    Hesitation to fire is real.
  - marcosdumay 7y ago
    Firing is too hard for humans, and too easy for bureaucracies.
greggman2 7y ago
I would like to see data on if these interview systems actually achieved their goal of rejecting bad candidates. I worked at a FAANG company and saw several people get let go for underperforming. I have no idea if they could have been productive doing something else but they were clearly not productive at the tasks assigned them or that they chose to do. I also saw several other people I thought were a candidate for going down the same path before I left. Whether they got moved to something they were more productive on or whether they were showing value in other ways I have no idea. The point being the hiring system didn't reject these people. My guess is they either got lucky like me, easy questions, or they are good at answering the questions but not good in actual production. For example one person that was let go was clearly smart and could make the code but they would refactor forever looking for perfection and end up taking 4x - 8x longer than others around them. Maybe in some place that's a plus but for our team we needed to ship and this person was not able to prioritize shipping and after several attempts they were let go. Another had a task that they were taking a long time on. Others on the team gave them the benefit of the doubt that it was harder than it looked but when they were finally let go it turned out it was not harder than it looked and was finished rather quickly. Another wrote bloated obfusticated code that seemed to be 6x the number of lines it needed to be. Maybe they could have been helped via code review. Yet another was one I interviewed and they passed the "can write code test" but in the entire time I was there I don't think I saw them submit a single PR. I have no idea what was filling their time. I don't know if that ratio of unproductive candidates is better or worse than anywhere else I've worked on average. Pretty much every place I worked before FAANG was 100 people or less and my team 30 people or less and those teams were 10 engineers or less. There were always 1 or 2 engineers in the company known for being slow or writing bad designs. Sounds about the same but I didn't measure.

lifeisstillgood 7y ago

My solution is smaller companies - that way the underperforming ones can be eliminated more easily.

TeMPOraL 7y ago
A federation. To be honest, if a large company has multiple departments that essentially govern themselves, perhaps they could just give them proper names, as if they were subsidiary companies? Because it seems to me that the mechanism behind the "best performers leave when they feel the average performance of workers in the company goes down" is really bounded by the company name. I.e. people in department of frobnification leave because of the people in department of foobing bars. But people in Frobnificator perhaps wouldn't leave because of people in FooBars.io, even if legally/administratively the two situations were equivalent.
- scarejunba 7y ago
  No, it's not that. It's that often, to make a new product you have to have multiple parts of the business work together. Then you will find yourself working with the other department and they'll frustrate you. Some places attempt to solve this by saying that you can use whatever vendors you need to solve the problem but often the best place to solve it is the other department since the solution is infeasible without them. The problem is that often the solution is infeasible with them because the guys who made them good have left and now you only have the lameos. As to why they can't operate as independent business unit? Well, there's the Theory of the Firm to tell you why that doesn't work often.
alien_ 7y ago
At Amazon, perhaps one of the largest and most successful employers, the solution to this is that when interviewing, everyone's main goal is to hire people better than the current average of the team. To ensure this, someone experienced with interviewing from outside the team/organization, so called bar raiser, is always added to the interviewing team. This person is there to ensure high quality hires and has hiring veto power. The rest of the interviewers are from within the team, likely under pressure to compromise on quality when they're desperate enough to hire someone, but they can't do it unless the bar raiser agreed.
- halflings 7y ago
  FWIW this wasn't my experience when interviewing with Amazon; I felt the bar was much lower than other companies I interviewed with, even when I wasn't that good at algorithmic problems. That's just one data point, and maybe what you're describing is for more senior hires?
  - kudokatz 7y ago
    I've found a generally higher bar than places like Microsoft but definitely a much lower bar than other FAANG companies, unicorns, and even tiny-but-potentially-elite startups run by people who know what they're doing.
    - halflings 7y ago
      Agreed, Microsoft was an even lower bar... but in a good way: I felt they just wanted someone with basic CS knowledge, whereas the Amazon interviewers seemed to think they were some sort of elite group of engineers.
- kudokatz 7y ago
  > At Amazon, perhaps one of the largest and most successful employers, the solution to this is that when interviewing, everyone's main goal is to hire people better than the current average of the team. At this point it's Kool-aid. In practice bar raisers have so much more interview experience that you can very easily pick them out, but so much of their behavior results in simply weeding out the bad. I have received LOTS of offers from Amazon over more than a decade (only one accepted at one point), and my friend used to be a bar raiser. In his first interview at a new company he was both rude to an interviewee and flat-out wrong technically on multiple points along with it. You can't raise the bar if you can't pick the bar raisers.

blablabla123 7y ago

>But I think that is far too onerous for candidates (and a big company) to have legs. Actually I did exactly this as a candidate at one place for an afternoon. I left after 2.5 months. :-) In the end I felt a bit sorry for them since they put a lot of effort into the process, being a really small company. On the other hand I really didn't want to stay there. It was technically really interesting and I could contribute but the business was run really badly IMHO

OnlineGladiator 7y ago

> But as you grow, that ten percent of underperforming people becomes large in absolute numbers, and is very painful to deal with. I think you could reduce the 10% to 1% simply by actively firing bad employees, and the easiest way to do that is with a probationary hiring period (let's say 6 months, maybe a year). I am fully aware of how systemic rot can be, and how hard it can be to remove. The real problem is when you hire a bad manager, because they will not only let the rot fester but encourage it to grow. I think you could wildly relax hiring standards for ICs and be incredibly successful. What we need are better practices around hiring managers more than anything.

johnm 7y ago
Alas, for all of the focus on the interview processes to "prevent" false positive hires, those same companies suck as actually getting rid of the bad hires. Some is rational risk aversion to lawsuits and the like but that clearly bs when you see just how fast people that aren't liked by someone high enough up the food chain are shown the door. The effect on morale and real productivity of getting rid of the toxic and worst performers is amazing.
- OnlineGladiator 7y ago
  The first job I had there was a probationary period - it was good and bad. It was good because you knew everybody there was extremely talented and you could rely on your coworkers. There was a bit of an understanding about the cut throat nature of the place though, and obviously that's not for everyone. But at the end of the day, when it came time to just get shit done, there was never any problem. I can't name that place without doxing myself, but it sounds like Netflix has a similar culture. Some people love it. Some people hate it. But I don't think many people would say they have an issue with employing less-than-stellar employees. Of course this is only "solving" (it's certainly not perfect) the problem of hiring for engineers, which most of us are on HN (at least I assume, but certainly more engineers than engineering managers). It does nothing to address the issue of hiring bad managers. I have no solution for that, other than to make sure the first manager is excellent. A players hire A players because they want to do the best work they can, and will go out of their way to hire people better than themselves. B players hire C players because they just want to make themselves look good. Once you have a bad manager, the assumption is everyone beneath him will be as bad or worse (obviously there are exceptions).

TeMPOraL 7y ago

> But as you grow, that ten percent of underperforming people becomes large in absolute numbers, and is very painful to deal with. I feel that beyond hiring, our entire society has a problem with tails of distributions. There is this strong movement behind standardization of everything everywhere to minimize the variance, but that not only eliminates both tails, it also lowers the mean. So e.g. here, in order to avoid exceptionally bad hires, companies not only forgo the chance of getting exceptionally good hires, they also lower the quality of a typical hire.

ChrisMarshallNY 7y ago

I was a hiring manager at a big company for years. We never did coding tests, and I like to think that I made good choices every time. I kept a high-functioning team together, under fairly humble pay, and stressful, sometimes demoralizing, conditions, for decades. I'm mediocre, at best, at these tests. I don't come from a traditional CS background (started as an EE). I tend to take unusual, hybrid approaches to solving problems; often incorporating elements of new-fangled tech with patterns that have been around for thirty years, and I've found that people get VERY uncomfortable with "thinking outside the box." I also tend to take some time arriving at the final release, going through iterations of improvement. My first bash is usually fairly naive and clumsy. It works, but not so well. I make each iteration improve on it, maintaining that working software throughout; so there's always something working and testable. Despite all the aspiration to "disruption," it seems that people don't like to step out of their comfort zones. Making software that SHIPS is a learned and earned skill, and one that I believe, can be most effectively demonstrated with portfolios. When a designer goes for a job, they bring with them a large black case. It's filled with their designs and working drafts. Much of the interview consists of the designer going through this portfolio with the hiring manager; discussing each example, and talking about why they did this, or why they didn't do that, etc. No design manager in their right mind would ignore that portfolio; instead, throwing a matchbook on the interview table, and asking the applicant to "Draw Spunky." I'm actually shocked that there is so little importance placed on software portfolios. A portfolio represents fairly substantial work product. That's why I find open-source contribution work to be so attractive.

xvector 7y ago
Unfortunately portfolios would make finding jobs very difficult for myself and my colleagues. We aren’t really allowed to talk about what we work on outside of what could easily fit in a vague, 1-sentence summary on LinkedIn. This is actually frustrating me quite a bit because as I look for new jobs people like to ask, in detail, what I worked on, and I can’t really tell them.
- ChrisMarshallNY 7y ago
  I completely understand. That was a big reason that I learned to draw people out when they interviewed. I asked them to "tell me stories" about projects they worked on, without getting detailed about specifics. In my experience, they were always able to demonstrate plenty of enthusiasm and creativity without revealing the family jewels. A half-hour whiteboard test would have been completely useless. I was usually able to establish a fairly comfortable level of tech knowledge fairly quickly, and the bulk of the interview was really about how well I thought they'd fit the team (NOTE TO SELF: Don't go for homogeneity). However, this isn't a problem that's unique to our industry. My father was in the CIA, and I never found out until just a few years before his death. Lawyers have legal cases they can't discuss, doctors have medical cases they can't discuss in detail, without violation of HIPAA, etc. Somehow or another, these folks are able to interview for jobs that often have a far greater risk than a line programmer on a major initiative. True, some are screw-ups, and that doesn't become apparent until after they are hired, but the same goes for an engineer that can ace every problem on HackerRank, but goes all to pieces, when presented with 100 KLoC of spaghetti code, and told to fix a problem (exactly what happened to me, on one of my first programming jobs. 100 KLoC of 1970s-vintage FORTRAN -I fixed the bug, but had to hold my nose, while doing it. I later learned that this was what everyone did). What bothers me most, is that a chief reason that I'm given for these tests, is that companies want to find people that can come up with innovative and unique solutions, yet they seem to actually have the opposite effect; filtering for people that only come up with standard solutions. I remember once doing a "take home" test that asked me to apply a third-party library. I did what I always do with dependencies; I encapsulated it, and that seemed to totally freak out the interviewer. To this day, I have no idea why they lost their bottle so badly over it. My solution worked extremely well, and it also gave the problem tremendous "future-proofing." That's why I encapsulate dependencies.

AlwaysBCoding 7y ago

it's a really simple problem honestly. Make a standardized test, but don't put an hour time limit on it. If you gave the exact same algorithm test with a 24-hour time limit vs. a 1-hour time limit then had a 1-hour interview explaining the solution to the problem you would be testing for something closer to programming acumen than memorization. I have no idea why tech companies find this so challenging.

halflings 7y ago
Cheating? Even with phone screens, some candidates still cheat at these interviews (have some engineer with them answering questions, or looking up similar questions online). I agree a full blown project over 24 hours would be better, but it's more costly to create questions and score them (if you want to do it in a way where cheating is hard/impossible). I've seen startups do this, and it works well for them, but might not be scalable for companies that hire thousands of employees each year.
- AlwaysBCoding 7y ago
  what even is cheating on an algorithm problem? looking up an answer online? asking someone in your social network to help you work through the problem? because that's what you actually do in the real world. as long as you can explain / defend your solution what does cheating even mean.
  - halflings 7y ago
    That's an idealistic way of looking at things. People would look-up solutions to that exact problem; it's very hard to come up with a unique problem that was never asked before, and questions leak pretty quickly. (especially at larger companies) The goal of a test is to evaluate whether you'd be a good employee; I agree algorithmic questions are not representative of day-to-day work, but evaluating your friend's ability to help you is out of scope. Companies want to hire someone that has decent programming skills; you can't rely on other people to solve all your problems, you need to have a minimum level of skill. [again, playing devil's advocate here; we all agree the process is suboptimal, but let's not ignore the negatives of some of the alternatives suggested here]

richmarr 7y ago

> I will say though, the problem is one of “standardization” across an organization where it’s too big for everyone to fit in a room. I think you've got a lot of this right (disclaimer: we've built the product I think you're describing) The don't think the most important problem is standardisation though, it's observability/instrumentation ie. if you don't measure what's working, you can't improve things. The very best tech companies measure quite a lot, and often look back at their hiring processes in the event of a mis-hire to figure out what went wrong and how they can avoid the same happening in future... but even then they only do that in exceptional cases because it's done fairly manually. That means they have low statistical significance and a stuttering cycle of learning. I believe they should be constantly looking at what's working well, for every hire. So that's what we built. Once your hiring pipeline is trivially visible, a lot of these questions go away. You can see what's working well and try new things in safety, you can optimise with your eyes wide open. One thing we did straight away was to deprioritise CVs and replace them with written scenario-based questions relevant to the job. If managed properly that takes your sift stage from a predictive power around r=0.3 to a performance we find typically above r=0.6. Far fewer early false negatives makes your hiring funnel (a) less leaky, (b) more open to pools of talent previously ruled out by clumsy CV sifting, and (c) potentially shorter as the improved sift accuracy allows companies to consider dropping their phone interview stage(s) Our NPS rating for HR teams is currently running at 85, and MRR churn is under 1% so there's clearly some value to the approach.

vinniejames 7y ago

"do something like pay candidates to come on site and work for a week" by doing this you've essentially cut your hiring pool down to 3 categories: new graduates; mediocre engineers who were fired from their last job; or terrible engineers who can't find a job.

roland35 7y ago
Or freelancers who probably aren't looking for full time work anyways! But yeah this is exactly why a weeklong trial won't work
hnick 7y ago
The other option is taking leave. In many countries people might have a week saved up, or maybe they work for one of those fancy companies with Unlimited Leave* * conditions apply.

marcosdumay 7y ago

If you can't stop bad people from accumulating, you will have a problem no matter how selective is your hiring process.

mavhc 7y ago
Can't you just unhire them?
- johnm 7y ago
  Yes but there's all sorts of silliness that people and companies need to unlearn so they can do this effectively and efficiently.
- WrtCdEvrydy 7y ago
  Hesitation to fire is real.
- marcosdumay 7y ago
  Firing is too hard for humans, and too easy for bureaucracies.

greggman2 7y ago

I would like to see data on if these interview systems actually achieved their goal of rejecting bad candidates. I worked at a FAANG company and saw several people get let go for underperforming. I have no idea if they could have been productive doing something else but they were clearly not productive at the tasks assigned them or that they chose to do. I also saw several other people I thought were a candidate for going down the same path before I left. Whether they got moved to something they were more productive on or whether they were showing value in other ways I have no idea. The point being the hiring system didn't reject these people. My guess is they either got lucky like me, easy questions, or they are good at answering the questions but not good in actual production. For example one person that was let go was clearly smart and could make the code but they would refactor forever looking for perfection and end up taking 4x - 8x longer than others around them. Maybe in some place that's a plus but for our team we needed to ship and this person was not able to prioritize shipping and after several attempts they were let go. Another had a task that they were taking a long time on. Others on the team gave them the benefit of the doubt that it was harder than it looked but when they were finally let go it turned out it was not harder than it looked and was finished rather quickly. Another wrote bloated obfusticated code that seemed to be 6x the number of lines it needed to be. Maybe they could have been helped via code review. Yet another was one I interviewed and they passed the "can write code test" but in the entire time I was there I don't think I saw them submit a single PR. I have no idea what was filling their time. I don't know if that ratio of unproductive candidates is better or worse than anywhere else I've worked on average. Pretty much every place I worked before FAANG was 100 people or less and my team 30 people or less and those teams were 10 engineers or less. There were always 1 or 2 engineers in the company known for being slow or writing bad designs. Sounds about the same but I didn't measure.