5 ms·

Before we criticize the current interview format and propose alternatives, we need to understand how we got here first. This is my understanding of what happene

Before we criticize the current interview format and propose alternatives, we need to understand how we got here first. This is my understanding of what happened (I wasn't there for most of this!). Leetcode-style interviews became popular in the mid 00s, primarily because they were used by hot tech companies of the time. The thing to understand is that at that time, the idea of asking people to write code during an interview what sort of revolutionary. Prior to that interviews had little structure. It wasn't unusual for the hiring manager to make the decision, some times based on credentials, recommendations, or trivial-like questions. This type of interview became wildly popular because it allowed budding unicorns to hire programmers of high quality at scale. The process was less biased than the alternative, reproducible and scalable. Here you have two blog posts [1][2] that show the line of thought at the time. The reality is that big tech has elevated leetcode type interview to an art. They have reached a near local optimal through years of experiments and refinements. It is working well for them so they don't have the need to take big risks such as completely revamping their hiring process. I love the topic of hiring and interviewing and I'd love to truly get at the bottom of which method works best. I like this article because it explicitly calls out shortcomings with typical alternatives that are not usually mentioned. I hope in the future a new crop of unicorns can take these practices to the next level and do a fair comparison. [1] https://www.joelonsoftware.com/2006/10/25/the-guerrilla-guide-to-interviewing-version-30/ https://www.joelonsoftware.com/2006/10/25/the-guerrilla-guid... [2] https://sites.google.com/site/steveyegge2/five-essential-phone-screen-questions https://sites.google.com/site/steveyegge2/five-essential-pho...

jsnk 4y ago
This is probably the best take in the entire comment section. I see whole bunch of people complaining about Leetcode (LC) style coding questions, but I don't really see any alternative that is as good as LC. LC interviews solve all the following criteria better than other interview formats in aggregate. - Objectivity: can you evaluate candidates as objectively as possible as opposed to subjectivity? - Scalability: can you evaluate many candidates fast with high throughput? - Accuracy: can you filter out bad candidates? Yes, the filter is often restrictive to the point where good candidates also get filtered out unfortunately. - Meritocratic: can you make sure the process is assessed based on the merit of solving LC questions well as opposed to having connection to someone you know at the company or having gone to some prestigious school or other companies? Leetcode format does good in all criteria above, or at least better than other formats below in aggregate. 1. Take home assignments. - Objectivity: bad - Scalability: bad. Once the question leaks, it's harder to come up with another question easily. Also doesn't scale for candidate side either. As a rule of thumb, I always decline interviews that require take home because it's useless to try hard as this is not applicable to interview at other companies. ROI is extremely low for candidates. - Accuracy: good. - Meritocratic: good 2. Coding questions with heavy emphasis on practicality (Stripe & Square) I was pretty impressed with questions from Stripe and Square. They created problems that are heavily related to string, array, hashmap manipulation, searching, replacing etc using regex and such. If you are not into LC questions, you might prefer this, but I actually found some of these questions harder than dynamic programming questions I got from Google. - Objectivity: good - Scalability: bad. Once the question leaks, it's harder to come up with another question easily. - Accuracy: good. - Meritocratic: good 3. Knowledge / Trivia questions about programming language or framework This might make sense a bit if you are specifically looking for iOS, Android or web developer but it still will need to be paired with LC questions. - Objectivity: good - Scalability: good - Accuracy: bad. Candidates can memorize and recite these answers to the questions. Also too many good candidates actually do not memorize the details about programming language or framework. Also it can easily obtained from just Googling so people can cheat easily. - Meritocratic: good 4. Pure behavioral - Objectivity: bad - Scalability: good - Accuracy: bad - Meritocratic: bad 5. Ex-coworker - Objectivity: bad - Scalability: bad - Accuracy: good. If you've worked with the candidate before, you'll have a pretty good grasp of whether or not he/she can do the job well. - Meritocratic: bad
- higeorge13 4y ago
  You miss an important criterion though which is “does my interview has anything to do with the actual work?”, and lc fails miserably. And this one criterion probably classifies lc even lower than take home test or coding questions.
- Izkata 4y ago
  > 1. Take home assignments. > Meritocratic: good I'd go with "iffy": because of the "take home" part there's no guarantee it's the candidate that did the work. This can probably be determined with a follow-up conversation during the in-person part, but without such an addition I don't see this format as "good" on this metric.
frostmatthew 4y ago
> the idea of asking people to write code during an interview what sort of revolutionary And I think it's a great idea, personally I would never consider working anywhere that hired devs without seeing them write some code. But my problem is with the types of questions asked at many companies, specifically the types that require weeks (or months!) of prepping. During my last job search one interview that stood out (positively) was a problem around parsing some HTTP headers (it started simple and then had layers of complexity added as I solved each one). It was honestly one of the best questions I've seen in an interview as it requires the candidate to be able to write code and solve a problem but without requiring/expecting the candidate to have prepped beforehand to learn (or brush up on) theoretical stuff far removed from the types of problems we actually solve on a daily bases (evidence of this disconnect is demonstrated by the fact people need to prep).
- SpicyLemonZest 4y ago
  I think this is a great example of the kind of implicit expectations that always pervade these discussions. I had a similar interview in my last job search, and for me it stood out negatively. At the time I’d never parsed any part of an HTTP response in my professional career, so I made some mistakes that look kinda silly if you know what you’re doing, and it was clear that the interviewer didn’t believe me when I explained that this isn’t a problem that comes up often in my field. I don’t want it to sound like I’m complaining, it’s not a big deal, but the interviews a web developer sees as neat and low-prep are exactly the ones I need to prepare extra for.
dijit 4y ago
Was it really the 00-s? I had two jobs at the turn of the decade and they were conversations about topics such as “where would I go to find authoritative information on x” (google is not the answer) and approaches to troubleshooting. Granted, I’m a script monkey (sysadmin) but some level of automation has always been part of the job. I’ve had precisely 1 leet-code style interview (2hours for 16 problems) and I couldn’t tolerate the pressure. The interview I had at google was much more humane than what people are attempting to cargo cult.

jsnk 4y ago

This is probably the best take in the entire comment section. I see whole bunch of people complaining about Leetcode (LC) style coding questions, but I don't really see any alternative that is as good as LC. LC interviews solve all the following criteria better than other interview formats in aggregate. - Objectivity: can you evaluate candidates as objectively as possible as opposed to subjectivity? - Scalability: can you evaluate many candidates fast with high throughput? - Accuracy: can you filter out bad candidates? Yes, the filter is often restrictive to the point where good candidates also get filtered out unfortunately. - Meritocratic: can you make sure the process is assessed based on the merit of solving LC questions well as opposed to having connection to someone you know at the company or having gone to some prestigious school or other companies? Leetcode format does good in all criteria above, or at least better than other formats below in aggregate. 1. Take home assignments. - Objectivity: bad - Scalability: bad. Once the question leaks, it's harder to come up with another question easily. Also doesn't scale for candidate side either. As a rule of thumb, I always decline interviews that require take home because it's useless to try hard as this is not applicable to interview at other companies. ROI is extremely low for candidates. - Accuracy: good. - Meritocratic: good 2. Coding questions with heavy emphasis on practicality (Stripe & Square) I was pretty impressed with questions from Stripe and Square. They created problems that are heavily related to string, array, hashmap manipulation, searching, replacing etc using regex and such. If you are not into LC questions, you might prefer this, but I actually found some of these questions harder than dynamic programming questions I got from Google. - Objectivity: good - Scalability: bad. Once the question leaks, it's harder to come up with another question easily. - Accuracy: good. - Meritocratic: good 3. Knowledge / Trivia questions about programming language or framework This might make sense a bit if you are specifically looking for iOS, Android or web developer but it still will need to be paired with LC questions. - Objectivity: good - Scalability: good - Accuracy: bad. Candidates can memorize and recite these answers to the questions. Also too many good candidates actually do not memorize the details about programming language or framework. Also it can easily obtained from just Googling so people can cheat easily. - Meritocratic: good 4. Pure behavioral - Objectivity: bad - Scalability: good - Accuracy: bad - Meritocratic: bad 5. Ex-coworker - Objectivity: bad - Scalability: bad - Accuracy: good. If you've worked with the candidate before, you'll have a pretty good grasp of whether or not he/she can do the job well. - Meritocratic: bad

higeorge13 4y ago
You miss an important criterion though which is “does my interview has anything to do with the actual work?”, and lc fails miserably. And this one criterion probably classifies lc even lower than take home test or coding questions.
Izkata 4y ago
> 1. Take home assignments. > Meritocratic: good I'd go with "iffy": because of the "take home" part there's no guarantee it's the candidate that did the work. This can probably be determined with a follow-up conversation during the in-person part, but without such an addition I don't see this format as "good" on this metric.

frostmatthew 4y ago

> the idea of asking people to write code during an interview what sort of revolutionary And I think it's a great idea, personally I would never consider working anywhere that hired devs without seeing them write some code. But my problem is with the types of questions asked at many companies, specifically the types that require weeks (or months!) of prepping. During my last job search one interview that stood out (positively) was a problem around parsing some HTTP headers (it started simple and then had layers of complexity added as I solved each one). It was honestly one of the best questions I've seen in an interview as it requires the candidate to be able to write code and solve a problem but without requiring/expecting the candidate to have prepped beforehand to learn (or brush up on) theoretical stuff far removed from the types of problems we actually solve on a daily bases (evidence of this disconnect is demonstrated by the fact people need to prep).

SpicyLemonZest 4y ago
I think this is a great example of the kind of implicit expectations that always pervade these discussions. I had a similar interview in my last job search, and for me it stood out negatively. At the time I’d never parsed any part of an HTTP response in my professional career, so I made some mistakes that look kinda silly if you know what you’re doing, and it was clear that the interviewer didn’t believe me when I explained that this isn’t a problem that comes up often in my field. I don’t want it to sound like I’m complaining, it’s not a big deal, but the interviews a web developer sees as neat and low-prep are exactly the ones I need to prepare extra for.

dijit 4y ago

Was it really the 00-s? I had two jobs at the turn of the decade and they were conversations about topics such as “where would I go to find authoritative information on x” (google is not the answer) and approaches to troubleshooting. Granted, I’m a script monkey (sysadmin) but some level of automation has always been part of the job. I’ve had precisely 1 leet-code style interview (2hours for 16 problems) and I couldn’t tolerate the pressure. The interview I had at google was much more humane than what people are attempting to cargo cult.