14 ms·

> Unless your company can afford a month-long interview process for every candidate which the author suggests, this is the best we have. If the current leetcod

by tharne 5y ago

> Unless your company can afford a month-long interview process for every candidate which the author suggests, this is the best we have. If the current leetcode style of interviewing is the best that SV can come up with, despite having some of the best engineers and thinkers in the world and billions of dollars to spend, that's really sad.

amscanne 5y ago
Why is this sad? Standardized IQ tests are notoriously difficult to get right and may be legally questionable (Griggs v. Duke Power Co.). At some point you have to accept that there is no ideal way to interview, and that there are fundamental time/precision/recall trade offs. Big tech optimizes for low time and high precision, which means you certainly will end up with low recall (false negatives). I’m sure there are improvements to me made (and maybe the specific leetcode style is not ideal), but I don’t think the billions of dollars is relevant. It’s like saying it’s sad that tech companies haven’t improving on O(n log(n)) sorting despite their billions — it’s not possible.
- dragonwriter 5y ago
  > Standardized IQ tests are notoriously difficult to get right and may be legally questionable (Griggs v. Duke Power Co.). Griggs applies to any hiring criteria or practice that has a negative impact on a protected class compared to people outside the class (given the set of protected classes, this is essentially any hiring criteria or practice) without sufficient evidence of probative value on job performance. It is neither limited to things very much like IQ tests, nor are IQ tests any harder to justify than any other element of a hiring process.
  - amscanne 5y ago
    The point there is standardized. If you apply a standardized test, then it becomes feasible to measure and argue the disparate impact. For all their failings, leetcode-style interviews are probably less culturally-sensitive than standardized IQ tests. They probably do have (unintentional) disparate impact, but this seems like a really hard thing to measure (and possibly correct for).
    - dragonwriter 5y ago
      > If you apply a standardized test, then it becomes feasible to measure and argue the disparate impact. Yes, the existence of readily available statistics may make the unequal impact easier to show. > For all their failings, leetcode-style interviews are probably less culturally-sensitive than standardized IQ tests. “Culturally sensitive”, maybe, but they almost certainly have quite large unequal impacts adverse to protected classes (including “age, if over 40”), though the absence of external data raises the cost of proving the unequal impact. Also, certainly less demonstrably predictive of job performance in software development. > They probably do have (unintentional) disparate impact Probably, that, as well, but I don't think the age discrimination function largely is unintentional to start with.
      - amscanne 5y ago
        You are committing a corollary to the green lumber fallacy described by the article. Just because code challenges are not relevant to job duties doesn’t mean the results are irrelevant to job performance. They are a proxy intelligence test. General intelligence is the best predictor of success in almost any role (not just software engineering). This is what confounds people. They think the interviews are designed that way because they are supposed to be representative of the job — I don’t believe that’s the case. They are that way because they provide a strongly correlated signal of performance after hire, and big tech has decades of data for all different interview types. I’m very confident that if they had a better interview circuit that could be done in a ~day, they would be doing that. Obviously even at big tech, referrals and recommendations count for a lot. While I don’t like the low recall, I do think that “invert this binary tree” probably has less bias than a quiz on technology, or a design conversation (that seems way more susceptible). Perhaps it has a bias for a particular kind of computer science education and thinking, but at least that’s not a protected class. I’m not seeing the age connection, but I could imagine e.g. a computer science education in different countries emphasizing different skills over years (and leading to some candidates with a leg up on the “tests”).
        eropple 5y ago
        > They think the interviews are designed that way because they are supposed to be representative of the job — I don’t believe that’s the case. This may be the case. But even if it does act as an effective proxy--and I'm not sure it does; my worst yeses in interview loops have been very adroit programmers who I passed against my better judgment when my "not sure I want to be around this person every day" bells were ringing subtly in the back of my mind--then it has a different problem. You've now set expectations with the interviewee that oh yeah, we do hard stuff here. Then they go frob knobs or write frontend stuff all day. (This actually happened to me at my first job. I didn't know any better, of course. The hiring manager pumped up my tires with all the difficult scaling work, etcetera etcetera. Then I was writing HTML into templates for the first six months I was there because they needed a body to plug into the role.)
recursive 5y ago
Your sadness is not actionable. It will remain status quo until someone thinks of something better, regardless of ambient sadness levels.
paxys 5y ago
Silicon valley seems to be doing just fine with this system in place. If there is a better process, let's formalize and test it. Here's a hint - large companies have spent millions/billions of dollars over the years doing just that, and this is what the results show.
- aeronaute 5y ago
  I bet they have spent a lot of money on it. How about those folks at Google that, when they were unknowingly reviewing their own packets from when they originally interviewed, decided that they wouldn't even hire themselves? Google also found that interview performance isn't a good predictor of on-the-job performance. Sorry, but it's hard to take this style of interview seriously after hearing about these things.
  - mehrdada 5y ago
    > Google also found that interview performance isn't a good predictor of on-the-job performance. People repeat things like this without really understanding statistics and priors and what that assertion really means. Assuming that statement is true, it applies only to Googlers who have been hired by the hiring process, i.e. if you are beyond the cutoff threshold of the interview process, the ranking within that subset is not determined by the interview performance. That in no way implies that among all the interviewees, including the rejects, the job performance would not have been correlated with their interview performance, had they been hired, hypothetically.

by tharne 5y ago

amscanne 5y ago

Why is this sad? Standardized IQ tests are notoriously difficult to get right and may be legally questionable (Griggs v. Duke Power Co.). At some point you have to accept that there is no ideal way to interview, and that there are fundamental time/precision/recall trade offs. Big tech optimizes for low time and high precision, which means you certainly will end up with low recall (false negatives). I’m sure there are improvements to me made (and maybe the specific leetcode style is not ideal), but I don’t think the billions of dollars is relevant. It’s like saying it’s sad that tech companies haven’t improving on O(n log(n)) sorting despite their billions — it’s not possible.

dragonwriter 5y ago
> Standardized IQ tests are notoriously difficult to get right and may be legally questionable (Griggs v. Duke Power Co.). Griggs applies to any hiring criteria or practice that has a negative impact on a protected class compared to people outside the class (given the set of protected classes, this is essentially any hiring criteria or practice) without sufficient evidence of probative value on job performance. It is neither limited to things very much like IQ tests, nor are IQ tests any harder to justify than any other element of a hiring process.
- amscanne 5y ago
  The point there is standardized. If you apply a standardized test, then it becomes feasible to measure and argue the disparate impact. For all their failings, leetcode-style interviews are probably less culturally-sensitive than standardized IQ tests. They probably do have (unintentional) disparate impact, but this seems like a really hard thing to measure (and possibly correct for).
  - dragonwriter 5y ago
    > If you apply a standardized test, then it becomes feasible to measure and argue the disparate impact. Yes, the existence of readily available statistics may make the unequal impact easier to show. > For all their failings, leetcode-style interviews are probably less culturally-sensitive than standardized IQ tests. “Culturally sensitive”, maybe, but they almost certainly have quite large unequal impacts adverse to protected classes (including “age, if over 40”), though the absence of external data raises the cost of proving the unequal impact. Also, certainly less demonstrably predictive of job performance in software development. > They probably do have (unintentional) disparate impact Probably, that, as well, but I don't think the age discrimination function largely is unintentional to start with.
    - amscanne 5y ago
      You are committing a corollary to the green lumber fallacy described by the article. Just because code challenges are not relevant to job duties doesn’t mean the results are irrelevant to job performance. They are a proxy intelligence test. General intelligence is the best predictor of success in almost any role (not just software engineering). This is what confounds people. They think the interviews are designed that way because they are supposed to be representative of the job — I don’t believe that’s the case. They are that way because they provide a strongly correlated signal of performance after hire, and big tech has decades of data for all different interview types. I’m very confident that if they had a better interview circuit that could be done in a ~day, they would be doing that. Obviously even at big tech, referrals and recommendations count for a lot. While I don’t like the low recall, I do think that “invert this binary tree” probably has less bias than a quiz on technology, or a design conversation (that seems way more susceptible). Perhaps it has a bias for a particular kind of computer science education and thinking, but at least that’s not a protected class. I’m not seeing the age connection, but I could imagine e.g. a computer science education in different countries emphasizing different skills over years (and leading to some candidates with a leg up on the “tests”).
      - eropple 5y ago
        > They think the interviews are designed that way because they are supposed to be representative of the job — I don’t believe that’s the case. This may be the case. But even if it does act as an effective proxy--and I'm not sure it does; my worst yeses in interview loops have been very adroit programmers who I passed against my better judgment when my "not sure I want to be around this person every day" bells were ringing subtly in the back of my mind--then it has a different problem. You've now set expectations with the interviewee that oh yeah, we do hard stuff here. Then they go frob knobs or write frontend stuff all day. (This actually happened to me at my first job. I didn't know any better, of course. The hiring manager pumped up my tires with all the difficult scaling work, etcetera etcetera. Then I was writing HTML into templates for the first six months I was there because they needed a body to plug into the role.)

recursive 5y ago

Your sadness is not actionable. It will remain status quo until someone thinks of something better, regardless of ambient sadness levels.

paxys 5y ago

Silicon valley seems to be doing just fine with this system in place. If there is a better process, let's formalize and test it. Here's a hint - large companies have spent millions/billions of dollars over the years doing just that, and this is what the results show.

aeronaute 5y ago
I bet they have spent a lot of money on it. How about those folks at Google that, when they were unknowingly reviewing their own packets from when they originally interviewed, decided that they wouldn't even hire themselves? Google also found that interview performance isn't a good predictor of on-the-job performance. Sorry, but it's hard to take this style of interview seriously after hearing about these things.
- mehrdada 5y ago
  > Google also found that interview performance isn't a good predictor of on-the-job performance. People repeat things like this without really understanding statistics and priors and what that assertion really means. Assuming that statement is true, it applies only to Googlers who have been hired by the hiring process, i.e. if you are beyond the cutoff threshold of the interview process, the ranking within that subset is not determined by the interview performance. That in no way implies that among all the interviewees, including the rejects, the job performance would not have been correlated with their interview performance, had they been hired, hypothetically.