9 ms·

A perspective from the other side of the desk (playing devil's advocate here): There's a fundamental skill that a good programmer has to have, and that is to b

A perspective from the other side of the desk (playing devil's advocate here): There's a fundamental skill that a good programmer has to have, and that is to be able to take a novel problem that they haven't seen before and break it down to solve it in a sensible way. There are plenty of programmers who fake their way through a career without having that skill. They just copy stuff and never really understand it. They're fine for some types of programming career, but if your business involves solving new and novel problems then you have to know which type of programmer you're hiring. A contrived live coding exercise gives you a strong signal on this. It does have a decent chance of producing a false negative, but only a very small chance of a false positive, and that's the trade that has to be made with this kind of approach. Is a better option to not do this kind of assessment, hire the person, find out that they can't do the hard bits and then fire them within 6 months? I'll take some convincing of that...

yieldcrv 3y ago
> but if your business involves solving new and novel problems then you have to know which type of programmer you're hiring. Its more that businesses don’t know what type of programming they do and are just hazing people for no reason More like one person is just hazing people due to a strongly held unobjective opinion like yours Don’t know the better option, just the limitations of this one
pavlov 3y ago
> “take a novel problem that they haven't seen before and break it down” But who does that professionally as a stand-up performance, clock ticking, a judge breathing down your neck who has been equipped with a script that tells him things like “if candidate doesn’t do X within the first ten minutes it’s very bad”? Doing well in that situation depends more on social performance skills than problem-solving skills. You’re essentially trying to infer the interview script that the interviewer has in mind and act it out convincingly. If someone says “I’d like to start by taking ten minutes alone in another room and think it out first”, will they get hired? Probably not. It’s not the expected performance even if the solution is fine.
- reidjs 3y ago
  That’s why he says this approach is more likely to produce false negatives than false positives
  - gymbeaux 3y ago
    False negatives are always better than false positives. Every company who does leetcode interviews agrees with that.
- hnfong 3y ago
  As for the time limit, it's relative. All things equal, if you can complete the problem in 10 minutes and the company can hire somebody that can complete it in 9, then objectively the other candidate is the stronger one. As for nervousness for being forced to partaking in a stand-up performance, I'd argue that "social performance skills" can work against you, since the more "antisocial" you are, the more you can ignore (or are oblivious to) what other people think about you and you can focus on your task. It's only people who actually have the minimum requisite "social skills" that would be conscious of other people intensely watching them "perform". But even if you're right -- interviewing is inherently a "social performance" activity. If you're not coding you're doing some other stand up performance to present yourself anyway.
  - robin_reala 3y ago
    They’re only objectively the strong candidate if that’s the sole thing the company values. It’s unusual for that to be the case.
    - hnfong 3y ago
      With the "all things equal" qualifier, it's sufficient that completing the task faster is a positive signal rather than a negative one. I mean, this is very elementary.
  - Retric 3y ago
    Normal interviews are mostly a scripted performance where people largely regurgitate answers to predictable questions. So the issue isn’t that it’s a stand up performance it’s that you need to split your attention between the performance and solving some trivial problem. Becoming really good at programming requires being able to focus on an actually difficult problem not work through a simplified example while entertaining an audience. There might be a correlation between solving trivial problems quickly and being able to figure out a heisenbug from some multithreaded monstrosity, but it’s not strong enough for minor differences in performance to be meaningful.
darth_avocado 3y ago
> that is to be able to take a novel problem that they haven't seen before But with platforms like leetcode and more, the candidates are split into those who have had the time to prepare (and see the problem before) and candidates who can’t put in that time. And most modern interview processes do not give you credit for thinking through it. You only get through if you solve it the right way, and fast. There is no time to “think and solve a novel problem you’ve never seen before”.
- JohnFen 3y ago
  > And most modern interview processes do not give you credit for thinking through it. You only get through if you solve it the right way, and fast. Interview processes that do it that way are fundamentally broken. I wouldn't want to work at such a company. Interviews are two-way streets, and that would be a case of the company failing the interview. The real benefit of this sort of interview practice is not learning whether or not the candidate can arrive at the correct answer. If we're at the point where this level of interview is even happening at all, you have (or should have) reasonable confidence that the candidate is capable of solving the problem. Whether or not that they do so in the interview is irrelevant. The real benefit is in being able to see how the candidate thinks through the problem. While I think these whiteboard exercises are not the best approach, when I've been the interviewer at companies that required it, I chose a very difficult problem, gave a time limit, and told the candidate that I do not expect them to actually arrive at the solution within the given time. I just want to see how they approach the problem.
codeflo 3y ago
I’m also most often on the hiring side of this. If novel problem solving is what you’re actually testing, I would agree. But too many interviewers just think they’re testing that, when what they’re actually testing is whether a candidate has seen a certain memorizable trick before. I’m talking about the “detect a cycle in a linked list” kind of question. If you ever actually need to do that in practice (though I would question the choices that lead up to that), it’s easy enough to google. The hard part in practice is figuring out that your messy practical problem decomposes into an algorithmic question, but that skill is rarely tested in interviews.
- kps 3y ago
  Detecting a cycle in a directed graph is useful (in certain domains I've worked in), but doesn't have a cute trick answer. [Edit: I in mind ‘Detecting the cycles’ rather than ‘Detecting a cycle’ but wrote the wrong thing. Mea culpa.] Your actual job will be more like implementing a maximally performant directed graph in safe Rust. (Not really. It won't be that interesting.)
  - addaon 3y ago
    > Detecting a cycle in a directed graph is useful (in certain domains I've worked in), but doesn't have a cute trick answer. Doesn't the standard cute trick answer for a linked list (tortoise and hare) work fine? Do two parallel breadth-first traversals of the graph starting at the root node, one at twice the speed of the other. (If there's multiple root nodes, introduce a unique super-root that points to the roots.) If there's at least one cyclic path, both iterations will enter the same cyclic path because iteration order is the same, at which point it reduces to the linked list problem.
    - Freire_Herval 3y ago
      if you can mutate the graph and mark a node as traversed that's the easiest way. If you can't then save the addresses to a hash set. Both are extra memory but you don't have to deal with "parallel BFS," which honestly is just over complicated imo. With "parallel bfs" comparing "layers" of traversal has runtime cost that effects the big oh so I think the above solution is better overall even though it feels cheap.
eesmith 3y ago
> be able to take a novel problem that they haven't seen before and break it down to solve it in a sensible way. ... A contrived live coding exercise gives you a strong signal on this The author suggests take-home coding exercises with "maybe ... a live call to discuss their solution" as giving a stronger signal than a live coding exercise. The author also suggested to "present them with a complex technical problem that requires architecture-level decisions across various parts of a technology stack, and have them talk through how they would approach it" as a better signal than a live coding interviews. So even if what you say is true, you aren't presenting a devil's advocate position. Instead, it makes it seem like you think the alternative to live coding is no assessment, which the author very clearly argues is not true.
adamc 3y ago
I used to do this in interviews -- give people problems not because I cared whether they got "the" answer, but because it gives you a chance to see their brain work. The problem with this is that many, many interviewers do this badly. They think they are looking for someone who gets the right answer. Or they are not good at assessing how other people break down and analyze problems. It is artificial. All interviews are artificial. This is why everyone would prefer to hire candidates they already know a lot about. But a good interviewer isn't looking for a perfect performance.
Freire_Herval 3y ago
[flagged]
fasterik 3y ago
I think there's an analogy here to GRE scores and other standardized tests. With GRE scores you can predict someone's GPA, but you can't necessarily predict their success at doing original research. That's because success as a researcher depends on personality traits that are hard to measure like creativity, independence, self-discipline, perseverance, and passion about a subject. Programming puzzles are the equivalent of the GRE for programming jobs. It might tell you something about the candidate, but I'm skeptical that it measures the most important traits and skills that really good programmers have. It's an empirical question, but your skill at solving programming puzzles might not predict your ability to systematically break down large problems, write readable, robust, well-documented code, or design and debug complex systems. I also have my own theory about this which might just be my own personal bias. I feel like standardized tests encourage conformity in thinking. By relying on standardized tests, you are selecting for individuals who are primarily motivated by external measures of success and approval. On the flip side, you are selecting against individuals who are intrinsically motivated to learn and build things, but who may not care about those external factors. My theory is that organizations with too much of the former and too little of the latter are going to have issues with groupthink and will be less likely to be innovative.
- hypefi 3y ago
  "I feel like standardized tests encourage conformity in thinking. By relying on standardized tests, you are selecting for individuals who are primarily motivated by external measures of success and approval. " Totally agree
throwaway675309 3y ago
I think we're seeing the consequences of not having a properly regulated field that you might see another more strongly regulated engineering disciplines such as electrical and mechanical. Since there is no CS equivalent of a professional engineering exam, employers have no guarantees that applicants will meet minimal qualifications, and so we continue to see an arms race of increasingly complicated whiteboard / leetcode interview processes.
gymbeaux 3y ago
Is there any other means of evaluating for this skill?

yieldcrv 3y ago

> but if your business involves solving new and novel problems then you have to know which type of programmer you're hiring. Its more that businesses don’t know what type of programming they do and are just hazing people for no reason More like one person is just hazing people due to a strongly held unobjective opinion like yours Don’t know the better option, just the limitations of this one

pavlov 3y ago

> “take a novel problem that they haven't seen before and break it down” But who does that professionally as a stand-up performance, clock ticking, a judge breathing down your neck who has been equipped with a script that tells him things like “if candidate doesn’t do X within the first ten minutes it’s very bad”? Doing well in that situation depends more on social performance skills than problem-solving skills. You’re essentially trying to infer the interview script that the interviewer has in mind and act it out convincingly. If someone says “I’d like to start by taking ten minutes alone in another room and think it out first”, will they get hired? Probably not. It’s not the expected performance even if the solution is fine.

reidjs 3y ago
That’s why he says this approach is more likely to produce false negatives than false positives
- gymbeaux 3y ago
  False negatives are always better than false positives. Every company who does leetcode interviews agrees with that.
hnfong 3y ago
As for the time limit, it's relative. All things equal, if you can complete the problem in 10 minutes and the company can hire somebody that can complete it in 9, then objectively the other candidate is the stronger one. As for nervousness for being forced to partaking in a stand-up performance, I'd argue that "social performance skills" can work against you, since the more "antisocial" you are, the more you can ignore (or are oblivious to) what other people think about you and you can focus on your task. It's only people who actually have the minimum requisite "social skills" that would be conscious of other people intensely watching them "perform". But even if you're right -- interviewing is inherently a "social performance" activity. If you're not coding you're doing some other stand up performance to present yourself anyway.
- robin_reala 3y ago
  They’re only objectively the strong candidate if that’s the sole thing the company values. It’s unusual for that to be the case.
  - hnfong 3y ago
    With the "all things equal" qualifier, it's sufficient that completing the task faster is a positive signal rather than a negative one. I mean, this is very elementary.
- Retric 3y ago
  Normal interviews are mostly a scripted performance where people largely regurgitate answers to predictable questions. So the issue isn’t that it’s a stand up performance it’s that you need to split your attention between the performance and solving some trivial problem. Becoming really good at programming requires being able to focus on an actually difficult problem not work through a simplified example while entertaining an audience. There might be a correlation between solving trivial problems quickly and being able to figure out a heisenbug from some multithreaded monstrosity, but it’s not strong enough for minor differences in performance to be meaningful.

darth_avocado 3y ago

> that is to be able to take a novel problem that they haven't seen before But with platforms like leetcode and more, the candidates are split into those who have had the time to prepare (and see the problem before) and candidates who can’t put in that time. And most modern interview processes do not give you credit for thinking through it. You only get through if you solve it the right way, and fast. There is no time to “think and solve a novel problem you’ve never seen before”.

JohnFen 3y ago
> And most modern interview processes do not give you credit for thinking through it. You only get through if you solve it the right way, and fast. Interview processes that do it that way are fundamentally broken. I wouldn't want to work at such a company. Interviews are two-way streets, and that would be a case of the company failing the interview. The real benefit of this sort of interview practice is not learning whether or not the candidate can arrive at the correct answer. If we're at the point where this level of interview is even happening at all, you have (or should have) reasonable confidence that the candidate is capable of solving the problem. Whether or not that they do so in the interview is irrelevant. The real benefit is in being able to see how the candidate thinks through the problem. While I think these whiteboard exercises are not the best approach, when I've been the interviewer at companies that required it, I chose a very difficult problem, gave a time limit, and told the candidate that I do not expect them to actually arrive at the solution within the given time. I just want to see how they approach the problem.

codeflo 3y ago

I’m also most often on the hiring side of this. If novel problem solving is what you’re actually testing, I would agree. But too many interviewers just think they’re testing that, when what they’re actually testing is whether a candidate has seen a certain memorizable trick before. I’m talking about the “detect a cycle in a linked list” kind of question. If you ever actually need to do that in practice (though I would question the choices that lead up to that), it’s easy enough to google. The hard part in practice is figuring out that your messy practical problem decomposes into an algorithmic question, but that skill is rarely tested in interviews.

kps 3y ago
Detecting a cycle in a directed graph is useful (in certain domains I've worked in), but doesn't have a cute trick answer. [Edit: I in mind ‘Detecting the cycles’ rather than ‘Detecting a cycle’ but wrote the wrong thing. Mea culpa.] Your actual job will be more like implementing a maximally performant directed graph in safe Rust. (Not really. It won't be that interesting.)
- addaon 3y ago
  > Detecting a cycle in a directed graph is useful (in certain domains I've worked in), but doesn't have a cute trick answer. Doesn't the standard cute trick answer for a linked list (tortoise and hare) work fine? Do two parallel breadth-first traversals of the graph starting at the root node, one at twice the speed of the other. (If there's multiple root nodes, introduce a unique super-root that points to the roots.) If there's at least one cyclic path, both iterations will enter the same cyclic path because iteration order is the same, at which point it reduces to the linked list problem.
  - Freire_Herval 3y ago
    if you can mutate the graph and mark a node as traversed that's the easiest way. If you can't then save the addresses to a hash set. Both are extra memory but you don't have to deal with "parallel BFS," which honestly is just over complicated imo. With "parallel bfs" comparing "layers" of traversal has runtime cost that effects the big oh so I think the above solution is better overall even though it feels cheap.

eesmith 3y ago

> be able to take a novel problem that they haven't seen before and break it down to solve it in a sensible way. ... A contrived live coding exercise gives you a strong signal on this The author suggests take-home coding exercises with "maybe ... a live call to discuss their solution" as giving a stronger signal than a live coding exercise. The author also suggested to "present them with a complex technical problem that requires architecture-level decisions across various parts of a technology stack, and have them talk through how they would approach it" as a better signal than a live coding interviews. So even if what you say is true, you aren't presenting a devil's advocate position. Instead, it makes it seem like you think the alternative to live coding is no assessment, which the author very clearly argues is not true.

adamc 3y ago

I used to do this in interviews -- give people problems not because I cared whether they got "the" answer, but because it gives you a chance to see their brain work. The problem with this is that many, many interviewers do this badly. They think they are looking for someone who gets the right answer. Or they are not good at assessing how other people break down and analyze problems. It is artificial. All interviews are artificial. This is why everyone would prefer to hire candidates they already know a lot about. But a good interviewer isn't looking for a perfect performance.

Freire_Herval 3y ago

[flagged]

fasterik 3y ago

I think there's an analogy here to GRE scores and other standardized tests. With GRE scores you can predict someone's GPA, but you can't necessarily predict their success at doing original research. That's because success as a researcher depends on personality traits that are hard to measure like creativity, independence, self-discipline, perseverance, and passion about a subject. Programming puzzles are the equivalent of the GRE for programming jobs. It might tell you something about the candidate, but I'm skeptical that it measures the most important traits and skills that really good programmers have. It's an empirical question, but your skill at solving programming puzzles might not predict your ability to systematically break down large problems, write readable, robust, well-documented code, or design and debug complex systems. I also have my own theory about this which might just be my own personal bias. I feel like standardized tests encourage conformity in thinking. By relying on standardized tests, you are selecting for individuals who are primarily motivated by external measures of success and approval. On the flip side, you are selecting against individuals who are intrinsically motivated to learn and build things, but who may not care about those external factors. My theory is that organizations with too much of the former and too little of the latter are going to have issues with groupthink and will be less likely to be innovative.

hypefi 3y ago
"I feel like standardized tests encourage conformity in thinking. By relying on standardized tests, you are selecting for individuals who are primarily motivated by external measures of success and approval. " Totally agree

throwaway675309 3y ago

I think we're seeing the consequences of not having a properly regulated field that you might see another more strongly regulated engineering disciplines such as electrical and mechanical. Since there is no CS equivalent of a professional engineering exam, employers have no guarantees that applicants will meet minimal qualifications, and so we continue to see an arms race of increasingly complicated whiteboard / leetcode interview processes.

gymbeaux 3y ago

Is there any other means of evaluating for this skill?