4 ms·

For a junior-ish software engineer level (say, 1-3 years out of college) who claims to have experience in C, this seems like a fine question, but for a senior s

For a junior-ish software engineer level (say, 1-3 years out of college) who claims to have experience in C, this seems like a fine question, but for a senior software engineer (which I think is what he was interviewing for) it doesn't seem all that great, except maybe as a first-round filter [1]? Not that it's terrible, and I guess if the candidate fails it then it certainly tells you a lot—but if they complete it successfully then it doesn't really seem to tell you much about their software engineering skills or depth of understanding. It's better than some silly challenges companies give, but there's so much room to ask more probing questions or provide more challenging tasks to help you gauge where the upper limits of a candidate's abilities are, instead of just the lower limits. [1] Edit: Looks like it indeed was a filter question, as I suspected: https://news.ycombinator.com/item?id=31065819 https://news.ycombinator.com/item?id=31065819

elromulous 5y ago
This is an issue with many coding problems. I'm not sure if there is a term for this is detection / estimation theory, but it provides what I'm going to call asymmetric information. In this particular case, negatives are true, but all positives must be presumed false. If you make a problem too hard, you're faced with asymmetry in the other direction.
- dataflow 5y ago
  I think the usual solution to that is to follow it up with "what if you wanted to {introduce new/harder requirement}?" and ask them to explain & make as much of the changes as time allows. To give an example, if you were to initially ask someone to implement depth-first traversal recursively, you might follow that up with asking them why you might end up needing to do that iteratively, then you might have them do that iteratively for pre-order depth-first traversal (easier), then follow that up with post-order iterative traversal (harder), then you might ask whether/why it might perform poorly (e.g., re-visited nodes blowing up exponentially), then ask them how they might mitigate it (keeping track of visited nodes), then ask them how they might mitigate the space requirement (lots of possibilities), then ask them how they might handle parallelizing it for some particular problem... I can keep going, but the point is you can keep something like this going until they start to struggle, and that at least tells you the upper limits of their abilities on at least that axis. And you can pace it by e.g. skipping stages as you get a better sense of their skills.
  - owyn 5y ago
    I wish I could downvote this a million times. Do you literally have to do that at work and do you actually talk this kind of wine tasting algorithm bullshit in interviews? This has nothing to do with software engineering as far as building software for a business or any kind of meaningful work. Do you even hire people? This has to be a troll account. I enjoy code golf on a casual slack channel, but you can send me every engineer who fails that question in your interviews and I bet I'd find a few good ones.
    - theamk 5y ago
      That's a strange reaction.. Are you always able to answer any quesion that comes up at work? I know I don't. And so seeing people's reaction to qestion that is too hard for them is a valuable signal. As long as one tells candidate in advance: "I am going to be asking a mix of questions, some easy some hard. I don't expect you to be able to answer every question", this sounds like a good question.
owyn 5y ago
Why do you think that's a bad question for a senior engineer? I think it's great. This is much closer to the actual work that software engineers have to do! That's what I do most of the time when I'm working with code, it's always an existing system that you need to add a feature to without breaking something else. What do you think is a good question for a "senior software engineer"?
- jfoutz 5y ago
  Please convince this director to allow this very safe change to go to prod.
  - bsder 5y ago
    Not ... a ... chance. You go through the normal change process like everybody else. There is no such thing as a "safe" change. The only exception to this is when stuff is on fire, and, practically by definition at that point, there is no such thing as a "safe" change.
    - deleted 5y ago
      [deleted]
    - jfoutz 5y ago
      Let me rephrase. Given a Tech document that addresses the risk of change, the rollback plan, the security implications etc, and a suite of metrics demonstrating load and failure conditions with appropriate alerting. And of course the PR with appropriate test suite, shown to work in a test environment in concert with the rest of the ecosystem, that stuff is just table sakes. convince a director to approve the change. Ideally a director that doesn't report to the same vp as your org.
- dataflow 5y ago
  Because it doesn't tell you the upper limits of their abilities, only the lower limits. You hardly get any insight into their ability to design a new system, or into how they might deal with problems that require more than 2 seconds of straightforward planning, etc. Again, for a senior engineer, it's not really a bad question, just a very mediocre warm-up/filter question. It's just too easy to copy-paste or modify a few functions and get the task over with, and the result ends up giving you very little information other than "this senior engineer can, in fact, complete a very straightforward task". Good questions aren't easy to come up with, I'm not an expert in interviewing by any means, but for a senior engineer you'd want to ask questions that start straightforward like this, but become progressively less straightforward (and maybe one that's a more subtle design issue) so you can get a better feel for their skills as you work with them.
wpietri 5y ago
Sorry, are there people who do an interview like this and then just treat the result as a boolean pass/fail? I mostly do pair-programming interviews. When I do it async and have them write something, we'll at least discuss the code together. So either way, I'm way less interested in whether they finished than how they did it and how they think about it. And maybe I'm just weird, but I'm not super interested in the "upper limits of a candidate's abilities". If they're using all their cleverness just to write something, neither they nor anybody else will be able to maintain it. Software is a team sport, so I want people who are good collaborators. Indeed, the "upper limits" thing makes me think of Aphyr's series of fictional technical interviews. (Start with the last one and work your way forward.) https://aphyr.com/tags/interviews https://aphyr.com/tags/interviews
eyelidlessness 5y ago
That’s, in my opinion, a good thing! It helps verify the engineer can competently code—in a real codebase, which is much more valuable (again IMO) than coding in a vacuum. Eliminating that question, which is often cited as the motivation for these sorts of challenges, is a cue that the candidate is qualified for further interviewing. That being out of the way leaves room to evaluate a lot of other things by: - enquiring about the candidate’s thoughts about the problem and their approach to solving it, on which (again IMO) this candidate would really shine; you’d likely get a much less thoughtful or thorough account from someone less experienced or talented - ask about concerns which would be typical in real world work but often get glossed over in interviews, like how they’d approach testing and SCM and review, or even just things they’d do differently outside the interview process - get more conversational and learn more about the actual value the candidate would potentially bring to your team/org, starting this with some greater confidence, trust and familiarity on both ends - dive deeper on technical validation if anything in the above feels questionable Those sorts of conversations convey much more meaningful information than any coding challenge that isn’t very focused on very specific prerequisites for the domain/role.
- ed25519FUUU 5y ago
  The ergonomics suck. Whose laptop do you use? I’d want my own. Oh they want me to use sublime? Maybe I prefer Vim with all of my macros. I’m a professional and I rely on my macros and tools. Maybe it’s a PC and I only develop on linux. It introduces an entire problem space where you’re evaluating something that’s neither important to you or your company. Way too many moving parts.
  - eyelidlessness 5y ago
    Coding challenges in general, if they’re part of the hiring process, need to be much more accommodating yes. Even if they reassure you they’re evaluating “how you think” etc, this kind of thing is an anxiety amplifier for many (including me). I haven’t interviewed on site with a coding challenge for nearly a decade, but last time I did I literally brought my own laptop and asked permission to use it (which was always declined). If I were interviewing today I’d literally walk out if that request wasn’t at least considered, and probably even still if it was declined without a really good explanation.

elromulous 5y ago

This is an issue with many coding problems. I'm not sure if there is a term for this is detection / estimation theory, but it provides what I'm going to call asymmetric information. In this particular case, negatives are true, but all positives must be presumed false. If you make a problem too hard, you're faced with asymmetry in the other direction.

dataflow 5y ago
I think the usual solution to that is to follow it up with "what if you wanted to {introduce new/harder requirement}?" and ask them to explain & make as much of the changes as time allows. To give an example, if you were to initially ask someone to implement depth-first traversal recursively, you might follow that up with asking them why you might end up needing to do that iteratively, then you might have them do that iteratively for pre-order depth-first traversal (easier), then follow that up with post-order iterative traversal (harder), then you might ask whether/why it might perform poorly (e.g., re-visited nodes blowing up exponentially), then ask them how they might mitigate it (keeping track of visited nodes), then ask them how they might mitigate the space requirement (lots of possibilities), then ask them how they might handle parallelizing it for some particular problem... I can keep going, but the point is you can keep something like this going until they start to struggle, and that at least tells you the upper limits of their abilities on at least that axis. And you can pace it by e.g. skipping stages as you get a better sense of their skills.
- owyn 5y ago
  I wish I could downvote this a million times. Do you literally have to do that at work and do you actually talk this kind of wine tasting algorithm bullshit in interviews? This has nothing to do with software engineering as far as building software for a business or any kind of meaningful work. Do you even hire people? This has to be a troll account. I enjoy code golf on a casual slack channel, but you can send me every engineer who fails that question in your interviews and I bet I'd find a few good ones.
  - theamk 5y ago
    That's a strange reaction.. Are you always able to answer any quesion that comes up at work? I know I don't. And so seeing people's reaction to qestion that is too hard for them is a valuable signal. As long as one tells candidate in advance: "I am going to be asking a mix of questions, some easy some hard. I don't expect you to be able to answer every question", this sounds like a good question.

owyn 5y ago

Why do you think that's a bad question for a senior engineer? I think it's great. This is much closer to the actual work that software engineers have to do! That's what I do most of the time when I'm working with code, it's always an existing system that you need to add a feature to without breaking something else. What do you think is a good question for a "senior software engineer"?

jfoutz 5y ago
Please convince this director to allow this very safe change to go to prod.
- bsder 5y ago
  Not ... a ... chance. You go through the normal change process like everybody else. There is no such thing as a "safe" change. The only exception to this is when stuff is on fire, and, practically by definition at that point, there is no such thing as a "safe" change.
  - deleted 5y ago
    [deleted]
  - jfoutz 5y ago
    Let me rephrase. Given a Tech document that addresses the risk of change, the rollback plan, the security implications etc, and a suite of metrics demonstrating load and failure conditions with appropriate alerting. And of course the PR with appropriate test suite, shown to work in a test environment in concert with the rest of the ecosystem, that stuff is just table sakes. convince a director to approve the change. Ideally a director that doesn't report to the same vp as your org.
dataflow 5y ago
Because it doesn't tell you the upper limits of their abilities, only the lower limits. You hardly get any insight into their ability to design a new system, or into how they might deal with problems that require more than 2 seconds of straightforward planning, etc. Again, for a senior engineer, it's not really a bad question, just a very mediocre warm-up/filter question. It's just too easy to copy-paste or modify a few functions and get the task over with, and the result ends up giving you very little information other than "this senior engineer can, in fact, complete a very straightforward task". Good questions aren't easy to come up with, I'm not an expert in interviewing by any means, but for a senior engineer you'd want to ask questions that start straightforward like this, but become progressively less straightforward (and maybe one that's a more subtle design issue) so you can get a better feel for their skills as you work with them.

wpietri 5y ago

Sorry, are there people who do an interview like this and then just treat the result as a boolean pass/fail? I mostly do pair-programming interviews. When I do it async and have them write something, we'll at least discuss the code together. So either way, I'm way less interested in whether they finished than how they did it and how they think about it. And maybe I'm just weird, but I'm not super interested in the "upper limits of a candidate's abilities". If they're using all their cleverness just to write something, neither they nor anybody else will be able to maintain it. Software is a team sport, so I want people who are good collaborators. Indeed, the "upper limits" thing makes me think of Aphyr's series of fictional technical interviews. (Start with the last one and work your way forward.) https://aphyr.com/tags/interviews https://aphyr.com/tags/interviews

eyelidlessness 5y ago

That’s, in my opinion, a good thing! It helps verify the engineer can competently code—in a real codebase, which is much more valuable (again IMO) than coding in a vacuum. Eliminating that question, which is often cited as the motivation for these sorts of challenges, is a cue that the candidate is qualified for further interviewing. That being out of the way leaves room to evaluate a lot of other things by: - enquiring about the candidate’s thoughts about the problem and their approach to solving it, on which (again IMO) this candidate would really shine; you’d likely get a much less thoughtful or thorough account from someone less experienced or talented - ask about concerns which would be typical in real world work but often get glossed over in interviews, like how they’d approach testing and SCM and review, or even just things they’d do differently outside the interview process - get more conversational and learn more about the actual value the candidate would potentially bring to your team/org, starting this with some greater confidence, trust and familiarity on both ends - dive deeper on technical validation if anything in the above feels questionable Those sorts of conversations convey much more meaningful information than any coding challenge that isn’t very focused on very specific prerequisites for the domain/role.

ed25519FUUU 5y ago
The ergonomics suck. Whose laptop do you use? I’d want my own. Oh they want me to use sublime? Maybe I prefer Vim with all of my macros. I’m a professional and I rely on my macros and tools. Maybe it’s a PC and I only develop on linux. It introduces an entire problem space where you’re evaluating something that’s neither important to you or your company. Way too many moving parts.
- eyelidlessness 5y ago
  Coding challenges in general, if they’re part of the hiring process, need to be much more accommodating yes. Even if they reassure you they’re evaluating “how you think” etc, this kind of thing is an anxiety amplifier for many (including me). I haven’t interviewed on site with a coding challenge for nearly a decade, but last time I did I literally brought my own laptop and asked permission to use it (which was always declined). If I were interviewing today I’d literally walk out if that request wasn’t at least considered, and probably even still if it was declined without a really good explanation.