5 ms·

I agree that code challenges aren't ideal. But we've been struggling with parsing resumes and titles to identify the competency of developers. Our current codin

I agree that code challenges aren't ideal. But we've been struggling with parsing resumes and titles to identify the competency of developers. Our current coding challenge isn't super hard and based on the type of code you'd see in our product. It asks you to read a simple javascript function, explain it, and then implement new functionality that can built by refactoring the existing method. It's about 20 lines of code and 50 lines of documentation. You can use any IDE that you like. We've had many candidates that look strong on paper (3+ years of experience, senior titles, etc) come in and struggle. Struggles ranged from basic code literacy (think for loop to traverse a nested json object) to basic programming like writing a function. We've especially noticed this more with coders coming from purely a javascript and front-end background (React/Angular, etc). They understand control structures like if/else but struggle with loops, setting variables, and basic higher level concepts like breaking a problem into smaller functions. My philosophy is for junior candidates, any level of competency is ok. Strong mentoring and experience on job usually can allow anyone to thrive and so you can prioritize soft skills and culture. But for more advanced roles, I don't know, it's hard to parse out the quality candidates without having some sort of technical interview. Definitely would love to hear how other's learning and experiences.

dhairya 6y ago
Another we'd love to hear opinions on. Based on a recent study where candidates are more likely to succeed if given private space and time to work on the problem, we'll disclose the coding challenge about 15 minutes prior to the interview allow the candidate to read it before joining the call. We struggle with the right amount of time provided for private review is. We want to be fair to all candidates and not implicitly favor candidates who have more free time to dedicate to the interview process than others. but at same time provide some sort of reflection space outside of a cold start to the challenge.
geebee 6y ago
I've heard a variant of this repeatedly. A company justifies technical exam style interviews on the grounds that many people who look good on paper "can't code". Meaning, can't program fizz buzz. Can't write a method or function. Can't write a loop, or an if/else conditional. But I can do all these things. I've gotten bounced from my interviews because (this was the official reason, anyway) I didn't get far enough with the coding in finding all matching subtrees in a binary tree, in 45 minutes, at the whiteboard. Companies should feel free to hire based on my ability to do this, they should not say they were just testing basic things like lists, loops, and conditionals, or even basic data structures and algorithms like binary trees or hash maps. These exams go way beyond that. This may not be the case for all, and it sounds like it is not the case for your process. But in the broader industry practice, I smell a motte-and-bailey fallacy here.
- dhairya 6y ago
  In our case our its based on our actual interview experience with candidates. We've been actively evolving our process and calibrating expectation based on the candidates that have come in. We've actually simplified the interview question considerably (as I mentioned the solution doesn't even require you write new logic, you can refactor existing logic that's provided). But we still see candidates struggle with the basics. To be fair this could also be a reflection on our ability to read resumes and experiences and our initial phone screen. But it is hard. My first job was at fortune 50 company, you could have the title of senior software engineer and not write a single line of code as part of your job. That being said i don't won't diminish your experiences. This is just our experience from the other side of the table. Also we don't have an HR department (we're a small startup). The interviews are conducted by engineers in the company and we try our best to empathize with candidates and be conscious of creating an interview process that we'd feel comfortable being a part of as well.
  - wendyshu 6y ago
    I don't see how this addresses geebee's point -- what makes you so sure struggling on programming puzzles implies incompetence on the job?
    - dhairya 6y ago
      we're not asking programming puzzles. i described our process above. struggling to define a function and basic code comprehension (e.g. tracing a for loop that traverses a json object in candidate's preferred language) is a strong signal that candidate is not a good fit a mid to senior level for us. If it was a junior level position, sure you can learn on the job.
      - wendyshu 6y ago
        How do you know that?
      - dhairya 6y ago
        in our case, the problem is a highly simplified version of functionality in our code base and is close to being representative of work they'd be doing if hired. i actually don't disagree with the prior poster. Technical interviews are highly variable and it's frustrating to go through poor interview processes. I've got bad interview stories as well. Being on the other side of table now, we're always interested in learning about better approaches hiring. Would love hear what has worked well for others.
itronitron 6y ago
For advanced roles I'd suggest separating out the evaluation of coding ability from the evaluation of a particular language. Let the candidate pick their language when you want to test how well they can write new code, and then do a separate evaluation of how deep they are on the language and technical stack you are using.
- dhairya 6y ago
  I agree with the sentiment. Honestly we don't check for syntax or even require the code to compile. We let the candidate write puesdo code which usally isn't super far python and javascript implementations of our challenge. We try to pay attention to the types of questions the candidates asks about context of the code and choices they make in designing a solution. That being said i'd be curious how you evaluate general software engineering ability (e.g. writing clean abstractions, decomposing monolithic methods, and general design). I feel most system design evaluations are a bit too high level or overly broad (e.g. design a library application or make a social network) and coding challenge are too low level.
  - scollet 6y ago
    I like to think of the higher level as creative problem solving. Are they (me in this case) going down rabbit holes? Are they painting a clear picture? Are they communicating that picture? Are they suggesting edge cases? Are they minimizing bloat? Is it too complicated? What are the most important aspects of the design in their eyes? Can they argue for them? Do they understand where a piece fits and if not, do they understand that something has to go there? Can you both step back and definitely outline a logical flow? There's not always one solution, but the approach counts. Lower level is logical problem solving. Can they reason the problem space? Do they start naively? Can they iterate? Can they optimize? Can they explain each of these steps for future iteration? Do you understand what they're writing? Would you work on that code yourself? There's not always one solution here either, but 90% is getting a naive solution that can be optimized by taking a second look. I feel that the naive should be found in 5-15 minutes and you can spend the rest of the time talking about how to optimize. I know I'm rehashing what you probably already know, but at a certain point interviewers are just playing games. If you've found someone you want to work with who can also reason about problems in a timeframe you see fit, well... that to me looks like gold. I like these questions because that's how real systems are going to be built. Take-homes are great, but there is just so much effort wasted in the process if the product can't be understood.
lubesGordi 6y ago
If you're interviewing someone for an 'advanced role', you can talk to them and find out if they know their shit. If you can't then you shouldn't be interviewing them. Senior devs can see where other devs are at pretty easily.

dhairya 6y ago

Another we'd love to hear opinions on. Based on a recent study where candidates are more likely to succeed if given private space and time to work on the problem, we'll disclose the coding challenge about 15 minutes prior to the interview allow the candidate to read it before joining the call. We struggle with the right amount of time provided for private review is. We want to be fair to all candidates and not implicitly favor candidates who have more free time to dedicate to the interview process than others. but at same time provide some sort of reflection space outside of a cold start to the challenge.

geebee 6y ago

I've heard a variant of this repeatedly. A company justifies technical exam style interviews on the grounds that many people who look good on paper "can't code". Meaning, can't program fizz buzz. Can't write a method or function. Can't write a loop, or an if/else conditional. But I can do all these things. I've gotten bounced from my interviews because (this was the official reason, anyway) I didn't get far enough with the coding in finding all matching subtrees in a binary tree, in 45 minutes, at the whiteboard. Companies should feel free to hire based on my ability to do this, they should not say they were just testing basic things like lists, loops, and conditionals, or even basic data structures and algorithms like binary trees or hash maps. These exams go way beyond that. This may not be the case for all, and it sounds like it is not the case for your process. But in the broader industry practice, I smell a motte-and-bailey fallacy here.

dhairya 6y ago
In our case our its based on our actual interview experience with candidates. We've been actively evolving our process and calibrating expectation based on the candidates that have come in. We've actually simplified the interview question considerably (as I mentioned the solution doesn't even require you write new logic, you can refactor existing logic that's provided). But we still see candidates struggle with the basics. To be fair this could also be a reflection on our ability to read resumes and experiences and our initial phone screen. But it is hard. My first job was at fortune 50 company, you could have the title of senior software engineer and not write a single line of code as part of your job. That being said i don't won't diminish your experiences. This is just our experience from the other side of the table. Also we don't have an HR department (we're a small startup). The interviews are conducted by engineers in the company and we try our best to empathize with candidates and be conscious of creating an interview process that we'd feel comfortable being a part of as well.
- wendyshu 6y ago
  I don't see how this addresses geebee's point -- what makes you so sure struggling on programming puzzles implies incompetence on the job?
  - dhairya 6y ago
    we're not asking programming puzzles. i described our process above. struggling to define a function and basic code comprehension (e.g. tracing a for loop that traverses a json object in candidate's preferred language) is a strong signal that candidate is not a good fit a mid to senior level for us. If it was a junior level position, sure you can learn on the job.
    - wendyshu 6y ago
      How do you know that?
    - dhairya 6y ago
      in our case, the problem is a highly simplified version of functionality in our code base and is close to being representative of work they'd be doing if hired. i actually don't disagree with the prior poster. Technical interviews are highly variable and it's frustrating to go through poor interview processes. I've got bad interview stories as well. Being on the other side of table now, we're always interested in learning about better approaches hiring. Would love hear what has worked well for others.

itronitron 6y ago

For advanced roles I'd suggest separating out the evaluation of coding ability from the evaluation of a particular language. Let the candidate pick their language when you want to test how well they can write new code, and then do a separate evaluation of how deep they are on the language and technical stack you are using.

dhairya 6y ago
I agree with the sentiment. Honestly we don't check for syntax or even require the code to compile. We let the candidate write puesdo code which usally isn't super far python and javascript implementations of our challenge. We try to pay attention to the types of questions the candidates asks about context of the code and choices they make in designing a solution. That being said i'd be curious how you evaluate general software engineering ability (e.g. writing clean abstractions, decomposing monolithic methods, and general design). I feel most system design evaluations are a bit too high level or overly broad (e.g. design a library application or make a social network) and coding challenge are too low level.
- scollet 6y ago
  I like to think of the higher level as creative problem solving. Are they (me in this case) going down rabbit holes? Are they painting a clear picture? Are they communicating that picture? Are they suggesting edge cases? Are they minimizing bloat? Is it too complicated? What are the most important aspects of the design in their eyes? Can they argue for them? Do they understand where a piece fits and if not, do they understand that something has to go there? Can you both step back and definitely outline a logical flow? There's not always one solution, but the approach counts. Lower level is logical problem solving. Can they reason the problem space? Do they start naively? Can they iterate? Can they optimize? Can they explain each of these steps for future iteration? Do you understand what they're writing? Would you work on that code yourself? There's not always one solution here either, but 90% is getting a naive solution that can be optimized by taking a second look. I feel that the naive should be found in 5-15 minutes and you can spend the rest of the time talking about how to optimize. I know I'm rehashing what you probably already know, but at a certain point interviewers are just playing games. If you've found someone you want to work with who can also reason about problems in a timeframe you see fit, well... that to me looks like gold. I like these questions because that's how real systems are going to be built. Take-homes are great, but there is just so much effort wasted in the process if the product can't be understood.

lubesGordi 6y ago

If you're interviewing someone for an 'advanced role', you can talk to them and find out if they know their shit. If you can't then you shouldn't be interviewing them. Senior devs can see where other devs are at pretty easily.