4 ms·

If you design a hiring process that values a coding challenge then of course people who do well at the coding challenge will do well in the process. The proble

If you design a hiring process that values a coding challenge then of course people who do well at the coding challenge will do well in the process. The problem with coding challenges is that they do not require the same skills that software engineering requires. The value of a hired software engineer is measured over many years, you can’t possibly measure the success of a prospect based on whether or not they get hired. You could hire a dozen people who grind leetcode all day to be one team, and hire 2 people who wouldn’t pass a coding challenge screener to be the other team, and the latter team could very plausibly out perform the former team over 12 months. My experience is that a company that has designed a hiring process that does not require a coding challenge has a much higher quality team because they’re not relying on something as arbitrary as a coding challenge. Instead, they’re assessing candidates on what is actually relevant to the company. They’re popular because they’re an easy way to cut down numbers, which makes them feel effective.

josephg 4y ago
> Instead, they’re assessing candidates on what is actually relevant to the company. Great question - what is relevant to the company? This is the #1 purpose of technical screening: To assess whether the candidate can do the job you're trying to hire them for. If the job involves programming, one of the things you need to assess is whether the person can program. Telling me your job history, verbally solving hypothetical architecture problems, or pointing to a github repository with some code in it does not assess whether you can program. All of these things are good signals, but past experience can be misleading, and github activity is trivially easy to fake. I've interviewed over 400 people. All of them passed an automated screening process before they talked to me. About half of the people I talked to failed to solve a simple 1st year programming problem, using their own computer and their favorite language in the half an hour we allotted to the task. Beginners I understand, but its shocking the number of people who have somehow worked in the industry for 20+ years yet only seem to be able to paw ineffectually at eclipse when you ask them to write fizzbuzz. Any interview process that doesn't screen these people out is useless. Its harsh but, if you are one of these people I love you but I don't want to hire you. Lots of people seem to hate programming challenges. But I've yet to hear a viable alternative. Whats yours?
- phphphphp 4y ago
  > All of these things are good signals, but past experience can be misleading, and github activity is trivially easy to fake. And coding challenges... have integrity? If someone goes to the effort to fake GitHub activity (whatever that means?) then why would they not also go to the effort to cheat in a coding challenge? You can learn more from past experience, job history, hypothetical contextual problems and GitHub repositories than you can from a 45 minute fizzbuzz exercise. If you cannot assess a candidate by having a conversation with them then how on earth do you expect to be able to work with them? If you need a fizzbuzz exercise to trust that they actually know what they're talking about (which, again, proves absolutely nothing other than their ability to do fizzbuzz) how can you trust them in a collaborative setting? The success of a screener cannot be measured by how many people in screens out, otherwise, the perfect screener for Software Engineering would be the ability to jump 20ft in the air from sitting down. > its shocking the number of people who have somehow worked in the industry for 20+ years yet only seem to be able to paw ineffectually at eclipse when you ask them to write fizzbuzz. So either there's an epidemic of people who can wax lyrical and provide meaningful insight into software engineering in a professional context but haven't actually worked in the industry... or you're churning through a dry checklist exercise of common interview questions that anyone who does the bare minimum preparation could answer. The point I make (to technical and non-technical people alike) when I'm involved in hiring is that if you cannot qualify a software engineer in a conversation then you're asking the wrong questions. Most people interviewing software engineers have no idea how to effectively assess someone, and in my experience end up reading off some list of "software engineer interview questions". The reason supposed experts can pass these interviews and then fail at actually programming is that the interviews are terrible, and they're just being asked questions they've heard a dozen times before because someone half-assed the process and found them via a "software engineer interview questions" blog post. If you need to know that someone can write code and you cannot confidently assess them in conversation, that's fine, not everyone has that ability, but the solution is to have them tackle a small contextual problem as a project (and pay them for the day of work) and not give them some arbitrary challenge that does not represent the real world. Coding challenge fans make the mistake of believing that there needs to be some step where you have an applicant do a little dance to prove that they can write code, and so a coding challenge is a natural and necessary part of the interview process and that anybody objecting to coding challenges has to provide an alternative that will have an applicant do a little dance to prove that they can write code. I've attended lots of interviews in my career, on both sides of the table, and I know exactly why coding challenges are used: because the rest of the process is so bad that someone who couldn't write code could easily get through. If you need coding challenges to prevent that, so be it, but it's because your interviews are bad. The fact that there's an entire cottage industry of leetcode training and people who spend months "grinding leetcode" should be evidence enough that coding challenges test a candidates ability to... do coding challenges. And, for the record, when I interview software engineers, I send them the questions I am going to ask in advance so they have time to research and prepare because that's what the real world is like... and I remain confident that even then I can still assess them effectively.

josephg 4y ago

> Instead, they’re assessing candidates on what is actually relevant to the company. Great question - what is relevant to the company? This is the #1 purpose of technical screening: To assess whether the candidate can do the job you're trying to hire them for. If the job involves programming, one of the things you need to assess is whether the person can program. Telling me your job history, verbally solving hypothetical architecture problems, or pointing to a github repository with some code in it does not assess whether you can program. All of these things are good signals, but past experience can be misleading, and github activity is trivially easy to fake. I've interviewed over 400 people. All of them passed an automated screening process before they talked to me. About half of the people I talked to failed to solve a simple 1st year programming problem, using their own computer and their favorite language in the half an hour we allotted to the task. Beginners I understand, but its shocking the number of people who have somehow worked in the industry for 20+ years yet only seem to be able to paw ineffectually at eclipse when you ask them to write fizzbuzz. Any interview process that doesn't screen these people out is useless. Its harsh but, if you are one of these people I love you but I don't want to hire you. Lots of people seem to hate programming challenges. But I've yet to hear a viable alternative. Whats yours?

phphphphp 4y ago
> All of these things are good signals, but past experience can be misleading, and github activity is trivially easy to fake. And coding challenges... have integrity? If someone goes to the effort to fake GitHub activity (whatever that means?) then why would they not also go to the effort to cheat in a coding challenge? You can learn more from past experience, job history, hypothetical contextual problems and GitHub repositories than you can from a 45 minute fizzbuzz exercise. If you cannot assess a candidate by having a conversation with them then how on earth do you expect to be able to work with them? If you need a fizzbuzz exercise to trust that they actually know what they're talking about (which, again, proves absolutely nothing other than their ability to do fizzbuzz) how can you trust them in a collaborative setting? The success of a screener cannot be measured by how many people in screens out, otherwise, the perfect screener for Software Engineering would be the ability to jump 20ft in the air from sitting down. > its shocking the number of people who have somehow worked in the industry for 20+ years yet only seem to be able to paw ineffectually at eclipse when you ask them to write fizzbuzz. So either there's an epidemic of people who can wax lyrical and provide meaningful insight into software engineering in a professional context but haven't actually worked in the industry... or you're churning through a dry checklist exercise of common interview questions that anyone who does the bare minimum preparation could answer. The point I make (to technical and non-technical people alike) when I'm involved in hiring is that if you cannot qualify a software engineer in a conversation then you're asking the wrong questions. Most people interviewing software engineers have no idea how to effectively assess someone, and in my experience end up reading off some list of "software engineer interview questions". The reason supposed experts can pass these interviews and then fail at actually programming is that the interviews are terrible, and they're just being asked questions they've heard a dozen times before because someone half-assed the process and found them via a "software engineer interview questions" blog post. If you need to know that someone can write code and you cannot confidently assess them in conversation, that's fine, not everyone has that ability, but the solution is to have them tackle a small contextual problem as a project (and pay them for the day of work) and not give them some arbitrary challenge that does not represent the real world. Coding challenge fans make the mistake of believing that there needs to be some step where you have an applicant do a little dance to prove that they can write code, and so a coding challenge is a natural and necessary part of the interview process and that anybody objecting to coding challenges has to provide an alternative that will have an applicant do a little dance to prove that they can write code. I've attended lots of interviews in my career, on both sides of the table, and I know exactly why coding challenges are used: because the rest of the process is so bad that someone who couldn't write code could easily get through. If you need coding challenges to prevent that, so be it, but it's because your interviews are bad. The fact that there's an entire cottage industry of leetcode training and people who spend months "grinding leetcode" should be evidence enough that coding challenges test a candidates ability to... do coding challenges. And, for the record, when I interview software engineers, I send them the questions I am going to ask in advance so they have time to research and prepare because that's what the real world is like... and I remain confident that even then I can still assess them effectively.