6 ms·

If we give you a real project to work on and you view that as "fear-based", "a hoop", and "nonsense" -- those are huge red flags. You'd rather solve B.S. puzzl

If we give you a real project to work on and you view that as "fear-based", "a hoop", and "nonsense" -- those are huge red flags. You'd rather solve B.S. puzzle questions, or debug basic algorithms on a whiteboard standing in front of an interviewer? I like real work. You should too. Otherwise, you are absolutely right, we shouldn't be working together.

danielweber 12y ago
I thought grandparent's "fear-based" stuff was over the top, but why are you so sure that your current strategy is the proper one, enough to dismiss other interview methods with talk of "B.S. puzzle questions"?
hga 12y ago
Indeed. The proposition is "If you finish this work successfully, we will put it in production and you'll get paid for it." It's hard to get more real in my book. I do agree it's a hoop (what hiring process doesn't have those, even if it's a mutual trusted friend vouching for them?), if you can't get it done then one or more of us has made a mistake, but....
- chris_mahan 12y ago
  Essentially then it's a contract-to-hire position.
kcorbitt 12y ago
That makes sense, but doesn't address the OP's other major point: how do you deal with people who either 1) are contractually prohibited from freelancing for you or 2) are too busy with their day job to be able to give an honest effort? Have you ever had a candidate that you suspected would be a good hire but was in that situation? How did you deal with it?
gknoy 12y ago
I think the GP's point was that if they are looking to get a job, they probably has many options of good places to work (even if perhaps not as good as yours) which would not require this of him, and therefore your audition can be seen as extra work which they do not need. Now, if I were Very Interested in working at Discourse, Fog Creek, or any similar place of work, rather than just at any place with a nice paycheck, I think the audition is a great idea, and something people would be willing to do. The GP's main objection is that you only get the best people who want to work at __your__ company enough to spend time on your auditions. There are likely many engineers like them who have a network of job opportunities compelling enough that the added draw of it being Stack Overflow, Discourse, or Fog Creek is not enough to merit the extra work.
- webjprgm 12y ago
  It might also work if the audition was applicable to many potential employers, like how talent scouts in sports and the arts go to events where the potential talent performs rather than having the person audition for each potential employer. (I'm not an expert on arts or sports so I might be off on my analogy.) I hear a lot about GitHub, blogs, and open source contributions which essentially act as your audition open to all talent scouts. Perhaps if the audition challenge could be posted to the applicant's GitHub account then it would be a bit more useful to the applicant. The challenge could be skipped if a project in the GitHub account already demonstrates proficiency in the desired application domains.
  - johnjlocke 12y ago
    The sports analogy is partially correct, but college prospects in all the major sports have to go to the talent recruiters before they are signed. (NBA tryouts, NFL Combine, etc.)
msoad 12y ago
Chris, I love to work for Stack Exchange. The problem is I don't like to work on .NET systems anymore. I don't want to lose my Unix command line tools! :)
aetherson 12y ago
The articles suggests asking candidates to consult for you for a few days or a week. That's just not very realistic. I'm all in favor of doing real work with a candidate instead of bullshit whiteboard problems, but candidates who already have a job do not have time to do your consulting job alongside their current job (and their job search!). Candidates who do not currently have a job perhaps have more time, but still probably are not that interested in suspending their job search to consult with you, and also (correctly) judge this to mean that you're skeptical about their abilities, and want to take a job with an employer who is more enthusiastic about them. That may be somewhat irrational of the candidate, but it's also true.
ProblemFactory 12y ago
I think every candidate prefers a real project to BS puzzle questions and whiteboard coding - it gives them the opportunity to show their real skills, and to also evaluate the culture and the team at the company. But only students, freelancers who want to quit freelancing, and the unemployed would be able to take "a few days to a few weeks" for such a project. For someone in full-time employment, a few weeks would be most of their vacation time for the entire year, just for one interview!
candybar 12y ago
It's certainly a hoop if you're already employed and have a multitude of options that I would presume the best and the brightest all do. It can come across as nonsense because if I'm good and know I'm good, I'd expect any potential employers to be able to recognize that easily without a trial period. And it certainly reflects a fear of commitment, warranted or not. I'm not sure why any in-demand developer would subject himself to that kind of trial period - not only is it against the terms of most employement agreements, it represents disproportionate commitment to a single job lead, which may be better spent on interviewing and receiving other offers. You, as an employer, are not special and the best and the brighest recognize that. I don't think it works well as a filter either - a technical interview can easily assess capabilities and while you're right that it cannot accurately assess work ethic or motivation, a trial period cannot either! A desperate candidate during a trial period is generally going to behave very differently than he would a year down the road when he feels bored and the job is secure in his mind. Is it a better filter in a vacuum? If you start with the same sample of candidates, all of whom had to go through whatever process you devise, would a trial period make it easier for you to find the best candidates? Of course. But having such a process significantly skews your candidate pool to the desperate. That's not necessarily a wrong approach to recruiting, but it has nothing to do with hiring the best and the brightest. In addition, unless you're extremely careful, you'll be mostly judging candidates based on their familiarity with your technology stack and workflow, as opposed to long-term potential because that's what short-term productivity depends upon. It may not apply to you in particular, but one other thing that would make me worry about a trial period is that it's indicative of two other personal flaws on the part of the hiring manager - he may be 1) indecisive and 2) technically incompetent. The indecisiveness part is obvious - not only is not being able to hire without a trial period a mark of indecisiveness by itself, it's also a hedge against future indecisiveness in getting rid of employees that don't work out. I also find that technically competent people can judge competence in others very well, very quickly. They aren't always right and there are qualities that only manifest over a longer period of time, but those qualities can't be judged in a few days of work. Paul Graham's advice to investors is apt here: http://paulgraham.com/angelinvesting.html http://paulgraham.com/angelinvesting.html "How do you be a good angel investor? The first thing you need is to be decisive. When we talk to founders about good and bad investors, one of the ways we describe the good ones is to say "he writes checks." That doesn't mean the investor says yes to everyone. Far from it. It means he makes up his mind quickly, and follows through. You may be thinking, how hard could that be? You'll see when you try it. It follows from the nature of angel investing that the decisions are hard. You have to guess early, at the stage when the most promising ideas still seem counterintuitive, because if they were obviously good, VCs would already have funded them." Stringing people along while not being able to make up their mind is what mediocre investors and VCs do - without seeing good evidence to the contrary, I'd guess that this applies to employers as well. Edit: in case my point wasn't clear, I'm not saying the approach is wrong or doesn't work or anything along those lines, merely that it's not how you hire the best and the brightest. Now, it's possible he meant "the best and the brightest" in the cliched way where everyone thinks they hire the best and the brightest, in which case it's fine. But in a literal sense, if your organization needs the best and the brightest, you won't get it following those suggestions. Edit2: the point of pg's advice isn't that being decisive somehow is correlated with risk-taking and thus higher returns. The point is that being indecisive doesn't lead to significantly better decisions and if you can't make decisions quickly, you won't get the very best deals. If you're known to be indecisive or advertise this up front, no one who can get other investors will come to you. This is absolutely the case for talent. Cultural fit is not a good reason either. If you can't judge whether someone fits in during the interview, you're probably not going to figure that out during the trial period during which you have a desperate person desperately trying to fit in. What's worse, this very mindset of "let's hire people who fit in" leads to the worst culture. If culture matters to you, hire people who are different. And yes, finding good engineers, regardless of cultural fit, is by far the hardest problem. If you're optimizing for cultural fit and you're not Google/Facebook-type talent magnet, you're not even in the competition for the best talent. This may be fine if your bar is low - and let's face it, almost everyone's is - but again we're no longer talking about the best and the brightest.
OutThisLife 12y ago
I like you.
kemiller 12y ago
It's not a matter of liking real work or not. Of course everyone prefers real work to silly logic puzzles, but that's a false dichotomy. It's about trust and commitment. You're asking your applicants to assume virtually all of the risk in the transaction. They have to either quit their job or violate the terms of employment. They have to go all-in on your company for weeks as opposed to considering multiple offers. This way of hiring may work for the companies that are already highly desirable (though I think it's still cruel), but it's absolutely lousy advice for unproven companies in a competitive hiring environment. You totally don't want to work with bad people. Not at all. So suck it up, hire the people who look promising, and fire them promptly if they don't work out. But absolutely, positively, do not encourage every little startup to strut around like they are doing top talent some sort of favor by even considering them.
- aetherson 12y ago
  You're absolutely correct, and I want to highlight one thing you said: Nobody likes firing people. Every manager I've ever worked with hates firing their employees -- even really bad ones. What the "audition" is all about is trying to make it so that you don't ever get into a situation where you hire someone, work with them for a month, and say, "Holy shit, this isn't working out at all, I'd better fire this guy." In France or something, where it's super-hard to fire people, maybe that makes sense. In California, it's mostly about trying to spare the feelings of the manager.
  - hga 12y ago
    I've found that bad managers tend to be unable to fire people, and one of the things I always tried to gauge when interviewing for a job was if the company was capable of firing. And, yeah, I've had to fire some people. It really sucks, no matter how deserved (well, I'm talking about just not being able to do the job, vs. "for cause" where it's white line stuff, fortunately never had one of those cases).
  - kemiller 12y ago
    I completely get that. I've had to fire people, and it really really sucks. But that's the point: you don't get to externalize that onto your candidates.
InclinedPlane 12y ago
I think you're setting up a false dichotomy here. The best job offers I've had in the past have been from people seeking me out and where the interview process was mostly a formality. I think that's true of a lot of folks out there (I can certainly point to a butt-load of examples I've witnessed). The idea of a random person walking in off the street and being assessed as being the norm for an interview process is, I think, erroneous. As too the idea that merely giving someone an audition project will ensure they are of the highest quality. Hell, I'd say that sometimes it's even possible to code side by side with someone for a year and not truly understand their potential. There's no silver bullet for assessing dev. skill level. The biggest fallacy is believing that whatever method you come up with won't have huge flaws and gaps in one way or another. I think the best to hope for is ending up with a good enough "batting average" to build a talented team.
- sliverstorm 12y ago
  I get a similar impression in my industry. The best-and-brightest automatically have ten different job offers, usually by virtue of the people they have worked side by side with in the past. You know, as in "I worked with this guy for 5 years back at company X, we want him".
kabdib 12y ago
I love real work. I hate doing basic stuff on whiteboards; puzzles and toy problems suck. But I firmly believe that doing what amounts to an internship is filtering out people that you would be ecstatic to have at your company. Most of the really good engineers I've worked with can go anywhere. When they're available it's by their own choice, and they're gone damned quick. (One fellow recently got fed up with things over a couple of months, made a call, and the email he got back said in part, "Apparently you aced your interview. When should we tell HR you can start?". Seriously, how do you compete with that?) The people are not on the street. These people will not suffer the gauntlet you have decided is necessary. They're going to think to themselves "That's cute," and look elsewhere. I won't do a weeks-long project. I know very few people who would. You CAN assess someone well in eight hours; if you can't get a good handle on someone's skills in that amount of time then you seriously need to improve your own interviewing skills. The message you are sending right now is, "We mistrust our interviewing expertise to the point that we want YOU to take a multi-week risk, really a very high risk, just so that we can be sure." Now, there are undoubtedly people who want to work at your company, and they are willing to go to any measure to do so. If you are all happy with each other, then I see no reason why our paths should cross. I guess this is the point where you can publically rationalize your strategy by bucketing me as a jerk, "sour grapes," someone you wouldn't want to work with anyway because of red flags and such. I won't steer you away from this, since I can't offer any proof that I'm not. But I'm darned sure you're missing good people, even if you're convinced I'm not one.
wisty 12y ago
I think you're bickering at cross purposes. An onsite interview can (and probably should) involve sitting you in front of their computer, and getting you to close a ticket. If you give people homework problems, then it's possible they will either palm them off to a more competent friend (or just pay $100 for a freelancer); or consider it an insulting waste of time. Neither will test whether the candidate is reliable. But at least an onsite test will be harder for them to fob off to someone else; and it's less of a waste of time. It might take them a day or more to set up a "hello world" build at home (depending on the build requirements, and how well you've automated things).
collyw 12y ago
I actually don't mind whiteboard stuff, as long as it is done at a higher level (not nit-picking about leaving a semicolon off). It allows you to actually discuss your thought process, which a homework assignment doesn't. And as someone else pointed out, how do you know the homework assignment was actually done by the candidate? Also say candidate A has a slightly cleaner solution than candidate B. How do you know that candidate A didn't spend 4 times as long as candidate B on the slightly less cleaner solution.

danielweber 12y ago

I thought grandparent's "fear-based" stuff was over the top, but why are you so sure that your current strategy is the proper one, enough to dismiss other interview methods with talk of "B.S. puzzle questions"?

hga 12y ago

Indeed. The proposition is "If you finish this work successfully, we will put it in production and you'll get paid for it." It's hard to get more real in my book. I do agree it's a hoop (what hiring process doesn't have those, even if it's a mutual trusted friend vouching for them?), if you can't get it done then one or more of us has made a mistake, but....

chris_mahan 12y ago
Essentially then it's a contract-to-hire position.

kcorbitt 12y ago

That makes sense, but doesn't address the OP's other major point: how do you deal with people who either 1) are contractually prohibited from freelancing for you or 2) are too busy with their day job to be able to give an honest effort? Have you ever had a candidate that you suspected would be a good hire but was in that situation? How did you deal with it?

gknoy 12y ago

I think the GP's point was that if they are looking to get a job, they probably has many options of good places to work (even if perhaps not as good as yours) which would not require this of him, and therefore your audition can be seen as extra work which they do not need. Now, if I were Very Interested in working at Discourse, Fog Creek, or any similar place of work, rather than just at any place with a nice paycheck, I think the audition is a great idea, and something people would be willing to do. The GP's main objection is that you only get the best people who want to work at __your__ company enough to spend time on your auditions. There are likely many engineers like them who have a network of job opportunities compelling enough that the added draw of it being Stack Overflow, Discourse, or Fog Creek is not enough to merit the extra work.

webjprgm 12y ago
It might also work if the audition was applicable to many potential employers, like how talent scouts in sports and the arts go to events where the potential talent performs rather than having the person audition for each potential employer. (I'm not an expert on arts or sports so I might be off on my analogy.) I hear a lot about GitHub, blogs, and open source contributions which essentially act as your audition open to all talent scouts. Perhaps if the audition challenge could be posted to the applicant's GitHub account then it would be a bit more useful to the applicant. The challenge could be skipped if a project in the GitHub account already demonstrates proficiency in the desired application domains.
- johnjlocke 12y ago
  The sports analogy is partially correct, but college prospects in all the major sports have to go to the talent recruiters before they are signed. (NBA tryouts, NFL Combine, etc.)

msoad 12y ago

Chris, I love to work for Stack Exchange. The problem is I don't like to work on .NET systems anymore. I don't want to lose my Unix command line tools! :)

aetherson 12y ago

The articles suggests asking candidates to consult for you for a few days or a week. That's just not very realistic. I'm all in favor of doing real work with a candidate instead of bullshit whiteboard problems, but candidates who already have a job do not have time to do your consulting job alongside their current job (and their job search!). Candidates who do not currently have a job perhaps have more time, but still probably are not that interested in suspending their job search to consult with you, and also (correctly) judge this to mean that you're skeptical about their abilities, and want to take a job with an employer who is more enthusiastic about them. That may be somewhat irrational of the candidate, but it's also true.

ProblemFactory 12y ago

I think every candidate prefers a real project to BS puzzle questions and whiteboard coding - it gives them the opportunity to show their real skills, and to also evaluate the culture and the team at the company. But only students, freelancers who want to quit freelancing, and the unemployed would be able to take "a few days to a few weeks" for such a project. For someone in full-time employment, a few weeks would be most of their vacation time for the entire year, just for one interview!

candybar 12y ago

It's certainly a hoop if you're already employed and have a multitude of options that I would presume the best and the brightest all do. It can come across as nonsense because if I'm good and know I'm good, I'd expect any potential employers to be able to recognize that easily without a trial period. And it certainly reflects a fear of commitment, warranted or not. I'm not sure why any in-demand developer would subject himself to that kind of trial period - not only is it against the terms of most employement agreements, it represents disproportionate commitment to a single job lead, which may be better spent on interviewing and receiving other offers. You, as an employer, are not special and the best and the brighest recognize that. I don't think it works well as a filter either - a technical interview can easily assess capabilities and while you're right that it cannot accurately assess work ethic or motivation, a trial period cannot either! A desperate candidate during a trial period is generally going to behave very differently than he would a year down the road when he feels bored and the job is secure in his mind. Is it a better filter in a vacuum? If you start with the same sample of candidates, all of whom had to go through whatever process you devise, would a trial period make it easier for you to find the best candidates? Of course. But having such a process significantly skews your candidate pool to the desperate. That's not necessarily a wrong approach to recruiting, but it has nothing to do with hiring the best and the brightest. In addition, unless you're extremely careful, you'll be mostly judging candidates based on their familiarity with your technology stack and workflow, as opposed to long-term potential because that's what short-term productivity depends upon. It may not apply to you in particular, but one other thing that would make me worry about a trial period is that it's indicative of two other personal flaws on the part of the hiring manager - he may be 1) indecisive and 2) technically incompetent. The indecisiveness part is obvious - not only is not being able to hire without a trial period a mark of indecisiveness by itself, it's also a hedge against future indecisiveness in getting rid of employees that don't work out. I also find that technically competent people can judge competence in others very well, very quickly. They aren't always right and there are qualities that only manifest over a longer period of time, but those qualities can't be judged in a few days of work. Paul Graham's advice to investors is apt here: http://paulgraham.com/angelinvesting.html http://paulgraham.com/angelinvesting.html "How do you be a good angel investor? The first thing you need is to be decisive. When we talk to founders about good and bad investors, one of the ways we describe the good ones is to say "he writes checks." That doesn't mean the investor says yes to everyone. Far from it. It means he makes up his mind quickly, and follows through. You may be thinking, how hard could that be? You'll see when you try it. It follows from the nature of angel investing that the decisions are hard. You have to guess early, at the stage when the most promising ideas still seem counterintuitive, because if they were obviously good, VCs would already have funded them." Stringing people along while not being able to make up their mind is what mediocre investors and VCs do - without seeing good evidence to the contrary, I'd guess that this applies to employers as well. Edit: in case my point wasn't clear, I'm not saying the approach is wrong or doesn't work or anything along those lines, merely that it's not how you hire the best and the brightest. Now, it's possible he meant "the best and the brightest" in the cliched way where everyone thinks they hire the best and the brightest, in which case it's fine. But in a literal sense, if your organization needs the best and the brightest, you won't get it following those suggestions. Edit2: the point of pg's advice isn't that being decisive somehow is correlated with risk-taking and thus higher returns. The point is that being indecisive doesn't lead to significantly better decisions and if you can't make decisions quickly, you won't get the very best deals. If you're known to be indecisive or advertise this up front, no one who can get other investors will come to you. This is absolutely the case for talent. Cultural fit is not a good reason either. If you can't judge whether someone fits in during the interview, you're probably not going to figure that out during the trial period during which you have a desperate person desperately trying to fit in. What's worse, this very mindset of "let's hire people who fit in" leads to the worst culture. If culture matters to you, hire people who are different. And yes, finding good engineers, regardless of cultural fit, is by far the hardest problem. If you're optimizing for cultural fit and you're not Google/Facebook-type talent magnet, you're not even in the competition for the best talent. This may be fine if your bar is low - and let's face it, almost everyone's is - but again we're no longer talking about the best and the brightest.

OutThisLife 12y ago

I like you.

kemiller 12y ago

It's not a matter of liking real work or not. Of course everyone prefers real work to silly logic puzzles, but that's a false dichotomy. It's about trust and commitment. You're asking your applicants to assume virtually all of the risk in the transaction. They have to either quit their job or violate the terms of employment. They have to go all-in on your company for weeks as opposed to considering multiple offers. This way of hiring may work for the companies that are already highly desirable (though I think it's still cruel), but it's absolutely lousy advice for unproven companies in a competitive hiring environment. You totally don't want to work with bad people. Not at all. So suck it up, hire the people who look promising, and fire them promptly if they don't work out. But absolutely, positively, do not encourage every little startup to strut around like they are doing top talent some sort of favor by even considering them.

aetherson 12y ago
You're absolutely correct, and I want to highlight one thing you said: Nobody likes firing people. Every manager I've ever worked with hates firing their employees -- even really bad ones. What the "audition" is all about is trying to make it so that you don't ever get into a situation where you hire someone, work with them for a month, and say, "Holy shit, this isn't working out at all, I'd better fire this guy." In France or something, where it's super-hard to fire people, maybe that makes sense. In California, it's mostly about trying to spare the feelings of the manager.
- hga 12y ago
  I've found that bad managers tend to be unable to fire people, and one of the things I always tried to gauge when interviewing for a job was if the company was capable of firing. And, yeah, I've had to fire some people. It really sucks, no matter how deserved (well, I'm talking about just not being able to do the job, vs. "for cause" where it's white line stuff, fortunately never had one of those cases).
- kemiller 12y ago
  I completely get that. I've had to fire people, and it really really sucks. But that's the point: you don't get to externalize that onto your candidates.

InclinedPlane 12y ago

I think you're setting up a false dichotomy here. The best job offers I've had in the past have been from people seeking me out and where the interview process was mostly a formality. I think that's true of a lot of folks out there (I can certainly point to a butt-load of examples I've witnessed). The idea of a random person walking in off the street and being assessed as being the norm for an interview process is, I think, erroneous. As too the idea that merely giving someone an audition project will ensure they are of the highest quality. Hell, I'd say that sometimes it's even possible to code side by side with someone for a year and not truly understand their potential. There's no silver bullet for assessing dev. skill level. The biggest fallacy is believing that whatever method you come up with won't have huge flaws and gaps in one way or another. I think the best to hope for is ending up with a good enough "batting average" to build a talented team.

sliverstorm 12y ago
I get a similar impression in my industry. The best-and-brightest automatically have ten different job offers, usually by virtue of the people they have worked side by side with in the past. You know, as in "I worked with this guy for 5 years back at company X, we want him".

kabdib 12y ago

I love real work. I hate doing basic stuff on whiteboards; puzzles and toy problems suck. But I firmly believe that doing what amounts to an internship is filtering out people that you would be ecstatic to have at your company. Most of the really good engineers I've worked with can go anywhere. When they're available it's by their own choice, and they're gone damned quick. (One fellow recently got fed up with things over a couple of months, made a call, and the email he got back said in part, "Apparently you aced your interview. When should we tell HR you can start?". Seriously, how do you compete with that?) The people are not on the street. These people will not suffer the gauntlet you have decided is necessary. They're going to think to themselves "That's cute," and look elsewhere. I won't do a weeks-long project. I know very few people who would. You CAN assess someone well in eight hours; if you can't get a good handle on someone's skills in that amount of time then you seriously need to improve your own interviewing skills. The message you are sending right now is, "We mistrust our interviewing expertise to the point that we want YOU to take a multi-week risk, really a very high risk, just so that we can be sure." Now, there are undoubtedly people who want to work at your company, and they are willing to go to any measure to do so. If you are all happy with each other, then I see no reason why our paths should cross. I guess this is the point where you can publically rationalize your strategy by bucketing me as a jerk, "sour grapes," someone you wouldn't want to work with anyway because of red flags and such. I won't steer you away from this, since I can't offer any proof that I'm not. But I'm darned sure you're missing good people, even if you're convinced I'm not one.

wisty 12y ago

I think you're bickering at cross purposes. An onsite interview can (and probably should) involve sitting you in front of their computer, and getting you to close a ticket. If you give people homework problems, then it's possible they will either palm them off to a more competent friend (or just pay $100 for a freelancer); or consider it an insulting waste of time. Neither will test whether the candidate is reliable. But at least an onsite test will be harder for them to fob off to someone else; and it's less of a waste of time. It might take them a day or more to set up a "hello world" build at home (depending on the build requirements, and how well you've automated things).

collyw 12y ago

I actually don't mind whiteboard stuff, as long as it is done at a higher level (not nit-picking about leaving a semicolon off). It allows you to actually discuss your thought process, which a homework assignment doesn't. And as someone else pointed out, how do you know the homework assignment was actually done by the candidate? Also say candidate A has a slightly cleaner solution than candidate B. How do you know that candidate A didn't spend 4 times as long as candidate B on the slightly less cleaner solution.