5 ms·

Work-sample questions is not free from its own evils. They are expensive to device. At big companies, significant portion of candidates dump out their interview

by sytelus 12y ago

Work-sample questions is not free from its own evils. They are expensive to device. At big companies, significant portion of candidates dump out their interview questions on Internet so that's a big challenge. Second, work-sample type of questions are also expensive to answer for candidates so you can probably just ask one question as part of whole interview and that's about it. So any conclusion is drawn from sample of 1 as opposed to sample of 5. Third, work-sample type of questions requires knowledge of engineering skills that candidate may or may not have developed well at point in time. At many companies, emphasis is how can we solve problem X from algorithms perspective and engineering is just something you learn on the way if you haven't already. Key is ability to figure out computer science part rather than language and tooling part. Overall, I think many companies blindly try to follow the model of Google, Facebook etc and fall on their face. If you expect your developer to write CRUD apps or do mostly plumbing work, there is no point in asking questions like find common ancestor in binary tree. At other companies, developers are expected to solve computational problems and engineering/plumbing/CRUD is small part of the solution pie. There it makes a LOT of sense to ask candidate to solve problems that requires deep computer science. Also asking as many of as possible so that your sample space is large and outcome has more confidence.

tptacek 12y ago
We had three work-sample exercises at Matasano. All told, they took candidates mid-single-digit hours to finish (many geeked out, or golfed on them; we did our best to keep this from happening --- you can read about this on our hiring page: http://www.matasano.com/careers http://www.matasano.com/careers This sounds onerous, but it is less onerous than the normal dev hiring process, which involves an onsite interview that eats the whole day. We did on-site interviews, but they took just 2.5 hours to complete; candidates would be out at lunch. Essentially, we were shifting some of the time candidates would spend sweating in a conference room to their couch instead. There are bad work-sample tests. "Bad" comes in a variety of flavors. I think bad work sample tests will out-predict the median interview. But there's a lot of room to optimize and improve this process, sure.
- elptacek 12y ago
  s/our/their
- skybrian 12y ago
  Hmm, that page says you still ask candidates to write a fuzzer?
- thedufer 12y ago
  My concern here is that it seems to signal a lack of respect for the candidate if the company expects them to complete a task with no matching employee time. Interviews are even; both sides clearly have skin in the game; a work-sample test could be given to any number of people at basically no cost. This is somewhat alleviated by the knowledge that you do a phone screen with each person prior to that stage, but it still leaves me with something of a bad feeling.
  - eru 12y ago
    I think Matasano can make up for some of that with a good reputation. (Plus the 80 bucks they pay for books.)
  - vonmoltke 12y ago
    Interviews aren't even. How many hiring discussions on various parts of the internet have talked about putting tens of hours into studying for the "undergrad CS pop quiz" interview in vogue at so many companies? How many of those same discussions have talked about how hiring is a "distraction" from the "real jobs" of the engineers involved at the company? The process is already badly asymmetric in addition to being artificial. At least the work sample ideal addresses one of those.
  - wavesounds 12y ago
    It totally depends on the work sample they ask you to do. If no thought went into it then you are correct. If, however, a company devises an interesting project to work on it show they also put some energy into this. Most importantly though I like programming, but I don't like speaking up in front of a group of strangers, so 6 hours of programming is much more enjoyable to me then even just 1 hour of interviewing.

by sytelus 12y ago

tptacek 12y ago

We had three work-sample exercises at Matasano. All told, they took candidates mid-single-digit hours to finish (many geeked out, or golfed on them; we did our best to keep this from happening --- you can read about this on our hiring page: http://www.matasano.com/careers http://www.matasano.com/careers This sounds onerous, but it is less onerous than the normal dev hiring process, which involves an onsite interview that eats the whole day. We did on-site interviews, but they took just 2.5 hours to complete; candidates would be out at lunch. Essentially, we were shifting some of the time candidates would spend sweating in a conference room to their couch instead. There are bad work-sample tests. "Bad" comes in a variety of flavors. I think bad work sample tests will out-predict the median interview. But there's a lot of room to optimize and improve this process, sure.

elptacek 12y ago
s/our/their
skybrian 12y ago
Hmm, that page says you still ask candidates to write a fuzzer?
thedufer 12y ago
My concern here is that it seems to signal a lack of respect for the candidate if the company expects them to complete a task with no matching employee time. Interviews are even; both sides clearly have skin in the game; a work-sample test could be given to any number of people at basically no cost. This is somewhat alleviated by the knowledge that you do a phone screen with each person prior to that stage, but it still leaves me with something of a bad feeling.
- eru 12y ago
  I think Matasano can make up for some of that with a good reputation. (Plus the 80 bucks they pay for books.)
- vonmoltke 12y ago
  Interviews aren't even. How many hiring discussions on various parts of the internet have talked about putting tens of hours into studying for the "undergrad CS pop quiz" interview in vogue at so many companies? How many of those same discussions have talked about how hiring is a "distraction" from the "real jobs" of the engineers involved at the company? The process is already badly asymmetric in addition to being artificial. At least the work sample ideal addresses one of those.
- wavesounds 12y ago
  It totally depends on the work sample they ask you to do. If no thought went into it then you are correct. If, however, a company devises an interesting project to work on it show they also put some energy into this. Most importantly though I like programming, but I don't like speaking up in front of a group of strangers, so 6 hours of programming is much more enjoyable to me then even just 1 hour of interviewing.