7 ms·

The best programmers do apply. Whatever you think a work-hire test is, that wasn't it. A work-hire test attracts the best programmers. It doesn't repel them.

The best programmers do apply. Whatever you think a work-hire test is, that wasn't it. A work-hire test attracts the best programmers. It doesn't repel them. It's a chance to demonstrate their skill. It's also far better: anything is better than the fake contests they're currently forced to endure. Which would you prefer? Spend a couple hours remotely fixing some bugs and adding a feature on a fake iOS app, or spend all day playing "dodge the bias" for a hit rate of ~60% at the cost of a vacation day? Why do you think the programmer from your article didn't get a job when you "sat back to watch him get a job"? You didn't tune your test to what the company was looking for. Any YC founders who are reading this: Refuse to talk to Triplebyte until they set up a work hire test for your company. Force them to do it, and force them to work with you to tune the test to the work you need. You will get a 100% hit rate. And you'll notice something else: Your retention rate will go way, way up. Want to not deal with firing people? Get them to show you they can do the work. Nothing else matters. The test needs to be set up so that the candidate can demonstrate they can do exactly the same things all other employees do on a day-to-day basis, or it's not a work-hire test.

ammon 11y ago
EDIT: I thought that sillysaurus3 was talking about a work trial period (days or a week), but it seems they are talking about asking question during that interview that are similar to actual work. I retract me objection :) If only it were that easy. I am not at all opposed to work-trial tests. They are almost certainly more accurate. But a high percentage of programmer will never apply to a company that requires one. Read any HN comment thread about the topic. It's very controversial. Lots of programmers are against it. Work-trials require MORE time upfront from the candidate, and of course still often lead to rejections (by design a hiring filter rejects most people). A failed work trial burns an entire week of vacation. An unemployed programmer (who is being paid the work-trial) may like it, but someone working another job may not.
- Swizec 11y ago
  If you're getting paid for a week of work, you can take a week of unpaid vacation from your day job. No vacation days burned. No money lost.
  - thaumasiotes 11y ago
    Huh? You can't just take unpaid vacation because you feel like it.
    - Swizec 11y ago
      Can't you? Your only obligation to the company is the time they pay you for. The time they don't pay you for is yours.
      - thaumasiotes 11y ago
        We're not longshoremen. Virtually nobody has a contract saying "just show up when and if you feel like it, and we'll pay you for that".
        Retra 11y ago
        I don't think Swizec was implying that this time couldn't be coordinated in advance with your employer.
        s73v3r 11y ago
        I think the point is that most employers would say "no."
  - bjt 11y ago
    I suspect that the flexible "unpaid vacation" scenario you're describing is much more rare than you think. If your team is pushing for a major release and you decide just not to show up, your days are numbered. If you spend the week working for another company, you are quite likely to be fired from the old one. Ammon mentioned Weebly, above. They are one company that I've seen require an on-site week as a contractor. I know of a Weebly candidate who lost her current job because the employer considered the leave to be job abandonment. Thankfully, she got the job at Weebly. Perhaps Weebly's weeklong trial is effective for them at weeding out bad hires. But I suspect most good programmers would never consider giving up a week of their lives to an extended job interview.
- FLUX-YOU 11y ago
  Why don't we let the interviewee choose between a few interview methods? Or bypass it if they think their public code proves their ability?
  - ammon 11y ago
    I think this is the best solution. It does raise issues of how you keep a consistent bar between the various options, but I think it's better than the alternatives. Currently we let people do a project track where they do a project on their own time, or an interview track where they answer interview questions. We also give them a choice of interview questions in a number of areas, to try to increase the probability that we see a strength.
- karmacondon 11y ago
  It seems like you're being forced into a false dichotomy of being either completely for work trials or completely against them, and that's unfair. But if your biggest objection is that it will take additional time, that depends entirely on the nature of the trial. I was given a work trial at my first job and it took about 40 minutes. It didn't go as smoothly as I'd hoped, but I would much much rather have done that than spent 40 minutes answering brain teasers in front of a white board. If I'm already taking a day off, I don't care if the interview lasts three hours or five. I'd rather work on a collaborative task with my potential teammates to see if it's going to be worth radically altering my life to take a new job. I understand that there are many programmers who are opposed to work trials, and they can say "I'm opposed to work trials and would rather demonstrate my abilities in another way." It seems like your research so far has been very comprehensive, why dismiss an effective technique because some people are opposed to it?
- sanderjd 11y ago
  It's really frustrating when people are clearly talking around each other, and either failing to notice or willfully continuing to do it. sillysaurus3: "Spend a couple hours remotely fixing some bugs and adding a feature on a fake iOS app" ammon: "A failed work trial burns an entire week of vacation." You're not talking about the same thing! sillysaurus3 is talking about doing a small amount of work that is similar to what the company does, under conditions similar to those you would have as an employee. You seem to be talking about short trial period contract-to-hire. Yes, the thing you're talking about sucks and is a huge turnoff to candidates, but (mostly speaking for myself here) what sillysaurus3 is suggesting is massively appealing.
eropple 11y ago
> A work-hire test attracts the best programmers. It doesn't repel them. I'm not the best programmer, but I'm comfortable I'd slot in the top quintile, probably top ten percent, of people that walk through a startup's door. And I'd never even return your call. No work-sample test I've ever seen would take less than four hours. That's a $550 opportunity cost for me at my standard rates. What the hell makes you think you deserve that for free? You aren't Google and you aren't Facebook. You need me more than I need you. And almost everybody else who's like you--and given the way you are acting, probably you too--will forget about me or blow me off at some point in the process, wasting my time even further. On the other hand, an in-person, discussion-based interview isn't work and isn't priced as work; not only does it tells me about the company and whether or not I actually want to work there, but I enjoy meeting new people in a professional setting (there are multiple companies where I've turned them down, but have become friends with people I've met through the process!). Look at my Github and decide if I can hack if you want, that's why it's there, but I don't work on effing spec.
- sillysaurus 11y ago
  Our culture is rejecting the one effective test we have. Do we care about equality, or not? I tried not mentionining this aspect, hoping people would realize on their own. But a remote work hire test is also mostly anonymous. It doesn't matter whether you're black, white, male, or female. All that matters is whether you can do the work. On the flipside, what you're saying is that you genuinely want to spend a vacation day meeting a new company instead of with your family or working on your own projects. And it's like, if you think you're a good dev, why wouldn't you leap at the opportunity to show it off? I get that it's a little annoying to spend a few hours on it, but the standard interview is literally random noise. Why subject your future to a random process? I don't know. I respect your view. I'm going to bow out now. Have a good week.
  - eropple 11y ago
    I get what you are saying. And equality is tremendously, spectacularly important; I have more than once been That Guy at companies, making management uncomfortable when asking why the whole place is White Dude Central. But I very strongly feel that your value prop isn't an effective one from the perspective of the employee. You are not telling me why I should give a damn about your work-hire test, as somebody you need to hire to make your company work. What are you offering, aside from Yet Another Startup with Yet Another Startup Problems and Yet Another Startup Under-Market Salary? Why should I be your monkey? And, FWIW, when salaried, I not once have taken a vacation day to interview. I've said "hey, I'll be in late," and because this industry is so deranged as to think that 50+-hour weeks are normal, no manager has had the temerity to get mad at me for taking a morning off because invariably it will be cashed in when I have to pull a sixteen-hour day to deal with a problem. (The days around Heartbleed earned me some goodwill at that gig, if you follow!)
thaumasiotes 11y ago
Triplebyte does notionally allow you to apply (to Triplebyte) via a trial project. But it's in addition to the interview(s), and the feeling they get from you during the interview trumps the project. They gave me the following feedback: > This was a tough decision and one that we were on the fence about. We really appreciate you taking the time to work on the take home project. We're aware this requires a substantial time commitment and we are really grateful that you invested the time in completing it. We thought you wrote a great, very full featured [trial project]. It was especially impressive how much you dug into the academics behind [the project]. > However we made the decision because we felt that while going through the project together during the interview, we didn't see the fluency of programming when adding to it that we had hoped for. While we specifically designed the take home project track to help overcome the difficulties of coding under time pressure with someone watching, we do still need to see a certain level of programming during the interview. This didn't seem to be the case here, where making changes to the project seemed to be slower and more difficult than we'd have liked. tptacek had some advice for hiring-through-a-work-test that I (from my armchair) agree with, which is that it's important to give everyone the same work test so your applicants are comparable. There's a little bit of tension (I don't think it's unresolvable) between that and "The test needs to be set up so that the candidate can demonstrate they can do exactly the same things all other employees do on a day-to-day basis, or it's not a work-hire test".
- dominotw 11y ago
  It always surprises to me to no end when I see people get any real feedback from the interview. Do these employers not care about getting sued, like everyone els.
  - thaumasiotes 11y ago
    That feedback may not be hopelessly vague, but it's also totally unactionable. They wouldn't let me reinterview, and interviewing was the only problem they identified.
mentat 11y ago
This is absolutely true. Doing work-hire tells me if it's work I'm interested in doing as well as showing that I can do the work. Interviews are a crap shoot of bias avoidance as sillysauras3 says. Please actually work on fixing it.
cmdkeen 11y ago
"The best programmers" aren't all fixing bugs in iOS apps. The blog post perfectly outlined people who are motivated by other things. I don't want to work at a company where "realistic" work is me working alone on doing assigned tasks. I identify with the "product engineer" concept they outlined, I want to talk UX and user testing in an interview not prove I can understand someone else's code base. Honestly many companies will get far more value of a programmer who has user sympathy and writes user-friendly applications than one who writes a fully tested, bug free and technically sexy app which users don't want to use.
s73v3r 11y ago
"A work-hire test attracts the best programmers. It doesn't repel them." Not really. The best programmers have other stuff to do besides your make-work silliness. "Get them to show you they can do the work. Nothing else matters." This is even more silliness. Suppose you have a diverse team, and the a candidate comes up, and passes the test, but is Donald Trump. Are you then going to say that only the work matters?
RodericDay 11y ago
I don't know if I'm a "best programmer", but I definitely enjoy "add a little feature to our fake app" tests. A friend of mine's company usually sends out a little canvas-based browser paint app with a bunch of features, and asks them to implement a few things, like flood fill. It's really enjoyable.
- TranquilMarmot 11y ago
  Favorite interview I've done was something similar to this; they gave me a small, pre-made project with tools they knew I was familiar with, asked me to implement some specific functionality, and then told me to "have fun with it for a few hours". So I did, and the project blew them away. Unfortunately, they then flew me out to their office, whiteboard hazed me, and sent me home. Called me a few weeks later to tell me I didn't get the job, but that they sent me a hoodie in the mail (felt like a consolation prize). It's a pretty comfy hoodie. They spent that VC money well.

ammon 11y ago

EDIT: I thought that sillysaurus3 was talking about a work trial period (days or a week), but it seems they are talking about asking question during that interview that are similar to actual work. I retract me objection :) If only it were that easy. I am not at all opposed to work-trial tests. They are almost certainly more accurate. But a high percentage of programmer will never apply to a company that requires one. Read any HN comment thread about the topic. It's very controversial. Lots of programmers are against it. Work-trials require MORE time upfront from the candidate, and of course still often lead to rejections (by design a hiring filter rejects most people). A failed work trial burns an entire week of vacation. An unemployed programmer (who is being paid the work-trial) may like it, but someone working another job may not.

Swizec 11y ago
If you're getting paid for a week of work, you can take a week of unpaid vacation from your day job. No vacation days burned. No money lost.
- thaumasiotes 11y ago
  Huh? You can't just take unpaid vacation because you feel like it.
  - Swizec 11y ago
    Can't you? Your only obligation to the company is the time they pay you for. The time they don't pay you for is yours.
    - thaumasiotes 11y ago
      We're not longshoremen. Virtually nobody has a contract saying "just show up when and if you feel like it, and we'll pay you for that".
      - Retra 11y ago
        I don't think Swizec was implying that this time couldn't be coordinated in advance with your employer.
        s73v3r 11y ago
        I think the point is that most employers would say "no."
- bjt 11y ago
  I suspect that the flexible "unpaid vacation" scenario you're describing is much more rare than you think. If your team is pushing for a major release and you decide just not to show up, your days are numbered. If you spend the week working for another company, you are quite likely to be fired from the old one. Ammon mentioned Weebly, above. They are one company that I've seen require an on-site week as a contractor. I know of a Weebly candidate who lost her current job because the employer considered the leave to be job abandonment. Thankfully, she got the job at Weebly. Perhaps Weebly's weeklong trial is effective for them at weeding out bad hires. But I suspect most good programmers would never consider giving up a week of their lives to an extended job interview.
FLUX-YOU 11y ago
Why don't we let the interviewee choose between a few interview methods? Or bypass it if they think their public code proves their ability?
- ammon 11y ago
  I think this is the best solution. It does raise issues of how you keep a consistent bar between the various options, but I think it's better than the alternatives. Currently we let people do a project track where they do a project on their own time, or an interview track where they answer interview questions. We also give them a choice of interview questions in a number of areas, to try to increase the probability that we see a strength.
karmacondon 11y ago
It seems like you're being forced into a false dichotomy of being either completely for work trials or completely against them, and that's unfair. But if your biggest objection is that it will take additional time, that depends entirely on the nature of the trial. I was given a work trial at my first job and it took about 40 minutes. It didn't go as smoothly as I'd hoped, but I would much much rather have done that than spent 40 minutes answering brain teasers in front of a white board. If I'm already taking a day off, I don't care if the interview lasts three hours or five. I'd rather work on a collaborative task with my potential teammates to see if it's going to be worth radically altering my life to take a new job. I understand that there are many programmers who are opposed to work trials, and they can say "I'm opposed to work trials and would rather demonstrate my abilities in another way." It seems like your research so far has been very comprehensive, why dismiss an effective technique because some people are opposed to it?
sanderjd 11y ago
It's really frustrating when people are clearly talking around each other, and either failing to notice or willfully continuing to do it. sillysaurus3: "Spend a couple hours remotely fixing some bugs and adding a feature on a fake iOS app" ammon: "A failed work trial burns an entire week of vacation." You're not talking about the same thing! sillysaurus3 is talking about doing a small amount of work that is similar to what the company does, under conditions similar to those you would have as an employee. You seem to be talking about short trial period contract-to-hire. Yes, the thing you're talking about sucks and is a huge turnoff to candidates, but (mostly speaking for myself here) what sillysaurus3 is suggesting is massively appealing.

eropple 11y ago

> A work-hire test attracts the best programmers. It doesn't repel them. I'm not the best programmer, but I'm comfortable I'd slot in the top quintile, probably top ten percent, of people that walk through a startup's door. And I'd never even return your call. No work-sample test I've ever seen would take less than four hours. That's a $550 opportunity cost for me at my standard rates. What the hell makes you think you deserve that for free? You aren't Google and you aren't Facebook. You need me more than I need you. And almost everybody else who's like you--and given the way you are acting, probably you too--will forget about me or blow me off at some point in the process, wasting my time even further. On the other hand, an in-person, discussion-based interview isn't work and isn't priced as work; not only does it tells me about the company and whether or not I actually want to work there, but I enjoy meeting new people in a professional setting (there are multiple companies where I've turned them down, but have become friends with people I've met through the process!). Look at my Github and decide if I can hack if you want, that's why it's there, but I don't work on effing spec.

sillysaurus 11y ago
Our culture is rejecting the one effective test we have. Do we care about equality, or not? I tried not mentionining this aspect, hoping people would realize on their own. But a remote work hire test is also mostly anonymous. It doesn't matter whether you're black, white, male, or female. All that matters is whether you can do the work. On the flipside, what you're saying is that you genuinely want to spend a vacation day meeting a new company instead of with your family or working on your own projects. And it's like, if you think you're a good dev, why wouldn't you leap at the opportunity to show it off? I get that it's a little annoying to spend a few hours on it, but the standard interview is literally random noise. Why subject your future to a random process? I don't know. I respect your view. I'm going to bow out now. Have a good week.
- eropple 11y ago
  I get what you are saying. And equality is tremendously, spectacularly important; I have more than once been That Guy at companies, making management uncomfortable when asking why the whole place is White Dude Central. But I very strongly feel that your value prop isn't an effective one from the perspective of the employee. You are not telling me why I should give a damn about your work-hire test, as somebody you need to hire to make your company work. What are you offering, aside from Yet Another Startup with Yet Another Startup Problems and Yet Another Startup Under-Market Salary? Why should I be your monkey? And, FWIW, when salaried, I not once have taken a vacation day to interview. I've said "hey, I'll be in late," and because this industry is so deranged as to think that 50+-hour weeks are normal, no manager has had the temerity to get mad at me for taking a morning off because invariably it will be cashed in when I have to pull a sixteen-hour day to deal with a problem. (The days around Heartbleed earned me some goodwill at that gig, if you follow!)

thaumasiotes 11y ago

Triplebyte does notionally allow you to apply (to Triplebyte) via a trial project. But it's in addition to the interview(s), and the feeling they get from you during the interview trumps the project. They gave me the following feedback: > This was a tough decision and one that we were on the fence about. We really appreciate you taking the time to work on the take home project. We're aware this requires a substantial time commitment and we are really grateful that you invested the time in completing it. We thought you wrote a great, very full featured [trial project]. It was especially impressive how much you dug into the academics behind [the project]. > However we made the decision because we felt that while going through the project together during the interview, we didn't see the fluency of programming when adding to it that we had hoped for. While we specifically designed the take home project track to help overcome the difficulties of coding under time pressure with someone watching, we do still need to see a certain level of programming during the interview. This didn't seem to be the case here, where making changes to the project seemed to be slower and more difficult than we'd have liked. tptacek had some advice for hiring-through-a-work-test that I (from my armchair) agree with, which is that it's important to give everyone the same work test so your applicants are comparable. There's a little bit of tension (I don't think it's unresolvable) between that and "The test needs to be set up so that the candidate can demonstrate they can do exactly the same things all other employees do on a day-to-day basis, or it's not a work-hire test".

dominotw 11y ago
It always surprises to me to no end when I see people get any real feedback from the interview. Do these employers not care about getting sued, like everyone els.
- thaumasiotes 11y ago
  That feedback may not be hopelessly vague, but it's also totally unactionable. They wouldn't let me reinterview, and interviewing was the only problem they identified.

mentat 11y ago

This is absolutely true. Doing work-hire tells me if it's work I'm interested in doing as well as showing that I can do the work. Interviews are a crap shoot of bias avoidance as sillysauras3 says. Please actually work on fixing it.

cmdkeen 11y ago

"The best programmers" aren't all fixing bugs in iOS apps. The blog post perfectly outlined people who are motivated by other things. I don't want to work at a company where "realistic" work is me working alone on doing assigned tasks. I identify with the "product engineer" concept they outlined, I want to talk UX and user testing in an interview not prove I can understand someone else's code base. Honestly many companies will get far more value of a programmer who has user sympathy and writes user-friendly applications than one who writes a fully tested, bug free and technically sexy app which users don't want to use.

s73v3r 11y ago

"A work-hire test attracts the best programmers. It doesn't repel them." Not really. The best programmers have other stuff to do besides your make-work silliness. "Get them to show you they can do the work. Nothing else matters." This is even more silliness. Suppose you have a diverse team, and the a candidate comes up, and passes the test, but is Donald Trump. Are you then going to say that only the work matters?

RodericDay 11y ago

I don't know if I'm a "best programmer", but I definitely enjoy "add a little feature to our fake app" tests. A friend of mine's company usually sends out a little canvas-based browser paint app with a bunch of features, and asks them to implement a few things, like flood fill. It's really enjoyable.

TranquilMarmot 11y ago
Favorite interview I've done was something similar to this; they gave me a small, pre-made project with tools they knew I was familiar with, asked me to implement some specific functionality, and then told me to "have fun with it for a few hours". So I did, and the project blew them away. Unfortunately, they then flew me out to their office, whiteboard hazed me, and sent me home. Called me a few weeks later to tell me I didn't get the job, but that they sent me a hoodie in the mail (felt like a consolation prize). It's a pretty comfy hoodie. They spent that VC money well.