4 ms·

Yeah, I'm a big fan of your approach. Unfortunately it seems quite difficult to get anyone to do it. Most companies have interview pipelines that are well-estab

Yeah, I'm a big fan of your approach. Unfortunately it seems quite difficult to get anyone to do it. Most companies have interview pipelines that are well-established and immutable, so only new companies are inclined to try it. And even Triplebyte unfortunately fell back on a combination of abstract coding tests and amorphous "cultural fit" criteria. It was a neat hack as a candidate to propose to the interviewing company that they do essentially the same thing as a work-sample test and hear them say yes. That only works if you propose they let you try your hand at doing their actual work. I wonder if it can be generalized?

tptacek 9y ago
I think asking candidates to do "real" work is bad for a bunch of reasons: * It's intrinsically exploitative, even when compensated, because real work done under similar terms is paid far more highly than prorated salary (it also does weird things to salary negotiation, by extracting an up-front concession on rate). * It requires different candidates to work on different problems, which means you're not getting a reliable record of how candidates do on the interview so you can tune it over time. * Because the best candidates will generally refuse to work under these terms, most firms that do paid up-front work have a fast-path for elite candidates, which is an obvious process failure.
- deleted 9y ago
  [deleted]
- mlevental 9y ago
  did you read the grandparent? you use the same bug by checking out a commit before the bug fix was applied.
tribythrowaway 9y ago
Triplebyte's flaw is like that of many other take-home tests: they favor people who lied about how long it took. When I did it, I had to implement a regex parser. I knew intuitively that the state machine approach was best, but if I read up on state machines and coding them, I would bust the 3 hour limit, so I did the best I could to deliver something working within the three hours. Feedback: "Obviously the state machine approach is better, which you didn't use. You're not qualified." Guess I should have just lied and gotten placed like the ones who did.
- tptacek 9y ago
  Time limits on these kinds of things don't make much sense to me.
- thaumasiotes 9y ago
  The state machine approach can't handle perl-compatible backreferences (since true regular expressions can't use them), which apparently some people want. That's feedback you can defend yourself from to some degree. When I implemented a regex parser for Triplebyte, the time limit was much longer than three hours, and I did use state machines. The feedback I got was "we thought you wrote a great, full-featured regex parser, but we need someone who comes off better in interviews".
  - tptacek 9y ago
    You can already see that this is a broken technical qualification process. If you can "defend yourself" from a demerit on the challenge's rubric, you have a subjective process. The smooth talkers will glide past regardless of their abilities, and the gifted developers that don't have interview skills will get washed out by it at random. If there's a point to outsourcing technical qualification, it's exactly to avoid this kind of problem!
morgante 9y ago
> And even Triplebyte unfortunately fell back on a combination of abstract coding tests and amorphous "cultural fit" criteria. I'm not sure when you went through Triplebyte's process, but I did recently and it definitely included a substantial practical programming section which involved writing an actual program and some debugging. They do have the difficulty of not controlling the downstream on-site interview process, so there's a limit to how much they can innovate. Ultimately their goal is to find candidates who can pass company interviews, some of which are still heavy on algorithms.

tptacek 9y ago

I think asking candidates to do "real" work is bad for a bunch of reasons: * It's intrinsically exploitative, even when compensated, because real work done under similar terms is paid far more highly than prorated salary (it also does weird things to salary negotiation, by extracting an up-front concession on rate). * It requires different candidates to work on different problems, which means you're not getting a reliable record of how candidates do on the interview so you can tune it over time. * Because the best candidates will generally refuse to work under these terms, most firms that do paid up-front work have a fast-path for elite candidates, which is an obvious process failure.

deleted 9y ago
[deleted]
mlevental 9y ago
did you read the grandparent? you use the same bug by checking out a commit before the bug fix was applied.

tribythrowaway 9y ago

Triplebyte's flaw is like that of many other take-home tests: they favor people who lied about how long it took. When I did it, I had to implement a regex parser. I knew intuitively that the state machine approach was best, but if I read up on state machines and coding them, I would bust the 3 hour limit, so I did the best I could to deliver something working within the three hours. Feedback: "Obviously the state machine approach is better, which you didn't use. You're not qualified." Guess I should have just lied and gotten placed like the ones who did.

tptacek 9y ago
Time limits on these kinds of things don't make much sense to me.
thaumasiotes 9y ago
The state machine approach can't handle perl-compatible backreferences (since true regular expressions can't use them), which apparently some people want. That's feedback you can defend yourself from to some degree. When I implemented a regex parser for Triplebyte, the time limit was much longer than three hours, and I did use state machines. The feedback I got was "we thought you wrote a great, full-featured regex parser, but we need someone who comes off better in interviews".
- tptacek 9y ago
  You can already see that this is a broken technical qualification process. If you can "defend yourself" from a demerit on the challenge's rubric, you have a subjective process. The smooth talkers will glide past regardless of their abilities, and the gifted developers that don't have interview skills will get washed out by it at random. If there's a point to outsourcing technical qualification, it's exactly to avoid this kind of problem!

morgante 9y ago

> And even Triplebyte unfortunately fell back on a combination of abstract coding tests and amorphous "cultural fit" criteria. I'm not sure when you went through Triplebyte's process, but I did recently and it definitely included a substantial practical programming section which involved writing an actual program and some debugging. They do have the difficulty of not controlling the downstream on-site interview process, so there's a limit to how much they can innovate. Ultimately their goal is to find candidates who can pass company interviews, some of which are still heavy on algorithms.