4 ms·

Yeah, I saw that coming a mile away. Don't you remember uni? Students always breathed a sigh of relief when the final exam was a take home project. Now add thi

Yeah, I saw that coming a mile away. Don't you remember uni? Students always breathed a sigh of relief when the final exam was a take home project. Now add this to the fact that a vast majority of your applicants are going to be feeding your assignment directly into an assistive LLM. As an interviewer, you really can't win. - If you have a take-home test, people will either complain that it's too involved, and time consuming. - If you do whiteboarding, people will claim that it's not representative of the actual job. - If you do paired programming, people will claim it's unfair to those who don't test well under pressure.

skydhash 2y ago
That's because these three methodologies test different things. Take-home is the best representative, but it's heavily skewed by the prize. Whiteboarding only test for algorithms knowledge and a bit of problem-solving. Paired programming relies more on communication. And the last two add time pressure that rarely exists in real world scenario. The fact is, it all depends on the person you need and the team they will be slotted in. But it seems that the ones who know the criteria is rarely involved in these matters.
fishtoaster 2y ago
That's why I offered the choice: different people do well under different circumstances. I figured that it'd allow me to hire a great dev who hates timed challenges or a great dev that hates take-home busywork. In practice, though, everyone like the take-homes, so ¯\_(ツ)_/¯ As for feeding it into an LLM, I go back and forth on that. With their current capabilities, the project is slightly too complex for that to work. You could get a lot of help from an AI, but you still have to put the pieces together yourself. And I'm fine with that! If AI tooling makes you a faster/better dev on a test, I'd expect it to do the same in real work. Longer-term, though, I worry that LLMs still won't be able to "write the whole thing" for real work, but will be able to "write the whole thing" for takehome tests. And as such, I'll have to figure out how to handle that.

skydhash 2y ago

That's because these three methodologies test different things. Take-home is the best representative, but it's heavily skewed by the prize. Whiteboarding only test for algorithms knowledge and a bit of problem-solving. Paired programming relies more on communication. And the last two add time pressure that rarely exists in real world scenario. The fact is, it all depends on the person you need and the team they will be slotted in. But it seems that the ones who know the criteria is rarely involved in these matters.

fishtoaster 2y ago

That's why I offered the choice: different people do well under different circumstances. I figured that it'd allow me to hire a great dev who hates timed challenges or a great dev that hates take-home busywork. In practice, though, everyone like the take-homes, so ¯\_(ツ)_/¯ As for feeding it into an LLM, I go back and forth on that. With their current capabilities, the project is slightly too complex for that to work. You could get a lot of help from an AI, but you still have to put the pieces together yourself. And I'm fine with that! If AI tooling makes you a faster/better dev on a test, I'd expect it to do the same in real work. Longer-term, though, I worry that LLMs still won't be able to "write the whole thing" for real work, but will be able to "write the whole thing" for takehome tests. And as such, I'll have to figure out how to handle that.