3 ms·

I agree about Kagi, but about your process wanted to ask: do you really feel like having them write the code as a take-home task adds any signal to the process?

• deleted 1y ago
  [deleted]
• davezatch 1y ago
  Leaving aside the non-trivial question of candidate nervousness in live coding, I do find the "ask questions about take-home" process very illuminating. If it came from a friend or chatGPT, you'll learn that quickly because you can ask them stuff like "show me how you're implementing FOO. OK, if the requirements change, and now we need to BAR, what do you need to change across the app?" If they wrote the code and understood the task, they should understand what needs to change, how to do it, etc, and you can do that part with them live. Maybe someday LLMs will be fast enough that you can't catch someone constantly waiting for the AI to help them, at least as of May 2025 we're not there yet.
  • fenomas 1y ago
    Respectfully, you've only answered the thing I pre-replied to in my final paragraph. My question was why the take-home part adds signal?
• stingraycharles 1y ago
  This is a data engineer’s fizzbuzz and you would be surprised at how many people actually fail it. If they didn’t write it themselves, they will absolutely fail in the follow-up session where the real evaluation happens and I ask them to scale the code, handle crash safety without using transactions, etc.
  • fenomas 1y ago
    Okay, but what signal is added by administering it as a take-home task? (Even moreso if it's a fizzbuzz level task, I'd have thought - actual fizzbuzz being famous as a coding interview task, after all.)
    • stingraycharles 1y ago
      Some people are nervous during live coding challenges, which is why we decided to allow this to be taken home, and the candidate becoming familiar with the problem domain in their own time, rather than "live" under pressure. Would you think doing this live in a screen sharing session adds more value? How would you prepare the candidate for this?
      • fenomas 1y ago
        I see, thanks for clarifying! > Would you think doing this live in a screen sharing session adds more value? In my experience, hugely yes. Above all else I really like that the time commitment is fixed on both sides - no homework for the candidate, and we're not asking them to invest any more time than we are. Also doing things interactively gives a lot of leeway for adjusting and avoiding wasted time. If somebody's got one section basically solved, I might cut in and say "yeah that bit is great, let's call it finished and look at this other bit". If they're obviously having a lot of trouble, I might ask about their current work and maybe switch tasks if there's a better fit. Or if they're obviously coasting easily I might cut in with "how would you scale this?" questions that give more signal than watching them finish up the loose ends. Another thing I like is that we can keep the task specification simple, like what you'd get from a PM, and it's up to the candidate whether they want to ask questions or jump in. I imagine that with take-home tasks you either need to give out pretty specific requirements, or else have people interpreting the task a variety of different ways. > How would you prepare the candidate for this? We explain at the beginning that the goal is write code for an hour and watch how they work, and there's no hard requirement for what needs to be finished by the end. And they're welcome to search the web, use AI, or anything else they'd typically do while working. And I've only done a few dozen of these, but personally I've never seen anybody get particularly nervous. I think nerves are a bigger issue with HR style "pass all the tests within the time limit" tests, where the danger is that somebody can hit a wall and never get past it. That doesn't happen with my format, because if somebody is stuck I just give them hints until they're unstuck (since I get no signal from watching somebody scratch their head).