4 ms·

To me, as someone who hires and gives candidates take home assignments, Kagi’s assignment is huge and disrespectful of the candidate’s time. Surely they’re (pro

To me, as someone who hires and gives candidates take home assignments, Kagi’s assignment is huge and disrespectful of the candidate’s time. Surely they’re (probably unintentionally) filtering for candidates that have a lot of time to waste on projects like this — while people who are busy (the kind of people you want) will pass. Surely there must be other ways to have an idea on a candidate’s skills. In our case, hiring people with data engineering skills, we just ask of them a simple ETL challenge (pull data from zip file, transform it, insert into any database). We leave ambiguities in there and there are some Easter eggs in the dataset (eg null values where you would not expect it, incorrectly formatted CSV) that we use to evaluate how well a candidate can perform. We timebox it at 4 hours, but don’t give guidance in case they run out of time, that’s a good suggestion. During a follow up call, we review the code together and we’ll ask them questions on how to improve the code (“what if the dataset doesn’t fit in memory”, etc), which is what the actual technical assessment is. At that point they should already feel somewhat comfortable with the problem domain, and you can assess their real skills.

Teever 1y ago
The solution to this problem ultimately needs to be a regulatory one. People should be paid for the time they spend in interviews. You make that the law and this ambiguous hoop jumping bullshit goes away real fast. Companies will optimize for cost instead of dumping cost onto the prospective employees. This is good for the economy because it forces companies to innovate and optimize the interview process and it saves hundreds hours of totally economically unproductive time on the part of candidates.
- jv22222 1y ago
  Yes. It would also force them to screen resumes and do better profile research or they wouldn’t spend the money.
- quantadev 1y ago
  That's basically my feeling too. No one would ask for a maid to clean their house free for 4 hours, in order to decide if they want to hire her long term. And that's precisely what hiring managers have been getting away with for decades, because no one is pushing back. The managers are essentially abusing their power. It's all about abuse of power and utter arrogance.
quantadev 1y ago
But if you're unable to determine if someone has this knowledge via a phone call, instead of 4 hours of their work without you even being present, then that failing is on you, not them. If you can't judge someone's knowledge by asking questions, then you don't know how to come up with the right questions. Again that's on you. The only thing a 4-hr take home test will filter for is desperation. You'll get the most desperate candidates, and you'll do so by wasting days and days of people's time once you add it all up. It's just utterly disrespectful to demand these silly homework problems. I always take it that way. I take it personally, tbh. I find it absolutely insulting to even be asked and I simply refuse.
- seancorfield 1y ago
  I'm surprised this seemed to be voted down. I've been a hiring manager for over 30 years, and I never do "technical tests" -- no take-home, no live coding, none of it. I have a map of topics and questions, and I get the candidate chatting about their past projects, their approach, and what they liked/disliked about past projects and various technology they've used. It takes a maximum of one hour and usually close to about thirty minutes to make a yes/no decision on a candidate (sometimes it only takes ten minutes to make a no decision, and then it's a matter of trying to politely end the interview). I've interviewed hundreds of candidates this way over the years, and everyone I've hired has been capable of doing the job. Not once have I ever had to let someone go for lack of technical ability. Part of the problem is that we don't train people how to conduct interviews, and another part is "this is how I was interviewed, so this is how I'm going to interview other candidates" -- pure inertia. As an industry, we really need to do better. As for the OP, _if_ I had been administering a vague take-home project that had a 1-week delivery deadline, and a candidate peppered me with Qs and then presented a full proposal for the project for approval, prior to working on it... I would have rejected them. But I'm pretty certain I would have decided to reject them in my regular 30-60 "chat" interview, and I would not have moved on to the take-home project and wasted their time like that. So, again, I fault the interviewer(s) for not being able to filter candidates efficiently.
  - quantadev 1y ago
    I had an interview once to work for IPFS, with a guy who already knew my skill level well (decades of experience), because I was active on their web forum solving everyone's problems, for about a year. Even during the interview he mentioned the technical part of the interview wasn't even necessary, because he had total confidence in me, and didn't even ask technical questions at all. He spent the ENTIRE hour selling me on the job. I hardly had time to speak. Everything went perfectly fine. Then he asked me to do the take-home project at the very end, just because "everybody has to", and I politely told him sorry I don't do those. So that ended the opportunity. So you're exactly right it's absolutely an "inertia" thing.
fenomas 1y ago
I agree about Kagi, but about your process wanted to ask: do you really feel like having them write the code as a take-home task adds any signal to the process? At my current gig we do a 1-hour pair programming kind of thing, where we video-chat and watch the candidate work on a small, straightforward task. And as the interviewer I watch how they use the tools, where they go for docs, do they read the requirements, how do they test, etc. By the end I always feel like I have a strong picture of their ability, and the whole thing is capped at 90 minutes for both sides (adding time for their questions). If the candidate's code was written offline beforehand, I'd have no idea whether it was theirs, whether it came from a friend or chatGPT, whether it took ten minutes or ten hours, etc. Sure I could try to suss out those things during discussion, but isn't it better to observe directly?
- deleted 1y ago
  [deleted]
- davezatch 1y ago
  Leaving aside the non-trivial question of candidate nervousness in live coding, I do find the "ask questions about take-home" process very illuminating. If it came from a friend or chatGPT, you'll learn that quickly because you can ask them stuff like "show me how you're implementing FOO. OK, if the requirements change, and now we need to BAR, what do you need to change across the app?" If they wrote the code and understood the task, they should understand what needs to change, how to do it, etc, and you can do that part with them live. Maybe someday LLMs will be fast enough that you can't catch someone constantly waiting for the AI to help them, at least as of May 2025 we're not there yet.
  - fenomas 1y ago
    Respectfully, you've only answered the thing I pre-replied to in my final paragraph. My question was why the take-home part adds signal?
- stingraycharles 1y ago
  This is a data engineer’s fizzbuzz and you would be surprised at how many people actually fail it. If they didn’t write it themselves, they will absolutely fail in the follow-up session where the real evaluation happens and I ask them to scale the code, handle crash safety without using transactions, etc.
genewitch 1y ago
I don't know what ETL means (I could look it up) - regardless I can do that ETL challenge. Realtek SDR dongles and Python loved messing up CSV. The only question I'd have is if a null or and "incorrectly formatted" row was invalid or not. So I have to wonder, is this a junior position, or did too much hadoop rub off on me a decade and a half ago?
- stingraycharles 1y ago
  This is for a medior solution, but it’s more intended as a data engineer’s “fizzbuzz” and then in the follow-up call I’ll ask them how they would distribute the workload over multiple clusters, make it idempotent, etc — that’s where the real technical evaluation happens.
- mplanchard 1y ago
  "extract, transform, load" - it's an acronym often used to describe typical data wrangling from one source to another

Teever 1y ago

The solution to this problem ultimately needs to be a regulatory one. People should be paid for the time they spend in interviews. You make that the law and this ambiguous hoop jumping bullshit goes away real fast. Companies will optimize for cost instead of dumping cost onto the prospective employees. This is good for the economy because it forces companies to innovate and optimize the interview process and it saves hundreds hours of totally economically unproductive time on the part of candidates.

jv22222 1y ago
Yes. It would also force them to screen resumes and do better profile research or they wouldn’t spend the money.
quantadev 1y ago
That's basically my feeling too. No one would ask for a maid to clean their house free for 4 hours, in order to decide if they want to hire her long term. And that's precisely what hiring managers have been getting away with for decades, because no one is pushing back. The managers are essentially abusing their power. It's all about abuse of power and utter arrogance.

quantadev 1y ago

But if you're unable to determine if someone has this knowledge via a phone call, instead of 4 hours of their work without you even being present, then that failing is on you, not them. If you can't judge someone's knowledge by asking questions, then you don't know how to come up with the right questions. Again that's on you. The only thing a 4-hr take home test will filter for is desperation. You'll get the most desperate candidates, and you'll do so by wasting days and days of people's time once you add it all up. It's just utterly disrespectful to demand these silly homework problems. I always take it that way. I take it personally, tbh. I find it absolutely insulting to even be asked and I simply refuse.

seancorfield 1y ago
I'm surprised this seemed to be voted down. I've been a hiring manager for over 30 years, and I never do "technical tests" -- no take-home, no live coding, none of it. I have a map of topics and questions, and I get the candidate chatting about their past projects, their approach, and what they liked/disliked about past projects and various technology they've used. It takes a maximum of one hour and usually close to about thirty minutes to make a yes/no decision on a candidate (sometimes it only takes ten minutes to make a no decision, and then it's a matter of trying to politely end the interview). I've interviewed hundreds of candidates this way over the years, and everyone I've hired has been capable of doing the job. Not once have I ever had to let someone go for lack of technical ability. Part of the problem is that we don't train people how to conduct interviews, and another part is "this is how I was interviewed, so this is how I'm going to interview other candidates" -- pure inertia. As an industry, we really need to do better. As for the OP, _if_ I had been administering a vague take-home project that had a 1-week delivery deadline, and a candidate peppered me with Qs and then presented a full proposal for the project for approval, prior to working on it... I would have rejected them. But I'm pretty certain I would have decided to reject them in my regular 30-60 "chat" interview, and I would not have moved on to the take-home project and wasted their time like that. So, again, I fault the interviewer(s) for not being able to filter candidates efficiently.
- quantadev 1y ago
  I had an interview once to work for IPFS, with a guy who already knew my skill level well (decades of experience), because I was active on their web forum solving everyone's problems, for about a year. Even during the interview he mentioned the technical part of the interview wasn't even necessary, because he had total confidence in me, and didn't even ask technical questions at all. He spent the ENTIRE hour selling me on the job. I hardly had time to speak. Everything went perfectly fine. Then he asked me to do the take-home project at the very end, just because "everybody has to", and I politely told him sorry I don't do those. So that ended the opportunity. So you're exactly right it's absolutely an "inertia" thing.

fenomas 1y ago

I agree about Kagi, but about your process wanted to ask: do you really feel like having them write the code as a take-home task adds any signal to the process? At my current gig we do a 1-hour pair programming kind of thing, where we video-chat and watch the candidate work on a small, straightforward task. And as the interviewer I watch how they use the tools, where they go for docs, do they read the requirements, how do they test, etc. By the end I always feel like I have a strong picture of their ability, and the whole thing is capped at 90 minutes for both sides (adding time for their questions). If the candidate's code was written offline beforehand, I'd have no idea whether it was theirs, whether it came from a friend or chatGPT, whether it took ten minutes or ten hours, etc. Sure I could try to suss out those things during discussion, but isn't it better to observe directly?

deleted 1y ago
[deleted]
davezatch 1y ago
Leaving aside the non-trivial question of candidate nervousness in live coding, I do find the "ask questions about take-home" process very illuminating. If it came from a friend or chatGPT, you'll learn that quickly because you can ask them stuff like "show me how you're implementing FOO. OK, if the requirements change, and now we need to BAR, what do you need to change across the app?" If they wrote the code and understood the task, they should understand what needs to change, how to do it, etc, and you can do that part with them live. Maybe someday LLMs will be fast enough that you can't catch someone constantly waiting for the AI to help them, at least as of May 2025 we're not there yet.
- fenomas 1y ago
  Respectfully, you've only answered the thing I pre-replied to in my final paragraph. My question was why the take-home part adds signal?
stingraycharles 1y ago
This is a data engineer’s fizzbuzz and you would be surprised at how many people actually fail it. If they didn’t write it themselves, they will absolutely fail in the follow-up session where the real evaluation happens and I ask them to scale the code, handle crash safety without using transactions, etc.

genewitch 1y ago

I don't know what ETL means (I could look it up) - regardless I can do that ETL challenge. Realtek SDR dongles and Python loved messing up CSV. The only question I'd have is if a null or and "incorrectly formatted" row was invalid or not. So I have to wonder, is this a junior position, or did too much hadoop rub off on me a decade and a half ago?

stingraycharles 1y ago
This is for a medior solution, but it’s more intended as a data engineer’s “fizzbuzz” and then in the follow-up call I’ll ask them how they would distribute the workload over multiple clusters, make it idempotent, etc — that’s where the real technical evaluation happens.
mplanchard 1y ago
"extract, transform, load" - it's an acronym often used to describe typical data wrangling from one source to another