4 ms·

Man, I hate take-homes. When I see instructions that say “This project tests your ability not only to code, but to deal with ambiguity and open-endedness”, it

Man, I hate take-homes. When I see instructions that say “This project tests your ability not only to code, but to deal with ambiguity and open-endedness”, it actually means either a) this exercise has been so poorly conceived that we didn’t bother with a rubric or b) the work environment is so chaotic, we never clearly define specs or requirements for anything. That, and the instruction to simultaneously “show off your skills” and deliver a pragmatic functional solution are completely at odds. Good simple solutions aren’t flashy.

danpalmer 1y ago
To me, take-homes that have UI or API integrations are a bit of a red flag, because UI code (for most roles) is relatively boring, and API integrations are a lot of faff with not much signal. Cool you can make an HTTP request, cool you've got a basic CRUD editing setup. It's a lot of code that takes a lot of time and tells me almost nothing about how you code, hell, AI tools will happily generate these things in no time and at a pretty good quality. What I want to see is an engineer implement an awkward bit of business logic. Does it become a million nested if statements and a "here be dragons" comment at the top, or do they identify the right patterns and build something that I can reason about when reading the code? This is far more valuable in the job, more signal in the interview, and much harder for AI to get right. It also takes less time.
- parpfish 1y ago
  I’ve seen a few that let you fork a repo with all that boilerplate set up and you just have a few stubbed methods to fill out, and that seems reasonable. But going from 0->1 on an app is so much grunt work and I doubt the reviewers would even look into all of it
  - worthless-trash 1y ago
    I believe they don't. I think its a filter that they want people they can push into too much work to apply.
  - danpalmer 1y ago
    I would say that's reasonable, although it again sorta depends on the stubbed methods and what you're trying to figure out from them, and I'd question the necessity of the boilerplate - can't you just implement the methods?
- dzhiurgis 1y ago
  Can I ask you and everyone else - why do AI is so good at UI/CRUD apps and terrible at business logic? I've been caught with this few times now. Spend ages trying to coerce AI to solve logic problem and end up just manually solving it myself. Whereas UIs are so good and usually near perfect from first prompt. I suspect it's the weak prompt. I need to learn and solve this before my brain completely atrophies (there must be Anthropic joke here somewhere hehe).
  - danpalmer 1y ago
    I doubt it's prompting. I think the issue is more likely that UI code is often similar, has a lot of examples online, and often doesn't require understanding data flow. This is why LLMs are great at "boring" React components, because they don't actually understand the flow of the data, but they don't need to. Business logic on the other hand is much more likely to be novel in some way, there are likely fewer similar examples for rules to be learnt from, etc. Obviously this is all gradations, LLMs can manage some business logic and mess up some UIs (they can't "see" the UI which doesn't help!), but this is my experience of them and fits with my understanding of the technology.
carlosjobim 1y ago
It's a start-up with a lot of different projects. It is expected that the work environment would be chaotic. People who look for a no-surprises type of work week should probably not apply to such a kind of workplace. > Good simple solutions aren’t flashy. Maybe showing off your skills is making a good and simple solution?
- parpfish 1y ago
  Sure, the work is chaotic. But why would the interview process need to be chaotic? You want to get the best signals of ability that you can, and one thing you can do to help that is to make sure that you’ve given an assignment that will be evaluated consistently on your end and understood uniformly by the participants
  - lmm 1y ago
    > Sure, the work is chaotic. But why would the interview process need to be chaotic? Because the interview is meant to measure how well someone would perform on the work? > You want to get the best signals of ability that you can, and one thing you can do to help that is to make sure that you’ve given an assignment that will be evaluated consistently on your end and understood uniformly by the participants Well sure, all else being equal, but if the cost of consistency is an assignment that doesn't reflect the actual work environment then that may outweigh the benefit.
    - parpfish 1y ago
      But why conflate “technical skill assessment” and “ambiguity handling” into one big stressful test? Doing two small tests toy can measure each skill independently A clear coding test to get technical skill and then some in person requirements gathering exercise or something to measure ambiguity handling. You’ll get better insight into their abilities with less effort for the interviewee AND the interviewer (if the startup is really that chaotic, do you really think they are doing a careful code review on a zillion different bespoke takehomes?)
      - 1y ago
deleted 1y ago
[deleted]
ncallaway 1y ago
> we never clearly define specs or requirements for anything. I mean, what if part of the role that’s being hired for is to help people clearly define the specs and requirements? I consider that a big part of what it means to be a senior software engineer. Maybe this exact format isn’t the best way to test for it, but I don’t think “we want to see if someone can deal with ambiguity” means all the work is ambiguous. It might just mean all work needs to go through a process to take it from “ambiguous requirements” to “clearly defined specs”
- duskwuff 1y ago
  > I mean, what if part of the role that’s being hired for is to help people clearly define the specs and requirements? Then I'd expect the interview process to focus on that process, not on the final deliverable. As it stands, the candidate tried to interact with the interviewer by trying to ask for clarification on the requirements, then making a detailed proposal, and got no actionable feedback from either.
  - theamk 1y ago
    Could one do a "terminal inspired, mutt-like email client" that completely satisfies OP's proposal (web-based, TEMPL, pocketbase, pulumi, etc...)? Sure, it would be possible. None of those are _required_ for the submission but you can totally use them if you wanted to. So the interviewer probably thought, "hey this person is going all out", and was absolutely honest in saying "Looking forward to receiving your submission" I cannot really blame the interviewer for not reminding the candidate that they should actually follow the requirements, in addition to all the optional stuff they mentioned in their spec.
    - josecodea 1y ago
      > Sure, it would be possible I have to be blunt, you have no idea what you are talking about. I am just going to say this: A terminal app is completely at odds with an online web client. If the interviewer is as clueless as you are, that explains two things: - I was never going to pass the interview in the first place - I was never going to get a reasonable conversation from this person, no amount of communication is going to solve incompetence from the interviewer's side Corollary: - Gross negligence from the company for assigning this interviewer

danpalmer 1y ago

To me, take-homes that have UI or API integrations are a bit of a red flag, because UI code (for most roles) is relatively boring, and API integrations are a lot of faff with not much signal. Cool you can make an HTTP request, cool you've got a basic CRUD editing setup. It's a lot of code that takes a lot of time and tells me almost nothing about how you code, hell, AI tools will happily generate these things in no time and at a pretty good quality. What I want to see is an engineer implement an awkward bit of business logic. Does it become a million nested if statements and a "here be dragons" comment at the top, or do they identify the right patterns and build something that I can reason about when reading the code? This is far more valuable in the job, more signal in the interview, and much harder for AI to get right. It also takes less time.

parpfish 1y ago
I’ve seen a few that let you fork a repo with all that boilerplate set up and you just have a few stubbed methods to fill out, and that seems reasonable. But going from 0->1 on an app is so much grunt work and I doubt the reviewers would even look into all of it
- worthless-trash 1y ago
  I believe they don't. I think its a filter that they want people they can push into too much work to apply.
- danpalmer 1y ago
  I would say that's reasonable, although it again sorta depends on the stubbed methods and what you're trying to figure out from them, and I'd question the necessity of the boilerplate - can't you just implement the methods?
dzhiurgis 1y ago
Can I ask you and everyone else - why do AI is so good at UI/CRUD apps and terrible at business logic? I've been caught with this few times now. Spend ages trying to coerce AI to solve logic problem and end up just manually solving it myself. Whereas UIs are so good and usually near perfect from first prompt. I suspect it's the weak prompt. I need to learn and solve this before my brain completely atrophies (there must be Anthropic joke here somewhere hehe).
- danpalmer 1y ago
  I doubt it's prompting. I think the issue is more likely that UI code is often similar, has a lot of examples online, and often doesn't require understanding data flow. This is why LLMs are great at "boring" React components, because they don't actually understand the flow of the data, but they don't need to. Business logic on the other hand is much more likely to be novel in some way, there are likely fewer similar examples for rules to be learnt from, etc. Obviously this is all gradations, LLMs can manage some business logic and mess up some UIs (they can't "see" the UI which doesn't help!), but this is my experience of them and fits with my understanding of the technology.

carlosjobim 1y ago

It's a start-up with a lot of different projects. It is expected that the work environment would be chaotic. People who look for a no-surprises type of work week should probably not apply to such a kind of workplace. > Good simple solutions aren’t flashy. Maybe showing off your skills is making a good and simple solution?

parpfish 1y ago
Sure, the work is chaotic. But why would the interview process need to be chaotic? You want to get the best signals of ability that you can, and one thing you can do to help that is to make sure that you’ve given an assignment that will be evaluated consistently on your end and understood uniformly by the participants
- lmm 1y ago
  > Sure, the work is chaotic. But why would the interview process need to be chaotic? Because the interview is meant to measure how well someone would perform on the work? > You want to get the best signals of ability that you can, and one thing you can do to help that is to make sure that you’ve given an assignment that will be evaluated consistently on your end and understood uniformly by the participants Well sure, all else being equal, but if the cost of consistency is an assignment that doesn't reflect the actual work environment then that may outweigh the benefit.
  - parpfish 1y ago
    But why conflate “technical skill assessment” and “ambiguity handling” into one big stressful test? Doing two small tests toy can measure each skill independently A clear coding test to get technical skill and then some in person requirements gathering exercise or something to measure ambiguity handling. You’ll get better insight into their abilities with less effort for the interviewee AND the interviewer (if the startup is really that chaotic, do you really think they are doing a careful code review on a zillion different bespoke takehomes?)
    - 1y ago

deleted 1y ago

[deleted]

ncallaway 1y ago

> we never clearly define specs or requirements for anything. I mean, what if part of the role that’s being hired for is to help people clearly define the specs and requirements? I consider that a big part of what it means to be a senior software engineer. Maybe this exact format isn’t the best way to test for it, but I don’t think “we want to see if someone can deal with ambiguity” means all the work is ambiguous. It might just mean all work needs to go through a process to take it from “ambiguous requirements” to “clearly defined specs”

duskwuff 1y ago
> I mean, what if part of the role that’s being hired for is to help people clearly define the specs and requirements? Then I'd expect the interview process to focus on that process, not on the final deliverable. As it stands, the candidate tried to interact with the interviewer by trying to ask for clarification on the requirements, then making a detailed proposal, and got no actionable feedback from either.
- theamk 1y ago
  Could one do a "terminal inspired, mutt-like email client" that completely satisfies OP's proposal (web-based, TEMPL, pocketbase, pulumi, etc...)? Sure, it would be possible. None of those are _required_ for the submission but you can totally use them if you wanted to. So the interviewer probably thought, "hey this person is going all out", and was absolutely honest in saying "Looking forward to receiving your submission" I cannot really blame the interviewer for not reminding the candidate that they should actually follow the requirements, in addition to all the optional stuff they mentioned in their spec.
  - josecodea 1y ago
    > Sure, it would be possible I have to be blunt, you have no idea what you are talking about. I am just going to say this: A terminal app is completely at odds with an online web client. If the interviewer is as clueless as you are, that explains two things: - I was never going to pass the interview in the first place - I was never going to get a reasonable conversation from this person, no amount of communication is going to solve incompetence from the interviewer's side Corollary: - Gross negligence from the company for assigning this interviewer