4 ms·

I only saw this mentioned in 1 other top-level comment -- ChatGPT (and yes, sorry, you may be sick of hearing about it). There's a perspective starting to come

I only saw this mentioned in 1 other top-level comment -- ChatGPT (and yes, sorry, you may be sick of hearing about it). There's a perspective starting to come up, ie ChatGPT is ruining take-homes. This tweet[1] is one example from the hiring side, but I wouldn't be surprised if it ends up flooding their inboxes and your submission suddenly is competing for time with a bunch of bots. [1] https://twitter.com/djsmith42/status/1661156114883567616 https://twitter.com/djsmith42/status/1661156114883567616

cortesoft 3y ago
If a developer can complete a take home test with ChatGPT, can they also do their regular job with ChatGPT?
- whateveracct 3y ago
  Take-home tests tend to be simplified and well-defined questions. Not like real life and definitely something ChatGPT would do better in.
- CSMastermind 3y ago
  Take home assignments are poor indicators of someone's actual software engineering ability. Unless you think of software engineers as some nameless, faceless, machines that you feed requirements into and get code out of. ChatGPT just highlights how poor of a job takehome assignments do at evaluating software engineering talent because it literally is a nameless, faceless, machine that you feed requirements into and get code out of. In a live interview with a competent interviewer they're watching to see your thought process, how well you articulate your thought process, how you problem solve, etc. Very little of the interview or their feedback about it should be focused on the syntax of your code. In addition to being disrespectful of a candidate's time, takehome interviews lose all those important signals.
  - imiric 3y ago
    Hard disagree. Following your logic, individual contributors working remotely are not software engineers, which is ludicrous. The thought process or tools they used to reach a solution is not as relevant as the solution itself. Interviewers shouldn't just review syntax, but efficiency, comprehension, algorithms, creativity, and many other attributes the code itself tells you. After all, these are things we spend most of our time evaluating in code reviews. Communication is also key, which can be evaluated in a live review of the code. Programmers are usually happy to talk about their code, and here you can assess not only if they actually wrote it, but what tradeoffs they made, what difficulties they ran into, how they would further improve it, etc. This is also a chance for the interviewee to evaluate the company's code review process; what they focus on, how _they_ communicate, etc. It's incredibly valuable for both sides. What live coding assesses most is the ability to think on your feet, and performance under the spotlight. This is an unnatural state for programming, which requires deep and creative thinking, often done best in isolation. Not to mention that it's heavily biased against individuals with any form of performance or social anxiety, which sends all the wrong signals you don't want when evaluating someone's programming abilities. AI tools are simply an evolution of the software engineer's toolbelt, and if they're allowed on the job itself, they should also be allowed in the take home assignment. It's a mistake to create fake working conditions for potential hires that don't exist for actual employees. At that point you're evaluating based on some fabricated environment, and won't be certain what it's like to work with this person, which is the only thing that ultimately matters.
    - CSMastermind 3y ago
      > Following your logic, individual contributors working remotely are not software engineers, which is ludicrous. Do you just send requirements to your remote engineers then get code back without ever talking to them? That seems like a really bad way to make software.
      - imiric 3y ago
        Of course not. Communication is still important, and you discuss and plan features that need to be implemented. What you don't do is hover over their shoulder and monitor how they solve problems.
shakes_mcjunkie 3y ago
I've been wondering if I should just use chatgpt or copilot for take home assessments now.
johnnyanmac 3y ago
The cynical side of me wants to yell "good". But leveling it out a bit: if your test can be cheated by a context-less machine given the right prompt, is it really an effective test? Are you okay with just hiring someone who has so far, at best (if we assume a far future) code as well as said context-less machine?

cortesoft 3y ago

If a developer can complete a take home test with ChatGPT, can they also do their regular job with ChatGPT?

whateveracct 3y ago
Take-home tests tend to be simplified and well-defined questions. Not like real life and definitely something ChatGPT would do better in.
CSMastermind 3y ago
Take home assignments are poor indicators of someone's actual software engineering ability. Unless you think of software engineers as some nameless, faceless, machines that you feed requirements into and get code out of. ChatGPT just highlights how poor of a job takehome assignments do at evaluating software engineering talent because it literally is a nameless, faceless, machine that you feed requirements into and get code out of. In a live interview with a competent interviewer they're watching to see your thought process, how well you articulate your thought process, how you problem solve, etc. Very little of the interview or their feedback about it should be focused on the syntax of your code. In addition to being disrespectful of a candidate's time, takehome interviews lose all those important signals.
- imiric 3y ago
  Hard disagree. Following your logic, individual contributors working remotely are not software engineers, which is ludicrous. The thought process or tools they used to reach a solution is not as relevant as the solution itself. Interviewers shouldn't just review syntax, but efficiency, comprehension, algorithms, creativity, and many other attributes the code itself tells you. After all, these are things we spend most of our time evaluating in code reviews. Communication is also key, which can be evaluated in a live review of the code. Programmers are usually happy to talk about their code, and here you can assess not only if they actually wrote it, but what tradeoffs they made, what difficulties they ran into, how they would further improve it, etc. This is also a chance for the interviewee to evaluate the company's code review process; what they focus on, how _they_ communicate, etc. It's incredibly valuable for both sides. What live coding assesses most is the ability to think on your feet, and performance under the spotlight. This is an unnatural state for programming, which requires deep and creative thinking, often done best in isolation. Not to mention that it's heavily biased against individuals with any form of performance or social anxiety, which sends all the wrong signals you don't want when evaluating someone's programming abilities. AI tools are simply an evolution of the software engineer's toolbelt, and if they're allowed on the job itself, they should also be allowed in the take home assignment. It's a mistake to create fake working conditions for potential hires that don't exist for actual employees. At that point you're evaluating based on some fabricated environment, and won't be certain what it's like to work with this person, which is the only thing that ultimately matters.
  - CSMastermind 3y ago
    > Following your logic, individual contributors working remotely are not software engineers, which is ludicrous. Do you just send requirements to your remote engineers then get code back without ever talking to them? That seems like a really bad way to make software.
    - imiric 3y ago
      Of course not. Communication is still important, and you discuss and plan features that need to be implemented. What you don't do is hover over their shoulder and monitor how they solve problems.

shakes_mcjunkie 3y ago

I've been wondering if I should just use chatgpt or copilot for take home assessments now.

johnnyanmac 3y ago

The cynical side of me wants to yell "good". But leveling it out a bit: if your test can be cheated by a context-less machine given the right prompt, is it really an effective test? Are you okay with just hiring someone who has so far, at best (if we assume a far future) code as well as said context-less machine?