3 ms·

I strongly disagree. Your ability to use ChatGPT effectivelly is highly dependent on your technical competence. The interview is meant to measure your acquire

I strongly disagree. Your ability to use ChatGPT effectivelly is highly dependent on your technical competence. The interview is meant to measure your acquired competence, because this is the harder part. Learning to leverage that competence using ChatGPT is very easy. I'd rather have a developer on my team that demonstrates high technical competence than one that is GPT-skilled, but doesn't know what questions to ask GPT nor how to judge its responses.

ben_w 3y ago
> Your ability to use ChatGPT effectivelly is highly dependent on your technical competence. Indeed. At this point, one thing I'd do is stick a candidate in front of some code that (a) didn't work, (b) which came from ChatGPT, and (c) which ChatGPT cannot itself fix, and see if the candidate can fix it.
- sanderjd 3y ago
  This is indeed an excellent idea for an interview. (By far my favorite hour of interviewing the last time around was the Stripe "debugging" interview, which is quite similar to this.) But maybe they will still use chatgpt to help them figure out the solution. A non-trivial number of my interactions with it are of this form, "no, that isn't right, fix this part and re-do the rest". And that should be fine. Or it should be fine to not do it that way. The goal is to get a sense for how the person you may or may not be working with approaches solving problems. Sometimes LLMs are part of how they do that, sometimes they aren't. And you can learn something about them from that, either way.
sanderjd 3y ago
> Your ability to use ChatGPT effectivelly is highly dependent on your technical competence. Ok, then that seems like a pretty reasonable thing to assess? > I'd rather have a developer on my team that demonstrates high technical competence than one that is GPT-skilled, but doesn't know what questions to ask GPT nor how to judge its responses. But "what questions does the candidate ask an LLM and how do they judge its responses" is part of the interview, if you don't forbid them from using an LLM! Now, if they don't want to use these tools, if that's not part of their normal process while working, then that's totally fine too. But if they're comfortable with these tools, if they are part of their normal set of things they use for their work, then you're doing yourself a disservice by designing an interview process that is incapable of accomodating that.
- rmbyrro 3y ago
  The interview is imperfect, very quick, and it's already hard to measure competence. As the article shows, it's much easier to mimick competence with the help of a chatbot. That obviously doesn't mean one actually is competent to produce good work in a real setting.
  - sanderjd 3y ago
    > As the article shows, it's much easier to mimick competence with the help of a chatbot. I don't think that's what the article shows. I think it shows that it's useless to ask "leetcode" questions and focus on the code produced rather than expecting candidates to walk through their thought process and show what tools they're using to aid it.
TrackerFF 3y ago
> Your ability to use ChatGPT effectivelly is highly dependent on your technical competence. In that case, let everyone use ChatGPT and similar tools. Those that know, will likely not use it much. Those that do now know, or are not too confident on themselves, will use it more.

ben_w 3y ago

> Your ability to use ChatGPT effectivelly is highly dependent on your technical competence. Indeed. At this point, one thing I'd do is stick a candidate in front of some code that (a) didn't work, (b) which came from ChatGPT, and (c) which ChatGPT cannot itself fix, and see if the candidate can fix it.

sanderjd 3y ago
This is indeed an excellent idea for an interview. (By far my favorite hour of interviewing the last time around was the Stripe "debugging" interview, which is quite similar to this.) But maybe they will still use chatgpt to help them figure out the solution. A non-trivial number of my interactions with it are of this form, "no, that isn't right, fix this part and re-do the rest". And that should be fine. Or it should be fine to not do it that way. The goal is to get a sense for how the person you may or may not be working with approaches solving problems. Sometimes LLMs are part of how they do that, sometimes they aren't. And you can learn something about them from that, either way.

sanderjd 3y ago

> Your ability to use ChatGPT effectivelly is highly dependent on your technical competence. Ok, then that seems like a pretty reasonable thing to assess? > I'd rather have a developer on my team that demonstrates high technical competence than one that is GPT-skilled, but doesn't know what questions to ask GPT nor how to judge its responses. But "what questions does the candidate ask an LLM and how do they judge its responses" is part of the interview, if you don't forbid them from using an LLM! Now, if they don't want to use these tools, if that's not part of their normal process while working, then that's totally fine too. But if they're comfortable with these tools, if they are part of their normal set of things they use for their work, then you're doing yourself a disservice by designing an interview process that is incapable of accomodating that.

rmbyrro 3y ago
The interview is imperfect, very quick, and it's already hard to measure competence. As the article shows, it's much easier to mimick competence with the help of a chatbot. That obviously doesn't mean one actually is competent to produce good work in a real setting.
- sanderjd 3y ago
  > As the article shows, it's much easier to mimick competence with the help of a chatbot. I don't think that's what the article shows. I think it shows that it's useless to ask "leetcode" questions and focus on the code produced rather than expecting candidates to walk through their thought process and show what tools they're using to aid it.

TrackerFF 3y ago

> Your ability to use ChatGPT effectivelly is highly dependent on your technical competence. In that case, let everyone use ChatGPT and similar tools. Those that know, will likely not use it much. Those that do now know, or are not too confident on themselves, will use it more.