4 ms·

You should just openly let them use chatgpt (assuming they can use it on the job too). When I interview people I try to create the same environment as the one t

You should just openly let them use chatgpt (assuming they can use it on the job too). When I interview people I try to create the same environment as the one they’ll be working in. They can use chatgpt, google, stack overflow, etc. I don’t care how many tools they have to use, as long as the work output is good and done in a reasonable time. I really don’t understand the obsession with coding on whiteboards or other situations that will literally never come up on the job. There will never be a time my employees can’t use google or chatgpt. In any case, you can tell pretty quickly how much someone knows about a topic just based on the questions they’re asking chatgpt.

recursive 3y ago
If they're just putting the question straight into GPT, then what benefit is the candidate bringing? I can use GPT myself, and for a lot cheaper than the cheapest candidate.
- kmoser 3y ago
  If the interview is for a position in which the candidate will be tasked with solving problems that ChatGPT is able to help with significantly, then they have just proven they are capable of doing the job. (If you have time to do this work yourself, why are you interviewing anybody at all?) Assuming the interview is to determine somebody's programming chops, without the benefit of ChatGPT, you'll have to ask questions where ChatGPT is little to no help. This was the conclusion of the article.
  - seattle_spring 3y ago
    But how will I know if they can implement a function to rotate a binary tree from memory?
    - kmoser 3y ago
      Why does it matter to you if they can do it from memory, if they can find the answer easily from ChatGPT? It's like asking "how can I tell if somebody knows the exact definition of a word without having to look it up?" If it's really important that they have that ability (e.g. because you will be asking them to perform other tasks which are not so easily solved by ChatGPT, or you simply don't want them using ChatGPT at their job for whatever reason), then you will have to devise an interview scenario where the candidate is incapable of using ChatGPT clandestinely, e.g. by bringing them into your office.
      - seattle_spring 3y ago
        I was being facetious.
- n_plus_1_acc 3y ago
  You evaluate whether they ceitically review the LLM answer or just take it at truth.
Terr_ 3y ago
Whoah, hold up: Why should we believe that success using an LLM to (possibly blindly) look up the answer to interview-questions will strongly correlate to success using an LLM to craft good code, properly tested, and their ability to debug it and fit it into an existing framework? Heck, at that point you aren't even measuring whether the candidate understood the question, nor their ability to communicate about it with prospective coworkers. If there are any questions where "repeat whatever ChatGPT says" seems like a fair and reasonable answer, that probably means it's a bad question that should be removed instead. Just like how "I'd just check the API docs" indicates you shouldn't be asking trivia about the order of parameters in a standard library method or whatever.
- jliptzin 3y ago
  Nothing I hire for requires someone to do the World’s Most Challenging ™ life or death problems under pressure from memory. I think that’s true for the vast majority of tech companies. If I need someone to wire up a database to a react interface, or write some cron scripts, or refactor an old nodejs codebase, that is all stuff that chatgpt would be a great tool to use. I don’t care whether they’re doing it from memory or not.
  - Terr_ 3y ago
    > Nothing I hire for requires someone to do the World’s Most Challenging [...] from memory That's a bit of a strawman: I didn't say anything about the ease/difficulty of the role being filled, and I implied rote memorization was not meaningful. To reiterate, interviews should measure good data for choosing between candidates. That's not happening when the given problem is solve-able by an LLM using a human as a proxy, everybody's just burning man-hours of company/applicant time on interview-theater that isn't useful for making a decision. (Well, not unless the hiring goals include "willingness to jump through hoops".)
    - jliptzin 3y ago
      And what if every problem the position I am hiring for is solvable by an LLM?

recursive 3y ago

If they're just putting the question straight into GPT, then what benefit is the candidate bringing? I can use GPT myself, and for a lot cheaper than the cheapest candidate.

kmoser 3y ago
If the interview is for a position in which the candidate will be tasked with solving problems that ChatGPT is able to help with significantly, then they have just proven they are capable of doing the job. (If you have time to do this work yourself, why are you interviewing anybody at all?) Assuming the interview is to determine somebody's programming chops, without the benefit of ChatGPT, you'll have to ask questions where ChatGPT is little to no help. This was the conclusion of the article.
- seattle_spring 3y ago
  But how will I know if they can implement a function to rotate a binary tree from memory?
  - kmoser 3y ago
    Why does it matter to you if they can do it from memory, if they can find the answer easily from ChatGPT? It's like asking "how can I tell if somebody knows the exact definition of a word without having to look it up?" If it's really important that they have that ability (e.g. because you will be asking them to perform other tasks which are not so easily solved by ChatGPT, or you simply don't want them using ChatGPT at their job for whatever reason), then you will have to devise an interview scenario where the candidate is incapable of using ChatGPT clandestinely, e.g. by bringing them into your office.
    - seattle_spring 3y ago
      I was being facetious.
n_plus_1_acc 3y ago
You evaluate whether they ceitically review the LLM answer or just take it at truth.

Terr_ 3y ago

Whoah, hold up: Why should we believe that success using an LLM to (possibly blindly) look up the answer to interview-questions will strongly correlate to success using an LLM to craft good code, properly tested, and their ability to debug it and fit it into an existing framework? Heck, at that point you aren't even measuring whether the candidate understood the question, nor their ability to communicate about it with prospective coworkers. If there are any questions where "repeat whatever ChatGPT says" seems like a fair and reasonable answer, that probably means it's a bad question that should be removed instead. Just like how "I'd just check the API docs" indicates you shouldn't be asking trivia about the order of parameters in a standard library method or whatever.

jliptzin 3y ago
Nothing I hire for requires someone to do the World’s Most Challenging ™ life or death problems under pressure from memory. I think that’s true for the vast majority of tech companies. If I need someone to wire up a database to a react interface, or write some cron scripts, or refactor an old nodejs codebase, that is all stuff that chatgpt would be a great tool to use. I don’t care whether they’re doing it from memory or not.
- Terr_ 3y ago
  > Nothing I hire for requires someone to do the World’s Most Challenging [...] from memory That's a bit of a strawman: I didn't say anything about the ease/difficulty of the role being filled, and I implied rote memorization was not meaningful. To reiterate, interviews should measure good data for choosing between candidates. That's not happening when the given problem is solve-able by an LLM using a human as a proxy, everybody's just burning man-hours of company/applicant time on interview-theater that isn't useful for making a decision. (Well, not unless the hiring goals include "willingness to jump through hoops".)
  - jliptzin 3y ago
    And what if every problem the position I am hiring for is solvable by an LLM?