4 ms·

As an interviewer, if a candidate can use chatGPT to give a better answer than other candidates, I'm not gonna mark them down for use of chatGPT. It's a tool,

As an interviewer, if a candidate can use chatGPT to give a better answer than other candidates, I'm not gonna mark them down for use of chatGPT. It's a tool, and if they can master it to make it useful, then credit to them. Alas, ChatGPT seems to be a jack of all trades, but master of none, which is gonna make it hard to pass my interviews which test very specific technical skills.

lucasmullens 3y ago
That's like being okay with a candidate Googling the answer during an interview. Not unheard of, but unusual. It seems hard to test someone's knowledge that way.
- londons_explore 3y ago
  I'm okay with them googling too. And I tell them that at the start. But if they take ages to lookup the answer when others just know the answer, it's gonna hurt their chances.
  - gtirloni 3y ago
    Sure, they can search it live but you have to assess if they understand what they found. Usually, if they really know their stuff, whatever they find is just gently pushing their working memory to connect the dots and give a decent answer. Otherwise it's pretty easy to ask a follow up question and see a candidate struggle. It's like in college when you're allowed to take textbooks to an exam. You can bet the professor spent more time crafting questions that you can't answer blindly. That being said, I think both types of questions have their place in an interview process. You can start with the no searching allowed questions in the beginning to assess real basic knowledge and, once you determine the candidate has some knowledge, you start probing more to see if they can connect the dots, maybe it's architecture decisions and their consequences, maybe it's an unexpected requirement and how they would react, etc.
- yunwal 3y ago
  Seems much easier than trying to prevent them from using google
- krisoft 3y ago
  > Not unheard of, but unusual. At my company we tell people that they should feel free to google or consult references at practical coding challenges. > It seems hard to test someone's knowledge that way. I don’t really want to test knowledge but skill. Can you do the thing? At work you will have access to these references so why not during the interview? Now that doesn’t mean that we are not taking note when you go searching and what you go searching for. If you told us that you spent the last 8 years of your life working with python and you totally blank on the syntax of how to write a class that is suspicious. If you don’t remember the argument order of some obscure method? Who cares. If you worked in so many languages that you don’t remember if the Lock class in this particular one is reentrant or not and have to look it up? You might even get “bonus points” for saying something like that because it demonstrates a broad interest and attention to detail. (Assuming that using a Lock is reasonable in the situation and so on of course :))
  - deleted 3y ago
    [deleted]
  - batch12 3y ago
    > I don’t really want to test knowledge but skill I do want to understand their knowledge. I'll preface questions with the disclaimer that I am not looking for the book definition of a concept, but to understand if the candidate understands the topic and to what depth. I'll often tell them that if they dont know, just say so. I'll start with a simple question and keep digging deeper until either they bottom out or I do.
- munk-a 3y ago
  The knowledge we're testing is related to how well you can do your job. Work isn't closed book - if you can quickly formulate a good query to grab any missing information off the internet then more power to you. I've worked with extremely stubborn people who were very smart and would spend a week trying to sort out a problem before googling it, there are some limited situations (highly experimental work) where this is valuable but... I no longer work with these people.
photonthug 3y ago
> As an interviewer, if a candidate can use chatGPT to give a better answer than other candidates, I'm not gonna mark them down for use of chatGPT. Tool usage is what separates us from animals and is generally ok where tools are available/expected, but in this case I think you misunderstand which tool we're talking about. The tool involved isn't actually chatGPT, it's more like strategic deception. Consider the structurally similar remark "as a voter, if a candidate can use lies to represent themselves as better than other candidates, I'm not gonna mark them down for use of dishonesty". The rest of this comment is not directed at you personally at all, but the number of folks in this thread who are extremely eager to make various excuses for dishonesty surprised me. The best one is "if dishonesty works, blame the interviewer". I get the superficial justification here like "should have asked better questions", but OTOH we all want fairly short interview processes, no homework, job-related questions without weird data-structures and algorithms pop-quizes, etc, so what's with the double standards? Hiring/firing is expensive, time-consuming, and tedious, and interviewing is also tedious. No one likes picking up the slack for fake coworkers. No one likes being lied to.
- WalterBright 3y ago
  > the number of folks in this thread who are extremely eager to make various excuses for dishonesty surprised me Not me. I see it all the time, online and offline. I suspect they think it confers status on themselves, but what actually happens is honest people wind up shunning them.
- verve_rat 3y ago
  The problem is that a good interview is only vaguely related to getting a good employee. Anyone can ace and interview and then slack off once they have to job. If someone aces the interview using an LLM and then does good work using that same LLM then what should the employer or other employees care? The work is getting done, so what's the problem? Compare a shitty worker to a deceptive one using an LLM. They both passed the interview and in both cases the work isn't being done. How are those two cases different?
  - recursive 3y ago
oceanplexian 3y ago
I remember the days when Greybeards would look down on me for using Google in my first IT job, they would harp on about how real Sysadmins use man pages and O’Reilly books to solve problems, and if you tried to Google something you were incompetent. I had college professors that told me you can’t use the Internet for research because the Internet is a not a legitimate source of information, only libraries can have real information. What happened to all those folks? They retired, and turned into Boomers who are now unable to function in society at a basic level and do things like online banking or operate a smartphone.
- esafak 3y ago
  On the other hand, they knew how their hardware worked. And if LLMs keep improving, we're going to reach the last generation that knew how software worked.
  - zerbinxx 3y ago
    We’re pretty close. I’m not sure that 51% of the people I work with understand what DNS is, what a call stack is, what the difference between inheritance and polymorphism is, or what a mutex is
- nonrandomstring 3y ago
  When I'm retired, sitting on the beach with my beer and a good book, please don't come bothing me that your smartphone banking and GPT arse-wiping assistant has gone berserk. You're on your own matey. But hey, you'll have Google.
kazinator 3y ago
Would you also say that, "as an interviewer, if a candidate can use their buddy to give a better answer than other candidates, I'm not going to mark them down for using their buddy"? Even if you don't mind that situation, shouldn't you get buddy's contact information and offer him the job?
- smcin 3y ago
  That's not a great analogy: you can't do the job with your buddy; whereas some interviewers are ok with, and even expect you to, use GenAI on the job daily. Depends on the interviewer and job expectations. A better analogy is an interview where you can use a calculator (and not be detected). If the interviewer were only to ask you simple arithmetic questions with numeric answers then sure you'd seem to do well. So interviewers adjust to not doing that.
  - toast0 3y ago
    > you can't do the job with your buddy I mean, why not? Me and my buddy are a team, hire us both or none of us. Split the salary if you must.
    - TeMPOraL 3y ago
      That's called contracting out and it's fine in this way - and so would be use of GPT, in that way and not to beat the interview.
    - kazinator 3y ago
      Sure, and also split the dental and other benefits, vacations, and share one building fob, parking pass, cubicle and computer. Also, split the food at the company dinner. :)
thegrim33 3y ago
To use a slightly more extreme example .. if you were hiring someone to maintain a nuclear power plant, and when you asked them a question about what actions to take to avoid a meltdown, and they had to ask ChatGPT to figure it out, would you really be OK with hiring that person to maintain your nuclear plant? When they don't actually have the knowledge they need to succeed, but instead have to rely on external tools to decide things? If they need to ask ChatGPT for the answer, how do they know if the answer is right? You really think that person, who relies on tools, is just as good of a hire as someone that fully internally knows what they need to know? Yeah, hiring someone to code a website isn't the same as maintaining a nuclear plant, but it's the same concept of someone that knows their craft vs. someone that needs to rely on tools. There's a major difference in my mind.
- fragmede 3y ago
  in terms of tools, I absolutely want the nuclear power plant engineer to use a wrench and pliars and tongs and a forklift and a machine while wearing a lead lined safety suit instead of wandering over to the reactor in a t-shirt to pull out the control rods with their bare hands. You could be Edward Teller and know everything there is to know about nuclear physics but you're not getting anywhere without tools. to your point though, a person needs both. all of one and none of the other is useless. You don't want someone who doesn't know what they're doing to play around disabling safety systems so you don't get Chernobyl, but for the everyday crud website you can just hire the coding monkey at a reduced cost.
- hackit2 3y ago
  I hope your statement is hyperbolic because we're all doomed if you expect a person to know how to operate a nuclear power plant. Normally, your testing if they can follow operational procedure that were created by people who designed the power plant in the first place. Similar it is unreasonable and bordering on negligence to assume a person has the skill set unique to your situation.
  - anileated 3y ago
    Like a university diploma is a signal of being able to learn or at least comply, use of a chatbot is a signal of not bothering enough to learn or comply. I can see how an applicant who cheats interview with chatbot would later not bother to internalize operation instructions for the job.
wakawaka28 3y ago
>As an interviewer, if a candidate can use chatGPT to give a better answer than other candidates, I'm not gonna mark them down for use of chatGPT. I think that makes you an incompetent interviewer, unless your questions are too hard for ChatGPT. In any case, solving the question without ChatGPT is more impressive than using it. Just like most other tools, like search engines or IDEs.

lucasmullens 3y ago

That's like being okay with a candidate Googling the answer during an interview. Not unheard of, but unusual. It seems hard to test someone's knowledge that way.

londons_explore 3y ago
I'm okay with them googling too. And I tell them that at the start. But if they take ages to lookup the answer when others just know the answer, it's gonna hurt their chances.
- gtirloni 3y ago
  Sure, they can search it live but you have to assess if they understand what they found. Usually, if they really know their stuff, whatever they find is just gently pushing their working memory to connect the dots and give a decent answer. Otherwise it's pretty easy to ask a follow up question and see a candidate struggle. It's like in college when you're allowed to take textbooks to an exam. You can bet the professor spent more time crafting questions that you can't answer blindly. That being said, I think both types of questions have their place in an interview process. You can start with the no searching allowed questions in the beginning to assess real basic knowledge and, once you determine the candidate has some knowledge, you start probing more to see if they can connect the dots, maybe it's architecture decisions and their consequences, maybe it's an unexpected requirement and how they would react, etc.
yunwal 3y ago
Seems much easier than trying to prevent them from using google
krisoft 3y ago
> Not unheard of, but unusual. At my company we tell people that they should feel free to google or consult references at practical coding challenges. > It seems hard to test someone's knowledge that way. I don’t really want to test knowledge but skill. Can you do the thing? At work you will have access to these references so why not during the interview? Now that doesn’t mean that we are not taking note when you go searching and what you go searching for. If you told us that you spent the last 8 years of your life working with python and you totally blank on the syntax of how to write a class that is suspicious. If you don’t remember the argument order of some obscure method? Who cares. If you worked in so many languages that you don’t remember if the Lock class in this particular one is reentrant or not and have to look it up? You might even get “bonus points” for saying something like that because it demonstrates a broad interest and attention to detail. (Assuming that using a Lock is reasonable in the situation and so on of course :))
- deleted 3y ago
  [deleted]
- batch12 3y ago
  > I don’t really want to test knowledge but skill I do want to understand their knowledge. I'll preface questions with the disclaimer that I am not looking for the book definition of a concept, but to understand if the candidate understands the topic and to what depth. I'll often tell them that if they dont know, just say so. I'll start with a simple question and keep digging deeper until either they bottom out or I do.
munk-a 3y ago
The knowledge we're testing is related to how well you can do your job. Work isn't closed book - if you can quickly formulate a good query to grab any missing information off the internet then more power to you. I've worked with extremely stubborn people who were very smart and would spend a week trying to sort out a problem before googling it, there are some limited situations (highly experimental work) where this is valuable but... I no longer work with these people.

photonthug 3y ago

> As an interviewer, if a candidate can use chatGPT to give a better answer than other candidates, I'm not gonna mark them down for use of chatGPT. Tool usage is what separates us from animals and is generally ok where tools are available/expected, but in this case I think you misunderstand which tool we're talking about. The tool involved isn't actually chatGPT, it's more like strategic deception. Consider the structurally similar remark "as a voter, if a candidate can use lies to represent themselves as better than other candidates, I'm not gonna mark them down for use of dishonesty". The rest of this comment is not directed at you personally at all, but the number of folks in this thread who are extremely eager to make various excuses for dishonesty surprised me. The best one is "if dishonesty works, blame the interviewer". I get the superficial justification here like "should have asked better questions", but OTOH we all want fairly short interview processes, no homework, job-related questions without weird data-structures and algorithms pop-quizes, etc, so what's with the double standards? Hiring/firing is expensive, time-consuming, and tedious, and interviewing is also tedious. No one likes picking up the slack for fake coworkers. No one likes being lied to.

WalterBright 3y ago
> the number of folks in this thread who are extremely eager to make various excuses for dishonesty surprised me Not me. I see it all the time, online and offline. I suspect they think it confers status on themselves, but what actually happens is honest people wind up shunning them.
verve_rat 3y ago
The problem is that a good interview is only vaguely related to getting a good employee. Anyone can ace and interview and then slack off once they have to job. If someone aces the interview using an LLM and then does good work using that same LLM then what should the employer or other employees care? The work is getting done, so what's the problem? Compare a shitty worker to a deceptive one using an LLM. They both passed the interview and in both cases the work isn't being done. How are those two cases different?
- recursive 3y ago

oceanplexian 3y ago

I remember the days when Greybeards would look down on me for using Google in my first IT job, they would harp on about how real Sysadmins use man pages and O’Reilly books to solve problems, and if you tried to Google something you were incompetent. I had college professors that told me you can’t use the Internet for research because the Internet is a not a legitimate source of information, only libraries can have real information. What happened to all those folks? They retired, and turned into Boomers who are now unable to function in society at a basic level and do things like online banking or operate a smartphone.

esafak 3y ago
On the other hand, they knew how their hardware worked. And if LLMs keep improving, we're going to reach the last generation that knew how software worked.
- zerbinxx 3y ago
  We’re pretty close. I’m not sure that 51% of the people I work with understand what DNS is, what a call stack is, what the difference between inheritance and polymorphism is, or what a mutex is
nonrandomstring 3y ago
When I'm retired, sitting on the beach with my beer and a good book, please don't come bothing me that your smartphone banking and GPT arse-wiping assistant has gone berserk. You're on your own matey. But hey, you'll have Google.

kazinator 3y ago

Would you also say that, "as an interviewer, if a candidate can use their buddy to give a better answer than other candidates, I'm not going to mark them down for using their buddy"? Even if you don't mind that situation, shouldn't you get buddy's contact information and offer him the job?

smcin 3y ago
That's not a great analogy: you can't do the job with your buddy; whereas some interviewers are ok with, and even expect you to, use GenAI on the job daily. Depends on the interviewer and job expectations. A better analogy is an interview where you can use a calculator (and not be detected). If the interviewer were only to ask you simple arithmetic questions with numeric answers then sure you'd seem to do well. So interviewers adjust to not doing that.
- toast0 3y ago
  > you can't do the job with your buddy I mean, why not? Me and my buddy are a team, hire us both or none of us. Split the salary if you must.
  - TeMPOraL 3y ago
    That's called contracting out and it's fine in this way - and so would be use of GPT, in that way and not to beat the interview.
  - kazinator 3y ago
    Sure, and also split the dental and other benefits, vacations, and share one building fob, parking pass, cubicle and computer. Also, split the food at the company dinner. :)

thegrim33 3y ago

To use a slightly more extreme example .. if you were hiring someone to maintain a nuclear power plant, and when you asked them a question about what actions to take to avoid a meltdown, and they had to ask ChatGPT to figure it out, would you really be OK with hiring that person to maintain your nuclear plant? When they don't actually have the knowledge they need to succeed, but instead have to rely on external tools to decide things? If they need to ask ChatGPT for the answer, how do they know if the answer is right? You really think that person, who relies on tools, is just as good of a hire as someone that fully internally knows what they need to know? Yeah, hiring someone to code a website isn't the same as maintaining a nuclear plant, but it's the same concept of someone that knows their craft vs. someone that needs to rely on tools. There's a major difference in my mind.

fragmede 3y ago
in terms of tools, I absolutely want the nuclear power plant engineer to use a wrench and pliars and tongs and a forklift and a machine while wearing a lead lined safety suit instead of wandering over to the reactor in a t-shirt to pull out the control rods with their bare hands. You could be Edward Teller and know everything there is to know about nuclear physics but you're not getting anywhere without tools. to your point though, a person needs both. all of one and none of the other is useless. You don't want someone who doesn't know what they're doing to play around disabling safety systems so you don't get Chernobyl, but for the everyday crud website you can just hire the coding monkey at a reduced cost.
hackit2 3y ago
I hope your statement is hyperbolic because we're all doomed if you expect a person to know how to operate a nuclear power plant. Normally, your testing if they can follow operational procedure that were created by people who designed the power plant in the first place. Similar it is unreasonable and bordering on negligence to assume a person has the skill set unique to your situation.
- anileated 3y ago
  Like a university diploma is a signal of being able to learn or at least comply, use of a chatbot is a signal of not bothering enough to learn or comply. I can see how an applicant who cheats interview with chatbot would later not bother to internalize operation instructions for the job.

wakawaka28 3y ago

>As an interviewer, if a candidate can use chatGPT to give a better answer than other candidates, I'm not gonna mark them down for use of chatGPT. I think that makes you an incompetent interviewer, unless your questions are too hard for ChatGPT. In any case, solving the question without ChatGPT is more impressive than using it. Just like most other tools, like search engines or IDEs.