3 ms·

I hold remote interviews and I can tell when candidates use AI to answer questions, real time with camera on. They repeat my technical question, pause for a fe

I hold remote interviews and I can tell when candidates use AI to answer questions, real time with camera on. They repeat my technical question, pause for a few seconds, their voice drops to a monotone and they quickly recite a bulleted list of low level technical details that sounds like a wikipedia page. I worry that candidates will learn to act more subtle, maybe configure their LLM to return with an anecdote around the tech in question, and practice "selling" their vocal communication.

dvh 1y ago
Students cheat when grades are more valuable than knowledge.
- Biologist123 1y ago
  Perfect.
- shinycode 1y ago
  Exactly, the problem is not the students but the amount of efforts the education system has to pour in to change and make it better (and take advantage of LLMs as a learning opportunity )
- jcranmer 1y ago
  Students cheat when they think grades are more valuable than knowledge. We already have a real-world scourge of tech bros who view learning about other subjects as beneath them and who rate their ignorance as more valuable than experts in that subject. This wonderful new world of AI is only going to exacerbate that problem.
- esafak 1y ago
  He's talking about graduates. There are people out there trying to hold down multiple remote jobs by cheating through interviews and collecting as many paychecks as possible until they get caught.
- spacemadness 1y ago
  I would say most technical interviewers cheat as they ask questions they themselves couldn’t solve on the spot.
- const_cast 1y ago
  Grades are how we try to measure knowledge, because knowledge is abstract and hard to measure. Obviously, the measurement of X runs the risk of being optimized as opposed to X - I see this in my job all the time, and we're all professionals. Point is, it's not a problem with academia and it's not easy to solve. If we could just magically peer into people's minds and figure out what they know definitively then we'd solve a lot of world issues. Alas, we must settle for numbers and papers instead.
Dracophoenix 1y ago
> They repeat my technical question, pause for a few seconds, their voice drops to a monotone and they quickly recite a bulleted list of low level technical details that sounds like a wikipedia page. Can you provide an example?
- runamuck 1y ago
  Me: "Tell me about your favorite tech stack." Them: "(Pause, eyes scan camera, monotone voice) There are many alternatives for a tech stack, each with their positives and negatives. Let's take a look at some of the more popular options..."
- esafak 1y ago
  https://koliber.com/articles/how-to-avoid-hiring-a-north-korean-spy-on-your-dev-team https://koliber.com/articles/how-to-avoid-hiring-a-north-kor... I encountered this imposter too.
  - akudha 1y ago
    So the very first recommendation is “check their LinkedIn”? What if I, as a candidate, don’t want to be on LinkedIn or any other social media? I understand companies want to protect themselves. They need to understand some people just don’t want to be on social media (HN aside). In other words, there needs to be a middle ground. Job hunting is already an arduous, demoralizing task. Forcing people to be on social media for a just another layer of stress
    - esafak 1y ago
      A work- and education history that can be validated helps. Put yourself in the shoes of a hiring manager. How would you identify imposters?
jeffbee 1y ago
Not even willing to do remote interviews any more.
jerf 1y ago
One of the things I would say I saw coming from a long ways away is that a lot of effort was put into detecting the "default" voice of LLMs, but people would eventually figure out how to kick them out of it, since it is after literally just a matter of asking. It took some months but word does seem to be finally getting out, I'm seeing the idea emerge in more and more places. I gave it the prompt "Suppose you are in a job interview for a front-end web position and someone asks you about how you use the React library and the hardest problem you even had to solve with it. How might you react, along with a somewhat amusing anecdote?"[1] and it did pretty well. I think I'd play with it a bit to see if I can still suppress some of the LLM-isms that came out, but a human could edit them out in real-time with just a bit of practice too... it's not like you can just read it to your interviewer, you will need to Drama Class 101 this up a bit anyhow. It'll be easier to improv a bit over this than a bare Wikipedia list. In other words, as with the question the article title asks, the question isn't about what happens "when" this starts being possible... the capability has run ahead of all but the most fervent AI user's understanding and it is already here. It's just a matter of the word-of-mouth getting around as to how to prompt the AIs to be less obvious. I also anticipate that in the next couple of years, the AI companies will be getting tired of people complaining about the "default LLM voice" and it'll shift to be something less obvious than it is now. Both remote interviews and college writing are really already destroyed, the news just hasn't gotten around to everybody yet. (In fact I suspect that "default LLM voice" will eventually become a sort of cultural touchstone of 2024-2026 and be deliberately used in future cultural references to set stories in this time period. It's a transient quality of current-day LLMs, easy to get them out of even today, and I expect future LLMs to have much different "default voices".) [1]: And in keeping with my own philosophy of "there's not a lot of value of just pasting in LLM responses" if you want to see what comes out you are welcome to play with it yourself. No huge surprises though. It did the job.
- tempodox 1y ago
  The cynic in me says there's no problem with that. Either their jobs will be just as LLM-heavy and prompt-jockeying was the right skill to learn, or they won't be able to hold any job because they didn't learn anything relevant, which would be just payment for the cheating.
msgodel 1y ago
Most of this stuff isn't that hard to learn and apply. If they put that much effort into setting up an LLM cheating tool they could have just built something with whatever tool you're talking about.

dvh 1y ago

Students cheat when grades are more valuable than knowledge.

Biologist123 1y ago
Perfect.
shinycode 1y ago
Exactly, the problem is not the students but the amount of efforts the education system has to pour in to change and make it better (and take advantage of LLMs as a learning opportunity )
jcranmer 1y ago
Students cheat when they think grades are more valuable than knowledge. We already have a real-world scourge of tech bros who view learning about other subjects as beneath them and who rate their ignorance as more valuable than experts in that subject. This wonderful new world of AI is only going to exacerbate that problem.
esafak 1y ago
He's talking about graduates. There are people out there trying to hold down multiple remote jobs by cheating through interviews and collecting as many paychecks as possible until they get caught.
spacemadness 1y ago
I would say most technical interviewers cheat as they ask questions they themselves couldn’t solve on the spot.
const_cast 1y ago
Grades are how we try to measure knowledge, because knowledge is abstract and hard to measure. Obviously, the measurement of X runs the risk of being optimized as opposed to X - I see this in my job all the time, and we're all professionals. Point is, it's not a problem with academia and it's not easy to solve. If we could just magically peer into people's minds and figure out what they know definitively then we'd solve a lot of world issues. Alas, we must settle for numbers and papers instead.

Dracophoenix 1y ago

> They repeat my technical question, pause for a few seconds, their voice drops to a monotone and they quickly recite a bulleted list of low level technical details that sounds like a wikipedia page. Can you provide an example?

runamuck 1y ago
Me: "Tell me about your favorite tech stack." Them: "(Pause, eyes scan camera, monotone voice) There are many alternatives for a tech stack, each with their positives and negatives. Let's take a look at some of the more popular options..."
esafak 1y ago
https://koliber.com/articles/how-to-avoid-hiring-a-north-korean-spy-on-your-dev-team https://koliber.com/articles/how-to-avoid-hiring-a-north-kor... I encountered this imposter too.
- akudha 1y ago
  So the very first recommendation is “check their LinkedIn”? What if I, as a candidate, don’t want to be on LinkedIn or any other social media? I understand companies want to protect themselves. They need to understand some people just don’t want to be on social media (HN aside). In other words, there needs to be a middle ground. Job hunting is already an arduous, demoralizing task. Forcing people to be on social media for a just another layer of stress
  - esafak 1y ago
    A work- and education history that can be validated helps. Put yourself in the shoes of a hiring manager. How would you identify imposters?

jeffbee 1y ago

Not even willing to do remote interviews any more.

jerf 1y ago

One of the things I would say I saw coming from a long ways away is that a lot of effort was put into detecting the "default" voice of LLMs, but people would eventually figure out how to kick them out of it, since it is after literally just a matter of asking. It took some months but word does seem to be finally getting out, I'm seeing the idea emerge in more and more places. I gave it the prompt "Suppose you are in a job interview for a front-end web position and someone asks you about how you use the React library and the hardest problem you even had to solve with it. How might you react, along with a somewhat amusing anecdote?"[1] and it did pretty well. I think I'd play with it a bit to see if I can still suppress some of the LLM-isms that came out, but a human could edit them out in real-time with just a bit of practice too... it's not like you can just read it to your interviewer, you will need to Drama Class 101 this up a bit anyhow. It'll be easier to improv a bit over this than a bare Wikipedia list. In other words, as with the question the article title asks, the question isn't about what happens "when" this starts being possible... the capability has run ahead of all but the most fervent AI user's understanding and it is already here. It's just a matter of the word-of-mouth getting around as to how to prompt the AIs to be less obvious. I also anticipate that in the next couple of years, the AI companies will be getting tired of people complaining about the "default LLM voice" and it'll shift to be something less obvious than it is now. Both remote interviews and college writing are really already destroyed, the news just hasn't gotten around to everybody yet. (In fact I suspect that "default LLM voice" will eventually become a sort of cultural touchstone of 2024-2026 and be deliberately used in future cultural references to set stories in this time period. It's a transient quality of current-day LLMs, easy to get them out of even today, and I expect future LLMs to have much different "default voices".) [1]: And in keeping with my own philosophy of "there's not a lot of value of just pasting in LLM responses" if you want to see what comes out you are welcome to play with it yourself. No huge surprises though. It did the job.

tempodox 1y ago
The cynic in me says there's no problem with that. Either their jobs will be just as LLM-heavy and prompt-jockeying was the right skill to learn, or they won't be able to hold any job because they didn't learn anything relevant, which would be just payment for the cheating.

msgodel 1y ago

Most of this stuff isn't that hard to learn and apply. If they put that much effort into setting up an LLM cheating tool they could have just built something with whatever tool you're talking about.