8 ms·

My company, a very very large company, is transitioning back to only in-person interviews due to the rampant amount of cheating happening during interviews. As

• wccrawford 2y ago
  When I was interviewing entry level programmers at my last job, we gave them an assignment that should only take a few hours, but we basically didn't care about the code at all. Instead, we were looking to see if they followed instructions, and if they left anything out. I never had a chance to test it out, since we hadn't hired anyone new in so long, but ChatGPT/etc would almost always fail this exam because of how bad it is at making sure everything was included. And bad programmers also failed it. It always left us with a few candidates that paid attention, and from there we figure if they can do that, they can learn the rest. It seemed to work quite well. I was recently laid off from that company, and now I'm realizing that I really want to see what current-day candidates would turn in. Oh well.
  • z3t4 2y ago
    For those tests I never follow the rules, I just make something quick and dirty because I refuse to spend unpaid hours. In the interview the first question is why I didnt follow the instructions, and they think my reason is fair. Companies seem to think that we program just for fun and ask to make a full blown app... also underestimating the time candidates actually spend making it.
    • MathCodeLove 2y ago
      If you’re spending the time applying and submitting something then you might as well spend the extra 30 minutes or so to do it right, no?
      • Aeolun 2y ago
        Any time someone says ‘should only take a few hours’ they’re far underestimating the time it actually takes.
      • halfcat 2y ago
        Not if you’re applying to hundreds, or thousands of jobs. Unless you know someone, it’s a quantity game.
        • dahart 2y ago
          I’ve screened a lot of resumes and given a lot of interviews over the years, and it’s usually obvious when people are trying the scattershot approach, they just don’t match. I feel like treating it like a quantity game is unlikely to improve your odds, and tbh spamming out hundreds or thousands of applications sounds like a miserable way to spend time. You could spend that time meeting and talking to people. I’ve never applied to more than 2 jobs at once, jobs that I actually want, and never had trouble getting at least one of them (and it still takes time and effort and some coding and interviews).
          • nsxwolf 2y ago
            It wouldn’t be obvious they’re using a scattershot approach when they’re a good match, though. I don’t see the downside.
            • dahart 2y ago
              Maybe not at the resume screening phase, but it’s usually still obvious once the interviews start when people aren’t interested in your specific company. Some people get lucky, sure, but the downside is that you have to get lucky, it’s wasting valuable time on low probability events. If you’re familiar with the statistical process of importance sampling, in my experience on both sides of the interview table, it’s effective and worthwhile to spend more time curating higher quality samples than to scatter and hope.
• emseetech 2y ago
  The industry (all industries really) might want to reconsider online applications, or at least privilege in-person resume drop-offs because the escalating ai application/evaluation war that's happening doesn't seem to be helping anyone.
  • datavirtue 2y ago
    No it's because AI shifted power over to the applicant.
    • emseetech 2y ago
      How so? Tons of companies are moving to AI automated intake systems because they're getting flooded with low-quality AI generated resumes. Of course, the original online applications systems were terrible already which is what encouraged people towards low effort in their applications so it's become a stale-mate.
    • bdangubic 2y ago
      this is very strange statement. in what world did AI possibly shift power to the applicant?? applicants have almost never been in shittier position than they are now and things are getting much, much worse by the day
      • ethin 2y ago
        Yeah I don't get this either. I've been looking for a job for like 3-4 years, even an entry-level one, since I graduated college in May of 22 and I still haven't found one. I'm probably doing something wrong (and that's a different discussion), but it's getting harder and harder to know if it's me or the AI applicants or the AI ATS system. And then we have the AI job seekers which are AI-created accounts trying to find employment -- I've already started to see a few of these pop up on Linked In. They were banned, but still, the fact it's happening all is a bit worrying if not predictable.
      • datavirtue 2y ago
        The applicant can use AI to build CVs, tailor them and submit more, faster. Negating a lot of the automated algorithms that were being used to filter (torture) applicants.
• mr_00ff00 2y ago
  So depressing to hear that “because of rampant cheating” As a person looking for a job, I’m really not sure what to do. If people are lying on their resumes and cheating in interviews, it feels like there’s nothing I can do except do the same. Otherwise I’ll remain jobless. But to this day I haven’t done either.
  • ghaff 2y ago
    Note also "And the majority of the time the answer is incorrect anyway." I haven't looked for development-related jobs this millennium, but it's unclear to me how effective a crutch AI is for interviews--at least for well-designed and run interviews. Maybe in some narrow domains for junior people. As a few of us have written elsewhere, I consider not having in-person interviews past an initial screen sheer laziness and companies generally deserve whoever they end up with.
  • ryandvm 2y ago
    I don't know, I kind of feel like leetcode interviews are a situation where the employer is cheating. I mean, you're admittedly filtering out a great number of acceptable candidates knowing that if you just find 1 in a 1000, that'll be good enough. It is patently unfair to the individuals that are smart enough to do your work, but poor at some farcical representation of the work. That is cheating. In my opinion, if a prospective employee is able to successfully use AI to trick me into hiring them, then that is a hell of a lot closer to the actual work they'll be hired to do (compared to leetcode). I say, if you can cheat at an interview with AI, do it.
    • kortilla 2y ago
      The employer sets the terms of the interview. If you don’t like them, don’t apply. What you’re suggesting here isn’t any different than submitting a fraudulent resume because you disagree with the required qualifications.
      • Apocryphon 2y ago
        On principle, what you’re saying has merit. In practice, the market is currently rife with employers submitting job postings with inflated qualifications, for positions that may or may not exist. So there’s bad actors all around and it’s difficult to tell who actually is behaving with integrity.
• wsintra2022 2y ago
  Is it cheating if I can solve the problem using the tools of AI, or is it just solving the problem?
  • risyachka 2y ago
    I guess its more of a question if you can solve the problem without AI. In most interview tasks you are not solving the task “with” ai. Its AI who solves the task while you watch it do it.
  • isbvhodnvemrwvn 2y ago
    For the goal of the interview - showing your knowledge and skills - you are failing miserably. People know what LLMs can do, the interview is about you.
  • twoparachute45 2y ago
    If you've been given the problem of "without using AI, answer this question", and you use an AI, you haven't solved the problem. The ultimate question that an interview is trying to answer is not "can this person solve this equation I gave them?", it's usually something along the lines of "does this person exhibit characteristics of a trustworthy and effective employee?". Using AI when you've been asked not to is an automatic failure of trust. This isn't new or unique to AI, either. Before AI people would sometimes try to look up answers on Google. People will write research papers by looking up information on Wikipedia. And none of those things are wrong, as long as they're done honestly and up front.
  • dahart 2y ago
    Interviews aren’t about solving problems. The interviewer isn’t interested in a problem’s solution, they’re interested in seeing how you get to the answer. They’re about trying to find out if you’ll be a good hire, which notably includes whether you’re willing and interested in spending effort learning. They already know how to use AI, they don’t need you for that. They want to know that you’ll contribute to the team. Wanting to use AI probably sends the wrong message, and is more likely to get you left out of the next round of interviews than it is to get you called back. Imagine you need to hire some people, and think about what you’d want. That’ll answer your question. Do you want people who don’t know but think AI will solve the problems, or do you want people who are capable of thinking through it and coming up with new solutions, or of knowing when and why the AI answer won’t work?
• paxys 2y ago
  > it's wild to me how many candidates think they can get away with it Remember that you are only catching the candidates who are bad at cheating.
  • johnnyanmac 2y ago
    That's fine. The ones who are "good cheaters" are probably smarter than many honest people. Think about those school days where your smartest peers were cheating anyway, despite teaching you organically earlier on. Those kinds of cheaters do it to turn an A into an A+, not because they don't understand the material.
• guywithahat 2y ago
  I mean they could be googling things; I’ve definitely googled stuff during an interview. I do think in-person interviews area important though, I did some remote final interviews with Amazon and they were all terrible
• IG_Semmelweiss 2y ago
  I'm using AI for interview screeners for nontechnical roles that require knowledge work. The AI interviewing app is very very basic, its just a wrapper put together by an eng, with enough features to prevent cheating. Start with recording the session and blocking right-click, and you are halfway there. Its not hard. The AI app has helped me surface top candidates. I don't even look at resumes anymore. There's no point. I interview the top 10 out of 200, and then do my references and select.
  • zx8080 2y ago
    If there's no independent verification, how do you know it's really top 10? (not middle 10, or random 10?)
    • IG_Semmelweiss 2y ago
      because there's scores, and the interview is highly customized and has an answer key that has been tested beforehand by the test creators.
• cuuupid 2y ago
  So far 3 of the 11 people we interviewed have been clearly using ChatGPT for the >>behavioral<< part of the interview (like, just chatting about background, answering questions about their experience). I find that absolutely insane, if you cannot hold a basic conversation about your life without using AI then something is terribly wrong. We actually allow using AI in our in-person technical interviews, but our questions are worded to fail safety checks. We'll talk about smuggling nuclear weapons, violent uprising, staging a coup, manufacturing fentanyl, etc. (within the context of system design) and that gives us really good mileage on weeding out those who are just transcribing what we say into AI and reading the response.
  • xarope 2y ago
    what actually happens to the interviewee? Do they suddenly go blank when they realise the LLM has replied "I'm sorry I cannot assist you with this", or they try to make something up?
    • cuuupid 2y ago
      Yeah pretty much, they either go silent for 2-3 minutes or leave the call and claim their internet has cut out and need to reschedule. Just one time someone got mad and yelled at the interviewer about nothing specific, just stuff like I’m not who you are looking for, you will never find anybody to hire.
  • dmazzoni 2y ago
    Ha ha, that's a great idea! I love the idea of embedding sensitive topics that ChatGPT and other LLMs will steer clear of, within the context of a coding question. Have you ever had any candidate laugh? Any candidates find it offensive?
    • cuuupid 2y ago
      We usually get laughs, some quick jokes, etc., some really involved candidates will ask if it’s worded that way to prevent using ChatGPT. No one’s found it offensive, the prompt is mostly neutral just very “dangerous activity” coded.
• hibikir 2y ago
  Some can be quite good at the cheating: At least good enough to get through multiple layers. I've been in hiring meetings where I was the only one of 4 rounds that caught the cheating, and they were even cheating in behaviorals. I've also been in situations with a second interviewer, where the other interviewer was completely oblivious even when it was clear I was basically toying with the guy reading from the AI, leading conversation in unnatural ways. Detection of AI in remote interviews, behavioral and technical, just has to be taught today if you are ever interviewing people that don't come from in-network recommendations. Completely fake candidates are way too common.
• topkai22 2y ago
  I haven’t been doing that much interviewing, but in the dozen or so candidates I’ve had I don’t think a single one has tried to use AI. I almost wish they would, as then at least I’d get past the first half of the question…
• aprilthird2021 2y ago
  I'm at the same company I think. I don't get why we can't just use some software that monitors clicking away or tabbing away from the window, and just tell candidates explicitly that we are monitoring them, and looking away or tabbing away will appear suspect.
• md-ayaz-me 2y ago
  I second this. My previous company started in-person interview for this very reason.