42 ms·

Ask HN: What is interviewing like now with everyone using AI?

• trustinmenowpls 2y ago
  I've been on both sides recently, and it hasn't really changed significantly. If you're heming and hawing you're not getting the job.
• iExploder 2y ago
  Name of the game now is not to get fired at all costs and weather the storm until the dust settles...
  • jppope 2y ago
    hiring market tightened up... that doesn't mean there isn't one
    • iExploder 2y ago
      > hiring market tightened up... that doesn't mean there isn't one tightened market is one thing, the absolute insanity of the recruitment process in last couple years with now AI thrown into the mix is really something to behold, test these waters at your own peril
      • blazing234 2y ago
        yeah the name of the game now is just to avoid any company that has shitty recruitment. you can tell in an instant if they are worth your time or not, which I'd say in Canada is about 90% a waste of time. someone who actually wants to hire will want you and wants to do whatever they can to get a good candidate.
        • jppope 2y ago
          Realistically its just the blind leading the blind. People have forgotten that a interview process was designed to avoid false positives, and that the companies who were most selective were providing top 1% comp and had brands that could carry that weight. If you are google and you were handing $500K in RSUs on top of $300K+ in salary, you better damn pick the right candidate... For some random SMB in shipping or something to be bashing people over the head with 10 step-leetcode-full-panel-10-hour-systems-design interviews they just don't get it. For starters they probably don't even have the talent to properly evaluate the prospect. So who are they helping?
    • johnnyanmac 2y ago
      nope, but the process is worse - Much less resposnes - For your sanity you want to make sure there aren't obvious signs of something being a ghost job, an H1B hire, an internal hire, or a farce because "we're always growing" (lying). - expect longer processes. Hasn't happened to me but 7+ stage interview processes is not uncommon these days, even outside of tech - accept that some processes will be frozen under your nose. especially because of longer processes crossing quarters - expect less respect in the process. They feel like they do not want you. You are expendable - don't bother negotiating in this market. You get a number and take it unless you already have a job. Even then they may simply pass you for someone more desperate. BTW wages are being suppressed; you're probably not getting pre 2023 salaries right now. Yeah... if you're not being abused at work, I'd just weather it out.
      • ElectroBuffoon 2y ago
        > wages are being suppressed "logically, necessarily" as Marc Andreessen says? https://futurism.com/the-byte/ai-investor-goal-crash-human-wages https://futurism.com/the-byte/ai-investor-goal-crash-human-w... Now if they could also crash factors like material, energy and profit... consumers could pay pennies using their centinaire incomes.
  • johnnyanmac 2y ago
    I sadly failed long ago. The company itself did not weather the storm.
• crazygringo 2y ago
  Not sure why interviews would change. Even if you're using ChatGPT heavily it's your job to ensure it's right. And you need to know what to ask it. So you still need all the same skills and conceptual understanding as before. I mean, I didn't observe interviews change after powerful IDE's replaced basic text editors.
  • fragmede 2y ago
    Because there's a format of interview that's basically a brainteaser that takes 45 minutes to think through and whiteboard some code for, but which is trivially solvable by copy and pasting a screenshot of the prompt into ChatGPT. This amounts to candidates being given the answer and then pretending to struggle with understanding your question and then figuring out a solution to it when really they're just stalling for time and then just copying the answer from one browser tab to the next.
    • Barrin92 2y ago
      If you're doing this at least face to face over Zoom and you can't tell that someone is copying answers from their second monitor and throwing ChatGPT explanations at you, you honestly need a better interviewer. I've done a lot of interviews over Zoom, and whenever someone cheats, by passing someone else's work off as their own (the weirdest thing I've ever encountered was someone having a friend on trying to feed them answers, which he admitted to later) it is so painfully obvious if you grill them a bit and throw a few curveballs.
      • fragmede 2y ago
        If you believe your catch rate is 100%, just because you've managed to identify a couple of people who were really bad at it, you might want to check your priors.
        • Barrin92 2y ago
          I didn't say my catch rate is 100%. But catching the people who are bad at it is the entire point. I don't care about not catching someone proficient enough they can hold up even in a direct conversation where they have to explain what they did. That's someone who is likely just insecure but smart enough. They still shouldn't cheat, but they're at least clever. I had an old teacher who used to joke that you're allowed to cheat during the exam but you can only bring a small note card, point being that if you put enough effort into preparing your cheating that it fit on the card you were good enough to just take the actual test. It's the people who have no idea what the hell they're doing and who try to pass of a solution they can't explain and by definition those people you will catch if you know how to interview.
• vunderba 2y ago
  I mentioned this in a different related post but there seems to be a pretty sad lack of basic integrity in the tech world where it's become a point of pride to develop and use apps which allow an applicant to blatantly cheat during interviews using LLMs. As a result, several of my friends who assist in hiring at their companies have already returned to "on-site" interviews. The funny thing about this is that these are 100% remote jobs - but the interviews are being conducted at shared workspaces. This is what happens when the level of trust goes down to zero due to bad actors.
  • ghaff 2y ago
    Largely past COVID it seems like sheer laziness or cheapness not to conduct in-person interviews for a professional job other than a short-term project after essentially an initial screen for all sorts of reasons that have little to do with cheating. I don’t care if the job is largely remote. As someone else noted, this used to be utterly standard. And frankly I’d probably just pass on someone who balked. Plenty of fish in the sea.
  • johnnyanmac 2y ago
    The crushingly long and disrespectful interview process cut the bridge off first. If the job market becomes a problem of scale, people (especially programmers) will scale up as such. As the interviewers have with ATC.
• vrosas 2y ago
  As someone currently job searching it hasn’t changed much, besides companies adding DO NOT USE AI warnings before every section. Even Anthropic forces you to write a little “why do you want to work here DO NOT USE AI” paragraph. The irony.
  • Pooge 2y ago
    They will very happily use AI to evaluate your profile, though :)
    • pizzalife 2y ago
      Applying at Anthropic was a bad experience for me. I was invited to do a timed set of leetcode exercises on some website. I didn't feel like doing that, and focused on my other applications. Then they emailed me a month later after my "invitation" expired. It looked like it was written by a human: "Hey, we're really interested in your profile, here's a new invite link, please complete this automated pre-screen thingie". So I swallowed my pride and went through with that humiliating exercise. Ended up spending two hours doing algorithmic leetcode problems. This was for a product security position. Maybe we could have talked about vulnerabilities that I have found instead. I was too slow to solve them and received some canned response.
    • x0x0 2y ago
      fyi, that's because (from experience) the last job req I publicly posted generated almost 450 responses, and (quite generously) over a third were simply not relevant. It was for a full-stack rails eng. Here, I'm not even including people whose experience was django or even React; I mean people with no web experience at all, or were not in the time zone requested. Another 20% or so were nowhere near the experience level (senior) requested either. The price of people bulk applying with no thought is I have to bulk filter.
      • Pooge 2y ago
        So you allow yourself to use AI in order to save time, but we have to put up with the shit[1] companies make up? That's good, it's for the best if I don't work for a company that thinks so lowly of its potential candidates. [1]: Including but not limited to: having to manually fill a web form because the system couldn't correctly parse a CV; take-home coding challenges; studying for LeetCode interviews; sending a perfectly worded, boot-licking cover letter.
• explorigin 2y ago
  Part of my resume review process is trying to decide if I can trust the person. If their resume seems too AI-generated, I feel less like I can trust that candidate and typically reject the candidate. Once you get to the interview process, it's very clear if someone thinks they can use AI to help with the interview process. I'm not going to sit here while you type my question into OpenAI and try to BS a meaningful response to my question 30 seconds later. AI-proof interviewing is easy if you know what you're talking about. Look at the candidates resume and ask them to describe some of their past projects. If they can have a meaningful conversation without delays, you can probably trust their resume. It's easy to spot BS whether AI is behind it or not.
  • ktallett 2y ago
    This, and tbh this has always been the best way. Someone who has projects, whether personal or professional, and has the capability to discuss those projects in depth and with passion will usually be a better employee than a leet code specialist.
    • remus 2y ago
      Doesn't even have to be a project per se, if they can discuss some sort of technical topic in depth (i.e. the sort of discussion you might have when discussing potential solutions to a problem) then that's a great sign imo.
    • polishdude20 2y ago
      My resume has a bunch of personal projects on there as well as work experience and the project experience seems to not help at all. Just rejections after rejections.
      • ktallett 2y ago
        My suggestion was in an ideal world which sadly this isn't. Your issue suggests they aren't tailored for each application, which could potentially be a reason. It is better to show why one project makes you a great fit as opposed to how many projects you have done. Sometimes the person in charge of hiring may not fully have all the expertise in the area they are hiring for.
• roland35 2y ago
  There are some tools that read your screen and can provide hints and solutions for coding type questions. I honestly don't trust myself to not mess it up, plus the whole ethics side of it, but I'm sure that will always be a problem for online assessments
• ktallett 2y ago
  The key is having interviewers that know what they are talking about so in-depth meandering discussions can be had regarding personal and work projects which usually makes it clear whether the applicant knows what they are talking about. Leetcode was only ever a temporary interview technique, and this 'AI' prominence in the public domain has simply sped up it's demise.
  • danielbln 2y ago
    This is the way. We do an intro call, an engineering chat (exactly as you describe), a coding challenge and 2 team chat sessions in person. At the end of that, we usually have a good feeling about how sharp the candidate is, of they like to learn and discover new things, what their work ethic is. It's not bullet proof, but it removes a lot of noise from the signal. The coding challenge is supposed to be solved with AI. We can no longer afford not to use LLMs for engineering, as it's that much of a productivity boost when used right, so candidates should show how they use LLMs. They need to be able to explain the code of course, and answer questions about it, but for us it's a negative mark of a candidate proclaims that they don't use LLMs.
    • satvikpendem 2y ago
      > The coding challenge is supposed to be solved with AI. We can no longer afford not to use LLMs for engineering, as it's that much of a productivity boost when used right, so candidates should show how they use LLMs. They need to be able to explain the code of course, and answer questions about it, but for us it's a negative mark of a candidate proclaims that they don't use LLMs. Do you state this upfront or is it some hidden requirement? Generally I'd expect an interview coding exercise to not be done with AI, but if it's a hidden requirement that the interviewer does not disclose, it is unfair to be penalized for not reading their minds.
      • ktallett 2y ago
        I would say as long as it is stated you can complete the coding exercise using any tool available it is fine. I do agree, no task should be a trick. I am personally of the view you should be able to use search engines, AI, anything you want, as the task should be representative of doing the task in person. The key focus has to be the programmer's knowledge and why they did what they did.
• MacsHeadroom 2y ago
  Changed enormously. Both resumes and interviews are effectively useless now. If our AI agents can't find a portfolio of original work nearly exactly what we want to hire you for then you aren't ever going to hear from us. If you are one of the 1 in 4000 applications who gets an interview then you're already 70% likely to get an offer and the interview is mostly a formality.
  • fifilura 2y ago
    Does that mean you will not hire anyone without a public portfolio?
    • Pooge 2y ago
      I thought that meant what you typically write in the "Experience" section. GP, am I wrong? Is everyone writing a "Projects" section by rewording what they wrote in "Experience"?! For me, "Projects" should strictly be personal projects. If not, maybe that's what I'm missing.
      • sshine 2y ago
        Projects are personal projects, or at least projects in which you did a distinguishable effort. They don't have to be public to the whole world, you can have links that are only in your resume. But if they're on GitHub, they have to be public, since there aren't unlisted repositories.
        • mdaniel 2y ago
          I actually believe that it would be possible to provide a read only clone url in a resume link but I don't know if a way to make a link to a browsable version (short of having a proxy server type setup, or, of course, a slim server protected by http basic)
        • Dwedit 2y ago
          Github does allow private repositories.
• mococa 2y ago
  I recently reviewed a medium-complexity assignment—just questions, no coding—and out of six candidates, I only approved one. The others were disqualified because their answers were filled with easily identifiable ChatGPT-generated fluff. And I had made it clear that they should use their own words.
• ryan-duve 2y ago
  My startup got acquired last year so I haven't interviewed anyone in a while, but my technical interview has always been: - share your screen - download/open the coding challenge - you can use any website, Stack Overflow, whatever, to answer my questions as long as it's on the screenshare My goal is to determine if the candidate can be technically productive, so I allow any programming language, IDE, autocompleter, etc, that they want. I would have no problem with them using GPT/Copilot in addition to all that, as long as it's clear how they're solving it.
  • silasdavis 2y ago
    I don't care how you're good at it so long as I can watch.
  • b112 2y ago
    I'd be fine with the GPT side of things, as long as I could somehow inject poor answers, and see if the interviewee notices and corrects.
    • cpursley 2y ago
      That's actually a horribly awesome idea.
    • htrp 2y ago
      the trick is to phrase the problem in a way that GPT4 will always give the incorrect answer (due to vagueness of your problem) and that multiple rounds of guiding/correcting are needed to solve.
      • gtirloni 2y ago
        That's pretty good because it can exhaust the context window quickly and then it starts spiraling out of control, which would require the candidate to act.
        • htrp 2y ago
• elzbardico 2y ago
  I have a colleague that uses AI to comment on RFCs. It is so clearly machine generated, that I wonder if I am the only one to see it. He is a good colleague though, but as he is a bit junior, it is still not clear to me if AI is helping him to improve faster or if it is hindering his deep learning of stuff.
• gibbonsrcool 2y ago
  Panel interviews seem to be more common. Curious if others have seen the same? I personally feel very uncomfortable coding in front of a group. First one of these I tried had like 5 people watching and I lost my nerve and bailed. :|
  • RevEng 2y ago
    Panels and live programming assignments are such an awful idea. Is that what the workplace is like? Doo they want people who can work under those conditions? I've been a working professional for 18 years who gives public talks regularly and I can still see myself clamming up in that situation. Everyone knows it's hard to think and type when you are being watched.
    • lubujackson 2y ago
      Worse, how little does the company value developer time? If you get hired there, how many brainnumbing panels will you need to be a part of? Stinks of a lack of focus, trust and quality.
• hot_gril 2y ago
  It's still remote. I don't get how you could pass an interview using ChatGPT unless it's purely leetcode.
• deleted 2y ago
  [deleted]
• jppope 2y ago
  I've been very curious about this and about how we should modify our hiring. Its obvious that an individual should be able to use AI companions to build better, faster, higher quality things... But the skillsets are sooo uneven now that its unfair to those who are with and without. I think it ultimately comes back to impact (like always) which has remained largely unchanged.
• bdcravens 2y ago
  I haven't done any hiring in a while, but my feelings on the matter: If they can talk through the technology and code fluently, honestly, I don't care how they do the work. Honestly I feel like the ability to communicate is a far more important skill than the precise technology. This is of course presumes you have a clue about the technology you're hiring for.
• kittikitti 2y ago
  I tell everyone to share their entire screen, have their video on, and start coding. It's not that different. Even as an interviewer, I experimented with the usual cheating techniques so I know what to look out for. The best are the AI teleprompters. If you can do the work with your own AI then I see no need to care as the business will not care either. The story is completely different for airgapped dark room jobs, but if you know you know.
  • crooked-v 2y ago
    > share their entire screen That would be a bit awkward with my 32:9 primary monitor.
    • mdaniel 2y ago
      I have my dual 4K Dells for day to day work but I unplug one and scale it to something reasonable for interviews because having consideration is part of the signal that I'm sending (and empathy, because I hate trying to read purple text on a black background myself when trying to view other people's setups)
      • johnnyanmac 2y ago
        Nah, I have a triple monitor setup. I'll share my primary screen and work on it. If they can't trust me enough to do that, then we're already on a tumultuous relationship. Not like I'd do anything. I use one screen to work, one for video, and one for work visuals. Seeing one screen would show if I had any hidden windows anyway.
  • blazing234 2y ago
    if your interviews are easily cheatable you should probably fix the interviews. back in the day when i was in university we were allowed to use any resources for exams cause you couldnt cheat easily.
  • locusofself 2y ago
    I'm curious what you mean by this as someone who has worked in a SCIF for years. What I've found (and I personally benefited from this) is that interviews are far more forgiving for people who have security clearances, because of the comparatively tiny available talent pool. AWS, Azure etc allow people to join as "SRE" (a bastardized term at this point) or "Cloud Engineer" and later switch to SWE pretty easily.
• acwan93 2y ago
  I don’t know the answer, but I’d like to share that I asked a simple question about scheduling a phone interview to learn more about a candidate. The candidate’s first response? “Memory updated”. That led to some laughs internally and then a clear rejection email.
  • buggy6257 2y ago
    My first read of this was they made a joke (not wise when scheduling for interviews sure but maybe funny) by intentionally responding that way. This is because my brain couldn't fathom what is likely the reality here -- that someone was just pumping your email thru AI and pumping the response back unedited and unsanitized, and so the first thing you got back was just the first "part" of the AI response. ...Christ.
    • john-radio 2y ago
      Yeah I don't know about this specific situation, but as someone who is on the job market, is a good developer, but can come off as a little odd sometimes, I often wonder how often I roll a natural 1 on my Cha check and get perceived as an AI imposter.
      • acwan93 2y ago
        If anything, coming across as “a little odd” can be a sign I’m actually talking to a human.
        • crooked-v 2y ago
          That's a good point. The major LLMs are all tilted so much towards a weird blend of corpo-speak with third-world underpaid English speaker influence (e.g. "delve", from common Nigerian usage) that having any quirks at all outside that is a good sign.
    • rantallion 2y ago
      I'm with you. Looking at the way people respond online to things now since LLMs and GenAI went mainstream is baffling. So many comments along the lines of "this is AI" when there are more ordinary explanations.
• sshine 2y ago
  I can't speak for job interviewing, but having recently completed 3rd-semester trade-school oral exams in Java programming: It is really important to watch people code. Anyone can fake an abstract overview.
• j-scott 2y ago
  In my most recent cycle, I didn’t ask to use AI and I was only warned once about using AI when I had the official language plugin for an IDE annotate some struct fields with json tags. I explained the plugin functionality and we moved on. When I was part of interviews on the other side for my former employer, I encountered multiple candidates who appeared to be using AI assistance without notifying the interviewers ahead of time or at all.
• NomDePlum 2y ago
  It can be weird. Seen some decent resumes for people that in the actual interview the candidate obviously has zero demonstrable knowledge of. Ask even the shallowest question and they are lost and just start regurgitating what feels like very bad prompt based responses. At that point it's just about closing down the interview without being unprofessional.
• lolinder 2y ago
  The traditional tech interview was always designed to optimize for reliably finding someone who was willing to do what they were told even if it feels like busywork. As a rule someone who has the time and the motivation to brush up on an essentially useless skill in order to pass your job interview will likely fit nicely as a cog in your machine. AI doesn't just change the interviewing game by making it easy to cheat on these interviews, it should be changing your hiring strategy altogether. If you're still thinking in terms of optimizing for cogs, you're missing the boat—unless you're hiring for a very short term gig what you need now is someone with high creative potential and great teamwork skills. And as far as I know there is no reliable template interview for recognizing someone who's good at thinking outside the box and who understands people. You just have to talk to them: talk about their past projects, their past teams, how they learn, how they collaborate. And then you have to get good at understanding what kinds of answers you need for the specific role you're trying to fill, which will likely be different from role to role. The days of the interchangeable cog are over, and with them easy answers for interviewing.
  • nouveaux 2y ago
    Have you spent a lot of time trying to hire people? I guarantee you there is no shadow council trying to figure out how to hire "busywork" worker bees. This perspective smells completely like "If I were in charge, things would be so much better." Guess what? If you were to take your idea and try to lead this change across a 100 people engineering org, there would be "out of the box thinkers" who would go against your ideas and cause dissent. At that point, guess what? You're going to figure out how to hire compliant people who will execute on your strategy. "talk about their past projects, their past teams, how they learn, how they collaborate" You have now excluded amazing engineers who suck at talking about themselves in interviews. They may be great collaborators and communicators, but freeze up selling themselves in an interview.
    • dakiol 2y ago
      My take is: - “big” tech companies like Google, Amazon, Microsoft came up with these types of tech interviews. And there it seems pretty clear that for most of their positions they are looking for cogs - The vast majority of tech companies have just copied what “big” tech is doing, including tech interviews. These companies may not be looking for cogs, but they are using an interview process that’s not suitable for them - Very few companies have their own interview process suitable for them. These are usually small companies and therefore the number of engineers in such companies is negligible to be taken into account (most likely, less than 1% of the audience here work at such companies)
• __loam 2y ago
  There's still plenty of engineers that can't code their way out of a paper bag
  • throwaway123lol 2y ago
    Well they're not really engineers then are they?
    • __loam 2y ago
      Neither are most programmers
• screaminghawk 2y ago
  I don't understand why an interviewer would ban the use of AI if they are allowed to use AI in the role. The interview is a chance to see how a candidate performs in a work like environment. Let them use the tools they will use on the job and see how well they can perform. Even for verbal interviews, if they are using ChatGPT on the side and can manage the conversation satisfactorily then more power to them.
  • mrweasel 2y ago
    What if their lack of knowledge runs so deep that you question if their even able to prompt the AI without step by step instructions? There's nothing wrong with a candidate going "Normally I'd prompt ChatGPT and get a skeleton project going" or saying "Look, I don't run around with the entire standard library in my head. I look that stuff up and sometimes that's with an LLM". The problem is when they can't go through the steps of solving a program, without the AI. I don't care about the details, or if you ask Copilot to do the API query code, because you don't want to write the error handling, that actually fairly reasonable, but if you can't prompt it to add the logic for a HTTP 403 then what's the point? In that case I'd rather hire someone who takes longer, but who knows that the 403 should probably redirect an unauthenticated user to the login page.
• A4ET8a8uTh0_v2 2y ago
  Buddy of mine recently got a position with the help of custom built model that was listening on the call and printed answers on another screen. The arms race is here and frankly, given that a lot of people are already using it at work, there is no way to stop it short of minute upon minute supervision and even biggest micromanagers won't be able to deal with it. Honestly, if I could trust that companies won't try to evaluate my conversation through 20 different ridiculous filters, I would probably argue that my buddy is out of line.. As it stands, however, he is merely leveling out the playing field. But, just life with WFH, management class does not like that imposition one bit.
• sergioisidoro 2y ago
  I've let people use GPT in coding interviews, provided that they show me how they use it. At the end I'm interested in knowing how a person solves a problem, and thinks about it. Do they just accept whatever crap the gpt gives them, can they take a critical approach to it, etc. So far, everyone that elected to use GPT did much worse. They did not know what to ask, how to ask, and did not "collaborate" with the AI. So far my opinion is if you have a good interview process, you can clearly see who are the good candidates with or without ai.
  • alpha_squared 2y ago
    We do the same thing. It's perfectly fine for candidates to use AI-assistive tooling provided that they can edit/maintain the code and not just sit in a prompt the whole time. The heavier a candidate relies on LLMs, the worse they often do. It really comes down to discipline.
    • sureIy 2y ago
      Discipline for what? To me it's the lack of skill. If the LLM spits out junk you should be able to tell. ChatGPT-based interviews could work just as well to determine the ability to understand, review and fix code effectively.
      • skeeter2020 2y ago
        >> If the LLM spits out junk you should be able to tell. Reading existing code and ensuring correctness is way harder than writing it yourself. How would someone who can't do it in the first place tell if it was incorrect?
        • JoshuaDavid 2y ago
          Make the model write annotated tests too, verify that the annotations plausibly could match the test code, run the tests, feed the failures back in, and iterate until all tests are green?
  • 2y ago
• twoparachute45 2y ago
  My company, a very very large company, is transitioning back to only in-person interviews due to the rampant amount of cheating happening during interviews. As an interviewer, it's wild to me how many candidates think they can get away with it, when you can very obviously hear them typing, then watching their eyes move as they read an answer from another screen. And the majority of the time the answer is incorrect anyway. I'm happy that we won't have to waste our time on those candidates anymore.
  • wccrawford 2y ago
    When I was interviewing entry level programmers at my last job, we gave them an assignment that should only take a few hours, but we basically didn't care about the code at all. Instead, we were looking to see if they followed instructions, and if they left anything out. I never had a chance to test it out, since we hadn't hired anyone new in so long, but ChatGPT/etc would almost always fail this exam because of how bad it is at making sure everything was included. And bad programmers also failed it. It always left us with a few candidates that paid attention, and from there we figure if they can do that, they can learn the rest. It seemed to work quite well. I was recently laid off from that company, and now I'm realizing that I really want to see what current-day candidates would turn in. Oh well.
    • z3t4 2y ago
      For those tests I never follow the rules, I just make something quick and dirty because I refuse to spend unpaid hours. In the interview the first question is why I didnt follow the instructions, and they think my reason is fair. Companies seem to think that we program just for fun and ask to make a full blown app... also underestimating the time candidates actually spend making it.
      • MathCodeLove 2y ago
        If you’re spending the time applying and submitting something then you might as well spend the extra 30 minutes or so to do it right, no?
• codr7 2y ago
  About time, testing coding skills that way was always a bad idea.
• Xmd5a 2y ago
  Shouldn't a portfolio of personal projects be enough ? In the past couple years I: - adapted Java's Regex engine to work on streams of characters - wrote a basic parametric geometry engine - wrote a debugger for an async framework - did innovative work with respect to whole-codebase transformation using macros Among other things. As for ChatGPT in the context of an interview, I'd only use it if I were asked to do changes on a codebase I don't know in limited time.
  • ganoushoreilly 2y ago
    I think the arguments is there is no way to validate it was you that did the work. There's been too many instances of groups that do interviews for others or the work for take homes to help get people placed. There was a big deal about some H1Bs a while back where the people that showed up didn't look anything like the people that interviewed. So I understand both sides. It's frustrating though when you've done a lot of work, as you've listed. I think in a good interview maybe going over that code and getting the chance to explain things you did, why you did, or issues you had, could also go a long way. Interviewing is tough, more so at scale.
  • gray_-_wolf 2y ago
    Bit annoying is that when companies ask for a portfolio, they often mean GitHub. Lot of non-technical hiring people I discussed this with were confused by the fact that there are other ways to contribute, like mailing lists.
  • blazing234 2y ago
    most experienced developers don't have a portfolio because they code for a living lol
    • Madmallard 2y ago
      I don't really think this is true? This might be true for developers that only work for companies and never in their own time maybe. My portfolio site is just one of the sites on my personal website that also hosts many of my projects. It wasn't much work to setup, and it provides organization and sharing capability so there's motivation to make it anyways.
• deleted 2y ago
  [deleted]
• deleted 2y ago
  [deleted]
• nimish 2y ago
  If your interview process is susceptible to AI then you don't need to hire for the job. Just use an AI and prompt it. The job you are therefore hiring for is now trivial. If it weren't, no amount of AI could pass your interview process.
  • probably_wrong 2y ago
    I belive this line of thinking mistakes the result with the process, similar to assuming that the reason companies ask people to reverse a linked list is because there's an unmet market demand for list-reversal algorithms. An interview has to be hard enough to filter those that are unqualified but also easy enough that the right person can pass with some minor preparation. If an interviewer asked me for the equivalent of production-ready code to add support for custom hardware in the Linux kernel I'd either reply with my freelance hourly rate or I'd end the interview.
    • johnnyanmac 2y ago
      >An interview has to be hard enough to filter those that are unqualified but also easy enough that the right person can pass with some minor preparation. Yeah, ideally. Meanwhile the market evolved to asking Leetcode hards on the spot. As usual, companies take a good concept and milk it to the point where you need to cheat or make studying leetcode your full time job to ace it. I'm not to sympathetic. (PS yes, I've been asked to do what was essentially spec work a few times. Woudl have been smarter to reply with an invoice, but I was too tired and just said no).
  • blazing234 2y ago
    lol what this is implying interview process actually showcases someone would be good at a job when it isnt when has leetcode ever translated to web dev... fuckin never lol.
    • nimish 2y ago
      My point exactly. Why do you let your interview process for web devs use leetcode?
• bagels 2y ago
  People are sending us emails that are not just written with chatgpt, but I think they've automated the process as well, as parts of the prompt slip in. You can see things in the emails like: "I provided a concise, polite response from a candidate to a job rejection, expressing gratitude, a desire for feedback, and interest in future opportunities."
  • buggy6257 2y ago
    Ignoring the hilarity who in their right mind is replying to job denial emails? What the hellll…
    • mdaniel 2y ago
      I for sure do, because I'm thrilled out of my mind they were polite enough to not just ghost me. I know better than to ask for feedback, or to expect any subsequent exchange, though
      • duskwuff 2y ago
        If they sent a personalized rejection email, maybe. Surely not to a boilerplate email from the ATS, though?
• themanmaran 2y ago
  On our side we've transitioned to only in person interviews. The biggest thing I've noticed is take home challenges have lost all value. Since GPT can plausibly solve almost anything you throw at it, and it doesn't give you any indication of how the candidate thinks. And to be fair, I want a candidate that uses GPT / Cursor / whatever tools get the job done. But reading the same AI solution to a coding challenge doesn't tell me anything about how they think or approach problems.
  • ghaff 2y ago
    I'm not a fan of take-home challenges anyway (for the most part). Anything non-trivial is a big time suck and you know some people will spend all weekend on you two hour assignment. Sometimes you have to. In my previous analyst stint a writing sample was pretty non-negotiable unless they could oint to publicly-published material--which was much preferred. ChatGPT isn't much use there except to save some time. It's very formulaic and wouldn't pass though, honestly, some people are worse on their own.
  • chrisfowles 2y ago
    From my perspective, you might be using take-home challenges incorrectly. The purpose for us is to have something that both sides are now somewhat familiar with, on which to base a technical conversation and ask questions. The actual solution delivered is a small part of the overall value.
• meter 2y ago
  For the time being, I’ve banned LLMs in my interviews. I want to see how the candidate reasons about code. So I try to ask practical questions and treat them like pairing sessions. - Given a broke piece of code, can you find the bug and get it working? - Implement a basic password generator, similar to 1Password (with optional characters and symbols) If you can reason about code without an LLM, then you’ll do even better with an LLM. At least, that’s my theory. I never ask trick questions. I never pull from Leetcode. I hardly care about time complexity. Just show me you can reason about code. And if you make some mistakes, I won’t judge you. I’m trying to be as fair as possible. I do understand that LLMs are part of our lives now. So I’m trying to explore ways to integrate them into the interview. But I need more time to ponder.
  • meter 2y ago
    Thinking out loud, here’s one idea for an LLM-assisted interview: - Spin up a Digital Ocean droplet - Add the candidate’s SSH key - Have them implement a basic API. It must be publicly accessible. - Connect the API to a database. Add more features. - Set up a basic deployment pipeline. Could be as simple as script that copies the code from your local machine to the server. Anything would be fair game. The goal would be to see how the candidate converses with the LLM, how they handle unexpected changes, and how they make decisions. Just a thought.
  • blazing234 2y ago
    for the first point you better provide a jira ticket with steps to get there (; i would just look at stack overflow for your second point lol...
• _lvbh 2y ago
  Run competitions. If you're hiring fresh grads, this is probably the best way to filter by skill. If you can use AI to beat all the other candidates that's a skill by itself. In practice, those that use AI rarely ever make it into the top 10. Add a presentation/demo as part of the criteria to filter out those with bad communication skills.
  • johnnyanmac 2y ago
    In my industry, competitions for hire are just a front for spec work. So YMMV.
• zitterbewegung 2y ago
  I listened in to someone interviewing people since many people used AI. It's the same with googling the answer it is very obvious that someone is taking too long to get to the answer and or you can't see a separate screen. Mitigation is literally looking at the text window and seeing if they are not typing / taking too long to even make a bad implementation. There is now a problem if you allow for google since google will autogen a gemini query to solve it.
• dijit 2y ago
  I've always just tried to hold a conversation with the candidate, what they think their strengths are weaknesses are and a little probing. This works especially well if I don't know the area they're strongest in, because then they get to explain it to me. If I don't understand it then it's a pretty clear signal that they either don't understand it well enough or are a poor communicator. Both are dealbreakers. Otherwise, for me, the most important thing is gauging: Aptitude, Motivation and Trustworthiness. If you have these three attributes then I could not possibly give a shit that you don't know how kubernetes operators work, or if you can't invert a binary tree. You'll learn when you need it; it's not like the knowledge is somehow esoteric or hidden.
  • punk_coder 2y ago
    This is how I interview potential hires. I’ll admit I haven’t interviewed someone below a senior level in probably 10 years, so I interview someone that has a resume with experience that I can draw from. I read what they’ve worked on and just go from there. I hope I never have to submit someone to some stupid take home test or Leet Code interview.
• fastily 2y ago
  I expect in person interviews are going to be the norm soon, assuming they’re not already. For now, the challenge I give candidates causes ChatGPT to produce convoluted code that a human never would. I then ask the person to explain the code line-by-line and they’re almost never able to give a satisfactory answer
• rachofsunshine 2y ago
  We haven't seen major issues with AI with candidates on camera. The couple that have tried to cheat have done so rather obviously, and the problem we use is more about problem-solving than it is about reverse-a-linked-list. This is borne out by results downstream with clients. No client we've sent more than a couple of people has ever had concerns about quality, so we're fairly confident that we are in fact detecting the cheating that is happening with reasonable consistency. I actually just looked at our data a few days ago to see how candidates who listed LLMs or related terms on their resume did on our interview. On average, they did much worse (about half the pass rate, and double the hard-fail rate). I suspect this is a general "corporate BS factor" and not anything about LLMs specifically, but it's certainly relevant.
• xyst 2y ago
  If a problem can be “trivially” solved by GPT. The problem is with your interview process, not the tool. It’s wild to me that interviewers still ask candidates for senior positions leetcode type questions. Yet the actual job is for some front end or devops position. The gap between interview and actual on job duties is very wide at many — delusional - companies.
  • blazing234 2y ago
    imagine trying to get a job doing CRUD and getting leetcode style questions.
    • locusofself 2y ago
      That is the world we live in from my experience. This is one of the reasons I decide to switch to management at 40 years old earlier this year - If I lose my job I don't really want to be grinding leetcode.
• epolanski 2y ago
  Nothing, we don't do technical interviews they are silly.
• cjkdkfktmymymyy 2y ago
  This conversation feels bizarrely tone deaf. The skill of being able to recall specific knowledge on demand is going away. How LLMs will evaluate a skill they are making obsolete is a question I am not sure I understand.
  • khazhoux 2y ago
    That is exactly what this thread is discussing. (You hardly needed a throwaway for this comment, though)
• khazhoux 2y ago
  With AI making traditional coding problems trivial, tech interviews are shifting toward practical, real-world challenges, system design, and debugging exercises rather than pure algorithm puzzles. Some companies are revisiting in-person whiteboarding to assess thought processes, while others embrace AI, evaluating how candidates integrate it into their workflow. There's also a greater focus on explaining decisions, trade-offs, and collaboration. Instead of banning AI, many employers now test how effectively candidates use it while ensuring they have foundational skills. The trend favors assessing problem-solving in real work scenarios rather than just coding ability under artificial constraints.
  • guitheeengineer 2y ago
    This sounds like it was generated by AI...
• sweca 2y ago
  My company actually encourages the use of AI. My interview process was one relatively complex take home, an explanation of my solutions and thinking, then a live "onsite" (via zoom) where I had to code in front of a senior engineer while thinking aloud. If I was incompetent, I could've shoved the problem into o1 on ChatGPT and probably solved the problems, but I wouldn't have been able to provide insight into why I made the design choices I made and how I optimized my solutions, and that would've ultimately gotten me thrown out of the candidate pool.
  • Aachen 2y ago
    What does onsite mean if it was a video call?
    • descendant 2y ago
      Just a new term used during and after COVID. It comes after the initial 'tech screen' round. They call is 'virtual onsite' sometimes. It involves giving multiple interviews in a very short duration. A positive decision in these 2-3 rounds pushes you forward to the final round with the hiring manager.
    • sweca 2y ago
      It just means a virtual onsite type interview process. Real time interviews for both skill and cultural fit.
• shihab 2y ago
  To get an idea of just how advanced cheating tools has become, take a look here: https://leetcodewizard.io/ https://leetcodewizard.io/ I think every interviewer, hiring manager ought to know or be trained on these tools, your intuition about candidate's behaviour isn't enough. Otherwise, we will soon reach a tipping point where honest candidates will be at a severe disadvantage.
  • paxys 2y ago
    Tbh I’m very happy these tools exist. If your company wants to ask dumb formulaic leetcode questions and doesn’t care about the candidate’s actual ability then this is what you deserve. If they can automate the interview so well then they should also be able to automate the job right? Or are your interview questions not representative of what the job actually entails?
    • pydry 2y ago
      Yeah, it's just a pity that human stupidity perpetuated leetcode as an interviewing tool to the point that AI had to kill it.... Im really happy it's finally broken though. Dumbest fad our industry ever had.
      • lubujackson 2y ago
        I dunno... estimating how many golf balls fit in a bus or explaining why manhole cover are round make leetcode look almost... useful by comparison.
    • shihab 2y ago
      I understand this sentiment for experienced developers. It is an imperfect signal. But what is in your opinion a better signal for junior or new grads? Every alternative I can think of is either worse, or sounds nice but impractical to implement in practice at scale. I don’t know about you, but most interviewers out there don’t have the ability to judge the technical merit of a bullshitters’s contribution to a class or internship project in half an hour, specially if it’s in a domain interviewer has no familiarity with. And by the way, not all of them are completely dumb, they do know computer science, just perhaps not as well as an honest competitor.
• deleted 2y ago
  [deleted]
• ungreased0675 2y ago
  I’ve had people using AI in interviews to answer simple “get to know you” questions. People just disconnect their brain and read whatever the machine says.
• gigatexal 2y ago
  It has always been more about the application of facts or knowledge than the retrieval of facts that makes for a really good interview. It’s better to know when to use a Linked list than how to make one (because I’d just use the one in the library). So the candidate can prompt well good. But how much of the knowledge can they apply to a problem or are they just masters of hacker rank (sic). But more often than not most interviewers are lazy and just use canned hacker rank style questions or if it’s not laziness it’s being too overworked to craft a really good interview.
• shayarma 2y ago
  They still find a way to make it horrible and degrading.
• n00b101 2y ago
  [flagged]
• n0rdy 2y ago
  Here where I live, home assignments with the follow-up tech discussion are way more common than the leetcode-like interviews. Therefore, I can't say that the process has already changed dramatically. Yes, I did review a couple of home assignments that had all the signs of being completely AI-generated. But it all became clear during the presentation of the solution, "why"-like questions, and extending the scope of the task to discuss the improvements. If the candidate could answer that, then AI was just a supplementary tool, which is great, as we also use Copilot, ChatGPT, and friends at work. If not, well, it's an obvious rejection. It happened twice, that the candidate on the other side was clearly typing during the interview, taking a pause for a second or two, and then reading from the screen. That's very obvious as of today, but I can see how it will become a problem one day with the future AI development in terms of the speed of responses, and better voice recognition techniques (so, no typing needed).
• 1270018080 2y ago
  You have to depend on your professional network to skip the BS of the modern day interview process (as an applicant and interviewer).
• Peterthomos 2y ago
  [dead]
• esafak 2y ago
  These days I ask real-world debugging questions where the root cause is not the error on the screen. I allow LLM use as long as I see how. In my unfortunate experience, candidates who covertly rely on LLMs also tend to have embellished resumes, so I try to root them out before asking technical questions.
• donatj 2y ago
  Oh man, I have kind of hunkered down and not interviewed anywhere since the start of COVID. I had not even thought about how AI might affect things let alone people not being in office. Last time I interviewed I spent about half of it standing at a whiteboard.
• mattbillenstein 2y ago
  I've only done a few interviews the past couple years, but I've asked people to turn off coding assistants and not use an LLM on my coding screen. I want to know how _they_ think and solve problems, not how the LLM does. And generally, the more junior people are just completely lost without it. They've become so dependent on it, they can't even google anything anymore. Their search queries are very weirdly conversational questions and the idea of reading the docs for whatever language or library they're using is totally foreign to them. I think it's really hampering the growth of junior devs - their reasoning and thought processes are just totally tuned to this conversational form of copy and paste programming, and the code is really bad. I think the bottom half of programmers may just LLM themselves out of any sort of job because they lose the ability to think and problem solve... Kinda sad imo.
  • politician 2y ago
    It was obvious from the moment ChatGPT went live that this was going to intellectually stunt junior developers.
• skeptrune 2y ago
  I have been asking people questions about git a lot. It's useful to figure out whether or not they care about the craft.
• sramam 2y ago
  I recently completed a take-home assignment with the following instructions: <instructions> This project is designed to evaluate your ability to: - Deconstruct complex problems into actionable steps. - Quickly explore and adopt new frameworks. - Implement a small but impactful proof of concept (PoC). - Demonstrate coding craftsmanship through clean, well-architected code. We estimate this project will take approximately 5–7 hours. If you find that it requires more time, let us know so we can adjust the scope. Feel free to use any tools, libraries, frameworks, or LLMs during this exercise. Also, you’re welcome to reach out to us at any time with questions or for clarification. </instructions> I used LLM-as-a-junior-dev to generate 95+% of the code and documentation. I'm just an average programmer, but tried to set a bar that if I was on the other side of the table, I'd hire anyone who demonstrated the quality of output submitted. - The 5-7 hour estimate was exceeded (however, I was the first one through this exercise). - IMHO the quality of the submission could NOT have been met in lesser time. - They had 3 tasks/projects: - a data science project, - a CLI based project and - a web app - They wanted each to be done in a different language. - I submitted my solution <38 hours of receipt of the assignment. - In any other world, the intensity of this exercise would cause a panic-attack/burn-out. - I slept well (2 nights of sleep), took care of family responsibilities and felt good enough to attack the next work-day. I've been on both sides of the table of many interviews. This was by far the most fun and one to replicate every chance I get. [EDITS]: Formatting and typos.
  • johnnyanmac 2y ago
    5-7 hour interview take homes is already a nightmare. LLM assistance or not, I would absolutely not bother with such an assignment unless I was far into the process. Meanwhile, I'm given such tasks half the time before I speak to any human.
    • sramam 2y ago
      That is a fair point. This was the final technical screen so definitely something worth doing in my case. The reason I posted a reply was there is a lot of negativity around AI in the hiring process. This was an excellent example of using AI to the benefit of all parties. Instead of nit-picking on stylistic things from a smaller code-sample, one can nit-pick on the implemented complexity. I think it is a higher quality signal.
• yieldcrv 2y ago
  I recently did an interview that that involved making pull requests for code updates across a time period That seemed to thwart AI use, at least thwart one-shotting, and require understanding and experience with working in an organization I liked that
  • Aachen 2y ago
    What does that even mean? Trickling in changes you want to make via multiple pull requests, spaced across days or months or whatever, instead of submitting it at once?
    • yieldcrv 2y ago
      across a few hours but, yes, exactly that
• Perenti 2y ago
  Can I hallucinate answers in an interview, and have them rated as acceptable?
  • MathMonkeyMan 2y ago
    Yes, it's called lying effectively.
• WatchDog 2y ago
  The last interview I ran was back when copilot came out. My company at the same had been using the same coding exercise for years, and many candidates inevitably published their code to github, so copilot was well trained on the exercise. I had a candidate that used copilot, still flubbed the interview, while ignoring perfectly good suggestions from the LLM.
• _sword 2y ago
  Even before LLMs were popularized, the shift to remote work made hiring awful in my experience. In finance roles, I had candidates who aced their tests and projects but then showed up to the job unable to competently use excel or write coherent sentences in English. Phone / zoom interviews all went fine, but clearly there was rampant cheating during remote projects.
• DustinBrett 2y ago
  "when problems are trivially solvable by GPT" only the fake leet code ones that were never really what you'd do at work anyway. Sadly that is what people are giving devs in interviews. Screen share to avoid cheating via AI, same as we were doing before AI when people could get friends or Google to help them cheat.
• tzury 2y ago
  Google. StackOverflow. ChatGPT/Claude et al. Those are all tools. Tools of the craft, so a professional candidate should be using it proficiently. How good one is in understanding a problem (the scope and source), and how good are they at designing a solution that is simple, yet will tackle other "upcoming" problems (configurable vs hard coded, etc). This is what one should care about. * StackOverflow early days raised the same question, and so were Google. trust me. I am old enough to remember this.
• deleted 2y ago
  [deleted]
• deadbabe 2y ago
  We now consider the employees that we hired prior to the arrival of AI to be the equivalent of “low background steel”. They have much stronger job security. Everyone hired after that is more suspect, and if they screw up too much or don’t perform well we just fire them quickly during the probation period, whereas previously it was rare for people to get fired during the probation period.
• skeeter2020 2y ago
  >> now when problems are trivially solvable by GPT. Only the trivial problems. We don't use AI during interviews but many try and it's always obvious. Delay after any question problem; textbook perfect initial answer; absolutely nothing when asked to go deeper on a specific dimension. It's nice because interviews that are scheduled for an hour are only lasting ~20 minutes in these situations, and we can cut them short.
• EGreg 2y ago
  I mean, it could be really bad Online, you don't know if the person you're interviewing is an AI or not. They could have an AI resume, AI persona, AI profile, and then someone shows up who looks like that and it could be a deepfake, then they do the coding challenges, and then you hire the person remotely, and they continue to do great work with the AI, but actually this person doesn't exist, they just created 100 such people and they're just AI agents. It sucks if you want to hire humans. And it sucks for the humans. But the work gets done for the same price and more reliably. So dunno
• fsloth 2y ago
  Trick problems and whiteboarding was always “jump through the hoops” bs IMHO. In-person dialogue with multiple team members and sufficiently complex take-home assignment remains a pretty good method. LLM are excellent API docs and should not be avoided.
• RomanPushkin 2y ago
  One must not forget that cheaters are now everyone, and it's likely you gonna be interviewed by a cheater. I've seen this multiple times already, recent is Meta interview ~8 months ago. Very low quality interviewer - complained about bug in the code, while there was no bug. Can't keep the conversation going, the same for system design - poor guy didn't even want to listen, and was pretty much rude. I would say that if it wasn't a pattern. So let's not pretend they're not cheaters. Call them out.
• prakhar897 2y ago
  I interviewed a while back and wrote about it here: https://www.softwaredesign.ing/blog/ai-is-the-reason-interviews-are-harder-now https://www.softwaredesign.ing/blog/ai-is-the-reason-intervi... The kind of core conclusion is that companies need to do in person interviews now. There's no other way to prevent cheating.
• topkai22 2y ago
  I’m trying something new on my next interview- I’ve had an LLM solve a coding problem and I’ve uploaded the output to github, I’m going to ask the candidate to evaluate the generated code adapt it to new technical and non functional requirements.
• madduci 2y ago
  Do people still insist on coding tasks? Why don't simply formulate questions that await a broader and deeper knowledge of things, which helps to know how far the candidate can go? And for questions, I don't mean "it's better a list or a set?", but something like: "you have an application like this, how can you improve it to perform X?"
  • edanm 2y ago
    Because most people who've been involved in tech interviewing have arrived at the same conclusion - these kinds of questions are too easy for candidates to answer in ways that make them appear great, while they can't actually program.
  • mrweasel 2y ago
    People still cheat, or try to. Some of our interviewers have seen candidates start out the interview being completely unable to answer the most basic questions about a programming language, only to flip around a few moments later and give almost encyclopaedic answers, with release dates for specific feature and comment on implementation details. One minute you can't do array slicing and now you're able to talk about the inner workings of the garbage collector? We're trying to be fair in hiring, every resume is manually read, interviews a conversations, there are no leet code, no tricks, no AI screening, but they are done remote and so far two out of three candidate clearly cheat, during a live interview, to cover up an impressive lack of general knowledge.
• andoveragain 2y ago
  > everyone Who is everyone?
• low_tech_punk 2y ago
  I'm actually glad AI use is revealing misalignment: If the AI is so good at it, why are we still hiring human to do the job? It just shows how the interview process is not measuring the right thing to start with.
• dinkumthinkum 2y ago
  I don’t really see why it’s so hard to interview someone remotely because of AI or speech to text. I’m not aware of any system that is so advanced or fast enough that having a video call and decent conversation skills and carefully listening to the candidate can’t identify 99.99% of people using AI. Even without video, it’s no hard to have a conversation and guage their skill level and ask questions and listen to the answers carefully. You can tell when the conversation is not fluid or the answe given is at a level that is a mismatch with their apparent sophistication. You can ask an unreasonably obscure or difficult question and see if you get an LLM answer. You can suss it out. You don’t have to do in-person interviews.
• deleted 2y ago
  [deleted]
• fhd2 2y ago
  The last time I've used a leet code style interview was in 2012, and it resulted in a bad hire (who just happened to have trained on the questions we used). I've hired something like 150 developers so far, and what I ended up with after a few years of trial and error: 1. Use recruiters and network: Wading through the sheer volume of applications was even nasty before COVID, I don't even want to imagine what it's like now. A good recruiter or a recommendation can save a lot of time. 2. Do either no take home test, or one that takes at most two hours. I do discuss the solution candidates came up with, so as long as they can demonstrate they know what they did there, I don't care too much how they did it. If I do this part, it's just to establish some base line competency. 3. Put the candidate at ease - nervous people don't interview well, another problem with non-trivial tasks in technical interviews. I rarely do any live coding, if I do, it's pairing and for management roles, to e.g. probe how they manage disagreement and such. But for developers, they mostly shine when not under pressure, I try to see that side of them. 4. Talk through past and current challenges, technical and otherwise. This is by far the most powerful part of the interview IMHO. Had a bad manager? Cool, what did you do about it? I'm not looking for them having resolved whatever issue we talk about, I'm trying to understand who they are and how they'd fit into the team. I've been using this process for almost a decade now, and currently don't think I need to change anything about it with respect to LLMs. I kinda wish it was more merit based, but I haven't found a way to do that well yet. Maybe it's me, or maybe it's just not feasible. The work I tend to be involved in seems way too multi faceted to have a single standard test that will seriously predict how well a candidate will do on the job. My workaround is to rely on intuition for the most part.
  • fhd2 2y ago
    To add: I can very well imagine this process isn't suitable for FAANG, so I can understand their university exam style approach to a degree. It's easy to arm chair criticise, but I don't know if I could come up with something better at their scale. These days, I'm mostly engaged by startups to help them build an initial team, I acknowledge that's different from what a lot of other folks hire for.
    • mnky9800n 2y ago
• bilekas 2y ago
  I've given a few interviews with take home screening small projects that I have been able to identify very quickly were majority generated, I usually don't mind it too much but when I asked why they chose certain patterns or why they went with their approach they couldnt give any "true" answer a week later. I feel that smaller things like syntax etc make perfect sense. But for larger things that involve a slightly higher complexity it becomes a bit grey. I likenit personally to writing. When I write things down as I'm trying to work things out or even trying to learn something I find I retain the data so much better and have a better picture in my mind of what's going on. That might just be personal preference for learning but if I copy straight from claud I know 100% I'm not going to remember anything about it the next day.
• VPenkov 2y ago
  My employer sends a take-home test. It is relatively easy and not very time-consuming. Its main purpose is to act as a basic filter and to provide some material to base an interview on. In the recent couple of years I have seen a lot more people ace the test and not do very well during the actual interview. Take-home exams feel like they would always be ineffective now.
• fergie 2y ago
  In my experience the real objective of the take home exercise is to gauge how compliant the candidate is, and how many obligations they have outside of work. Otherwise it always makes more sense to conduct simple, in-person assessments.
• qq66 2y ago
  Why do you have to evaluate the person without AI? Unless they won't be able to use AI on the job (for security reasons or whatever) it seems like it makes more sense to have them pull up their favorite AI and use it to solve a problem. Give them some buggy code and ask them to fix it with any tools they want.
• mr90210 2y ago
  Some recruiters have tried to record our initial interview with one of those services that automatically captures notes. I REFUSED.
• anothernewdude 2y ago
  I make it clear on my resume that I won't work for companies that use AI in their hiring processes.
• alkonaut 2y ago
  It's not the solution itself that is interesting to me, it's first finding out whether the person can go through the motions of solving it. Like reading instructions, submitting solutions etc. It filters out those who can't code at all or who can't read instructions. A surprisingly large chunk. If the person also pipes the problem through an LLM, good. To then select a good developer I'd test communication skills. Have them communicate what the pros/cons of several presented solutions are. And have them critique their own solution. To ensure they don't have canned answers, I might just swap the problem/solutions for the in-person bit. The problem they actually solved and how they did it isn't very important. It's whether they could read and understand the problem, formulate multiple solutions, describe why one would be chosen over another. Being presented with a novel problem and being asked on the spot to analyze it is a good exercise for developers (Assuming software development is the job we're discussing here). Just take the time to talk to people. The job is about reading and writing human language more than computer programming. Especially with the arrival of AI when every junior developer is now micro managing even more junior AI colleagues.
• yodsanklai 2y ago
  You can always interview in person. This has often been the norm, after some initial screening. I think it's the best option. Even remotely, normally a coding interview isn't a candidate typing things for 45 min on a screen. There are interactions, follow-up questions, discussions about trade-offs and so on... I suppose it's possible for a good candidate to cheat and get some extra points, but the interview isn't broken yet. You could also let the candidate use AI, and still gather all the relevant signals.
• littlestymaar 2y ago
  The coding interview I designed back in 2022 is still out of reach of all LLMs on the market while still being good at selecting people (I hired 6 people with it, all of them where great fit, out of 30ish applications so it doesn't seem to have a high false-negative rate either). The main takeaway is that if you make design your interview questions to match the actual skill you're looking for, AI won't be an issue because it doesn't have those skills yet. In short: ask questions that are straightforward in surface but deep beneath with trade-offs that must be weighted by asking questions to in interviewer.
• nathias 2y ago
  the biggest change I've seen as a remote dev is considerably less replies to my non-AI made cv, I think the competition for cheap labor got much harsher
• johnnguyen-etsy 2y ago
  a
• CoolCold 2y ago
  not a problem yet - we still start asking hard question first - on differences on TCP and UDP and which one is better and why - very deep rabbit whole I must say. Then simple questions - how do you manage k8s clusters and what you suggest for multi DC setups
• shafkathullah 2y ago
  it makes sense, because interview questions often lack novelty, it's usually very repeated, and almost all AI is very good at these, i think asking to switch of AI and doing a live coding or thinking outloud is the way forward. or else just give them a sample story point and see how well they respond, if they can do it well means they can work well in the company.
• ionwake 2y ago
  PREFACE: LONG RANT I dont know but Ill always remember the funniest thing I noticed once during my career in England.. A company called tripadvisor based in a very very small town where I was at the time a senior dev, working on my own things, had never reached out. yet I saw their ads and finally an actual article in a newspaper, basically where they were bragging about their latest hire. Let call him Pablo, who had apparently aced every single technical interview question, and so they had hired him, after interviewing tens of people. They were so happy with their hire, the article had been based on him and the "curiosity" that they were the first company to have hired him after he had failed I think it was something like 50 interviews. Obviously they couldn't believe how lucky they were to have finally found someone who could have completed all the technical tests perfectly. Now I have nothing against Pablo, and rooted for when when I read the article. But I found it hilarious, and still do almost a decade later, that this top tier company, based in a university town with perhaps the most famous university, had not realised they had simply over fitted for someone who could answer their leetcode selection perfectly. Not only not realised this but then commissioned the article. Eventually they reached out for an interview with me where the recruiter promised there was no test for me, then I was "surprised by one" in a room with a manager who hadnt had time to read my expection ( which is fine ), but when he walked in and saw I hadn't done it I was "walked out". The whole interview having taken less than about 10 minutes, when I was the most qualified senior developer for hundreds of miles who was available at the time. No Im honestly not tryign to brag Im just saying the town was so small there just couldnt have been more during that short time period I was available. I know this reads bitter,( my life is great now ) , I just remember it because, my point was at the time I was at my peak, and would have accepted the job if offered, but I was walked out within 10 mins. Honestly just sharing this insight, the moral of the story I think for me is companies never were great hiring and well if anything the advent of LLMs might actually improve as LLMs start to assess people based on their profiles and work? One can hope, I dont, I want an edge in this market with my company.
• _heimdall 2y ago
  I went through a round of interviews the second half of last year. Interviewing felt the same as it had over the last 5 or 10 years honestly. I had a few coding challenges, all were preinterview and submitted online or shared in a private repo. One company had an online quiz that was actually really interesting to take, the questions were all multiple choice but done really well to tease out someone's experience in a few key areas. For what its worth I don't use LLMs and the interview loop went about as I'd expect in a tough job market.
  • MathMonkeyMan 2y ago
    I've had the same experience lately, though I think I might be getting lucky with a few of these interviews. The leetcode questions, in particular, have been softball. I do appreciate that...
• joshstrange 2y ago
  It doesn’t affect our process, we even let people use AI/LLMs on the coding test. Just asked to add a small feature or make a small change after the present the initial code (without AI help). That makes it really easy to see who doesn’t understand the code they are presenting. I don’t care how much AI you use if you understand the code that it writes and can build on top of it.
• deleted 2y ago
  [deleted]
• thih9 2y ago
  Not everyone is using AI; and speaking as one such developer, interviewing is not fun.
• sarchertech 2y ago
  I do not understand the obsession with proving beyond a shadow of a doubt that an interviewer can code. Not being able to code is by far the easiest failure mode to deal with and I can deal with it more quickly by looking at resumes and firing people for outright lying about their abilities very quickly. What is much harder to detect is the person who gives up at the first sign of trouble. Or someone who likes to over abstract everything. Or someone who likes to spend all day nitpicking PRs. The absolute most damaging employee is the technical tornado midlevel who has prolific output and is good at figuring out how to get PRs through the code review process. I can only bring to begin imagine the kind of damage a person like that could do with an LLM, an inattentive manager, and a buddy willing to rubber stamp PRs.
  • benjiro 2y ago
    > firing people for outright lying about their abilities very quickly I assume your from the US? The issue with firing people quickly after hiring them is: * You need to pay whatever headhunter or other personnel that is involved. Some offer a cashback or reduced rate to find a new person, but your still out of the pocket. Do it a few too many times and the headhunter dump you. * You just wasted a ton of time of other employees, setting up this persons accounts, social benefits, contract negotiation, training time, and more. Again, this is not free. * Now you are back to looking for other people. You know those people that you rejected, well, the change is reduced that they will come back to your company, maybe they got hired somewhere else, maybe your rejection left a bad impression as you hired that lying person over a actually well qualified person that just did not ace the interview (happens to so many people, often because of nerves). The trick is, to have a that person (and others in the interview process) work a few days in your company, but this is not covered by a official work contract but a interim contract (of course, you pay those people, some companies are real bastard and do not pay). That can weed out the people that are good at lying about their skills. But to be honest, if you can not spot them during the interview stage, ... We had people coming in for Senior positions, and their CVs looked good, they knew how to talk the talk. But when asked for some incredible basic SQL question, they fumbled so much, because they used ORMs their entire life. But loved to place SQL skills on their CV. Seen multiple of those... And so often they then switch to going after the interviewer, thinking that by attacking the interviewer before the boss, that will help them. Yea, "Senior material"... With AI its going to be the same. People get comfortable using AI, but when you ask code questions without that AI to back them up, that skill gap will exposé themselves very fast during the actual interviews. I love using AI because it can remove so much boilerplate / repeat / great for finding doc info but you need to know your language. If you grew up with only AI, and do not grow beyond AI to solve every issue, your nothing more but a copy/past dev (AI is just a extension of that dev type). > Or someone who likes to over abstract everything. No no, the most annoying ones are those that come in and then want to implement everything into a style they are familiar with! Code rewrites to fit them, framework changes because they are familiar with them... Nothing more fun as hiring somebody to write Angular for 13k/month (talking almost 10 years ago), and then this person pushing Vue, to even rewriting part of the codebase into Vue. Like, we hired you with a insane salary because we are desperate for a Angular developer, if we wanted to rewrite everything to Vue, we will have hired somebody cheaper.
• hyperkinetic 2y ago
  Asking experienced engineers to do a coding test is stupid and will only give you candidates that can't think for themselves.
• lordhexd 2y ago
  Unless it’s for a senior role, why not just find the candidate you and your team can work best with? Skim through CVs and drop the first 20 an email asking them about their last team/boss or their day or to tell you the last thing they learnt. Something personal, the goal is to note their communication style. Pick 10 you feel matches the energy you want. From those 10 get 2 in the office every day for 2 weeks to see how they work with peers/competitors and your team. By the end of two weeks you should have a relatively good hire which you can work with to fit the team.
  • deleted 2y ago
    [deleted]
• miah_ 2y ago
  I feel like I'm one of the few people who won't touch the tech for anything work related. I've used it to generate character backstories in Dungeons and Dragons, but I wouldn't trust it for anything reality based. I don't know what everybody is smoking.
• acedTrex 2y ago
  we are doing in person interviews, I just interviewed someone a few days ago that was VERY CLEARLY using chatGPT audio to cheat. You could see his eyes jumping from word to word and the things he was saying was clearly gpt output. Its a sad world out there these days.
• syrusakbary 2y ago
  Pro tip: ask about their previous experience and go deep there. It will be impossible for the AI to generate fake experience on a user, and very easy to catch for the interviewee
• bl4ckm0r3 2y ago
  I have wrote an article a year ago about this. https://medium.com/fever-engineering/on-hiring-great-talent-6abdab0c770f https://medium.com/fever-engineering/on-hiring-great-talent-... tbh it's very hard, in general, to assess someone skills in ~1 hour, with the diverse set of problems we face everyday and people/companies often focus too much on "previous relevant experience" (do you know this?) instead of thought process and depth of understanding (how much have you understood what you have done) which, to me, gives more insights on someone personality and attitude and ability to learn new things and becomes harder to cheat on (you can memorize "cracking the code interview" and be a hero on leetcode and still have no idea about how to write good software, take decisions or work in a team :)
• code_for_monkey 2y ago
  Ive been interviewing at start ups to get out of my contractor job, but recently I gave up. I had this one interview where I had to remove nodes from a linkedlist. Pretty trivial, right? It is, but as I was writing out the solution, probably for the 200th time, I thought "Ive never used a linkedlist, ever" And I thought, I just cant do it anymore. I cant reverse one single more binary tree. I cant listen to some CTO tech bro in his early 30s explain to me why I need to be in an office he'll never set foot in for the 'company culture'. Ive had so many interviews just like this. Ive done 7 hour on sites where you break down algorithms, system design, app development. Its brutal out there, no one seems to have any idea what their looking for so they just put you through the ringer. So now im not interviewing, Im happy and content in my bank programming job. Im probably not a very good programmer, thats okay, Im a wonderful partner, family member, musician. Ill get by, hopefully, for another decade until everything collapses.
• z3phyr 2y ago
  Who do I want to work with? A person who can work with me, get stuff done and somewhat match vibe with their coworkers. They can be good in spelling bee, swimming, chess, marathon or leetcode, its on them. I do not get much useful information out of it. What I usually do is case study that I also do not know at the start of the interview. The case study does not imagine spherical cows and are not usually leetcode style. Its a case of role playing a problem. We brainstorm it together and I determine if I can work with them.
  • vanceism7_ 2y ago
    Thats a good point. That's probably a way better way to run an interview than anything else. Back in my highschool band, our band director used to say "play how you practice, and practice how you play" - this sounds like that rule applied pretty effectively. At the end of the day, our colleagues are people we problem solve with, so we should test things like that when we're hiring
• sebazzz 2y ago
  I think it is actually more of a potential disappointment for the candidate than the employer. Because when the candidate becomes the employee, the employer will notice this person will never stick out, never stands out, and will only be handed basic tasks. Implementing basic product features. But something more difficult won’t be entrusted.
• matrix87 2y ago
  I just got done interviewing with a big tech company and a successful outcome, I really don't understand how someone could even pull that shit off in an interview. It sounds harder than just doing it the honest way
• CharlieDigital 2y ago
  About 18 months ago, I interviewed with a YC startup and the first round was actually a code review. It included an API which included some SQL commands as well as a SQL schema. I pointed out missing validations, missing error handling, lack of indices, alternate data types, possible XSS vectors, etc. It was a really great format and I think one that creates better separation between good candidates and great candidates because it is more open-ended and collaborative. One of my favorite technical interviews I've engaged in. I think that now, with AI coding assistants becoming even more integral than those early days 18 months ago, this approach is more relevant than ever since it gives insight into how efficiently a candidate can quickly review AI generated code for correctness, defects, performance issues, and other gaps in quality. I liked this format so much that I ended up creating a small open-source tool for it to make it easier to manage this process: https://coderev.app https://coderev.app (https://github.com/CharlieDigital/coderev https://github.com/CharlieDigital/coderev) (The interviewer had to create a private GH repo and the repo itself didn't support commenting inline; I took my notes in a text file and reviewed it interactively with the interviewer).
• fragmede 2y ago
  Here's the question, here's the code ChatGPT produced, what's wrong with it, and why?
• yesiamyourdad 2y ago
  About 20 years ago, I used to do interviews that were "write a program to add two numbers together" (I think specifically I asked for a web application). It's trivial, right? There's actually a lot going on. They have to parse input, and sometimes you get strange things like "well if I make it use doubles then I'll cover all scenarios". You have opportunities to talk about error handling (bad characters in the input, int overflow, etc). You can talk about refactoring (now make it handle -,* & /). You can ask them about writing tests. Ask about how they'd handle arbitrarily large numbers. There's a bunch of ways you could take the conversation and really just talk about average developer activites. What I liked about that process is that it relied less on their ability to suss out a solution to some problem they'll never have to solve on the job and focused more on average activities. Sometimes I'd get a candidate who would go "wait, is this a trap?" and start asking a lot of questions - good! Now I got to see them refine requirements. Having them review a PR is a good exercise too, you can see how they are at feedback.
• koliber 2y ago
  I've been recruiting a lot over the past 8 years. First as a manager, later as an engineering department leader, and most recently in the role of a specialize recruiter. I've seen a few things change. The interviews themselves are for the most part unchanged. Occasionally I see someone who seems to be using AI during the interview. It's sort of obvious. They fist give a vague answer. It's as if they repeat the question. Then they answer while not maintaining eye contact. When asked a followup or a "why" question, they fall apart. They do poorly. I find more people pass the written screen nowadays and then bomb the interview. I'm guessing they use AI. C'est la vie. It seems that a lot more people are looking for work now than 3+ years ago. It seems that for roles that require top-notch coding ability, there are relatively few people on the market. Im guessing that great people are staying at their jobs longer now. Makes sense. After a candidate gives an answer, probe. Ask why they think that. Ask what alternatives they might consider. Watch their body language and how quickly they answer. For context, I hire very experienced developers in Poland for very demanding remote roles in the US.
• tharne 2y ago
  > What is interviewing like now with everyone using AI? Two main things I've seen: 1. Recommendations are a heck of a lot more important 2. Internal applicants are suddenly at a big advantage
• michelb 2y ago
  I think this mainly highlights how bad interviews were in general before AI.
• ciwchris 2y ago
  This question came up on Soft Skills Engineering[1]. To cut through the noise it's more important to have a blog or a project you contribute to which you can point to. And then yes, whiteboard type of questions which unfortunately are not reflective of day to day work. [1]: https://softskills.audio/2025/02/03/episode-446-wading-through-ai-slop-and-they-don't-get-git/ https://softskills.audio/2025/02/03/episode-446-wading-throu...
• andyish 2y ago
  For tech and nontech, it's terrible. Nontech roles are just a sea of prompt-generated answers, they don't tie into the applicants' experience and are usually about 200 words of waffle. If you're applying for something, type out a few sentences then give it to a prompt to refine. Don't just paste the question in. Tech roles we focus on the combination of soft skills and technical skills and so we've gone back to a 'pair' programming exercise and a whiteboard architecture exercise. In reality, it's just a spectator sport that we nudge them forward if needed. In Java looking for them to cover the basics of debugging a problem, writing a test, and mocking out some services. I would say 50% are unable to or unwilling to write a test to prove the error or don't know how to mock out a service. Whiteboard exercise looking for them to explain a system they've worked on in the past. We question some of their decisions and see how they handle themselves. Not many get defensive (though some do).
• aristofun 2y ago
  If your interview process can be solved by AI - it only means your interview process sucks. Which is the case for >90% of the companies I interviewed with (big and small)
• cooperreginald 2y ago
  [dead]
• cooperreginald 2y ago
  [dead]
• nvarsj 2y ago
  I'm pretty sure the majority of people I interview (in big tech) are now cheating with some kind of assistance. Usually it's obvious, but not always. I am just plodding along though, and hoping my company moves back to on-site interviews. Alternatively, interviews need to be designed for AI assistance (e.g. much more complex problems).