7 ms·

AI is the reason interviews are harder now

73 points by prakhar897 2y ago

michaelteter 2y ago
It doesn't matter if AI makes interviews harder. An utterly broken system is not practically worse when it becomes more broken. We were already well past the point of futility before AI became part of the system. The only real way to get a job is by the equivalent of mass spamming and luck (which also means low chance of fit if hired) or by human connection and familiarity (the one true way for anything in life).
- cranberryturkey 2y ago
  I just landed a contract today after 9 months of not working. It was dumb luck.
- ghaff 2y ago
  >by human connection and familiarity (the one true way for anything in life) Every job I've had since grad school was through personal connections. (And luck.) I suspect the state of SWE (and adjacent roles) for the past decade+ leads to a lot of SWEs believing that sending out a few applications will result in a great offer in a week. I'm not sure how much AI has to do with making spray and pray less effective or enabling various cheating in interviews--presumably leading at some point to only allowing in-person interviews after an initial screen--but it probably decreases the time spent on the initial filter and you start relying more on heuristics like where they went to school.
  - nicbou 2y ago
    The problem with connections is that some people have none. I help immigrants for a living, and the biggest underlying challenge for recent immigrants is the lack of a network. They truly start from scratch, so for a year or two they are often completely alone. Networking is absurdly effective, since all nodes stake their reputation every time they recommend another node. However it only helps those in the network. This leaves a lot of good candidates out. It can end up creating sort of exclusive club.
- classified 2y ago
  Agreed, it's a total gamble, but even in a game of luck there can be factors that make your odds even worse.
baron816 2y ago
I wish more companies would have interviewees conduct code reviews. Code reviews as an interview show a number of things you wouldn’t get from a typical interview—what opinions they have, what are the things they call out vs what they don’t waste time on, how they might communicate with another teammate, and more. And if we’re going to a world where AIs do much of the work and we just need to check that they implemented what we intended, those code review skill will still be highly relevant.
- nunez 2y ago
  I did a code review style interview at a startup a few years ago (loved them; got an offer, but my then-current employer countered surprisingly well). I really enjoyed it.
- kragen 2y ago
  gpt-4 is at least as good as i am at code reviews, so i don't think this solves the problem this post is about
  - Copenjin 2y ago
    The objective is not finding someone good at reviews, or even that gets a specific review "right". It's having a chance to understand how someone thinks, better evaluate their actual knowledge and test for some important red flags.
    - kragen 2y ago
      gpt-4 is only mediocre at many things, and i've found mistakes in its code reviews (not, to be sure, mistakes i couldn't have made), but one thing it's absolutely superhuman at is avoiding red flags
      - godelski 2y ago
        > gpt-4 is only mediocre at many things, and i've found mistakes in its code reviews >> gpt-4 is at least as good as i am at code reviews, so i don't think this solves the problem this post is about These two comments don't seem consistent. Honestly, "i've found mistakes in its code reviews" vs "it's absolutely superhuman at is avoiding red flags" is possibly not self-consistent. But I think you mean glaring mistakes? Which if I understand you correctly, than I'm not sure how you get the first conclusion unless you are saying you're mediocre (which is fine). But really it just makes it seem like you and the machine complement one another, rather than compete, which makes me not understand the original comment in context. But I mostly agree with Copenjin. Interviews are less about on the spot skill checks so much as learning how someone thinks and problem solves. Honestly, you could play a boardgame/videogame/cardgame with them and it would be an effective interview (asking them relevant questions during the game, and maybe even better if it's a relatively unknown game so they can zero shot it). The reason for this is that in real world work you are more concerned with how someone adapts to changing environments and thinks through situations. To see when they'll ask for help, what they might get stuck on, and how they strategize. Your employees will always be gaining new skills. And honestly, it is easier to take a lower skilled person who's more adaptable and driven and turn them into a great employee than it is to take someone who's got skills but will stagnate. But ymmv depending on the job and requirements. Sometimes you just need to fill a seat.
infotainment 2y ago
Prior to the pandemic, in-person interviews were the norm at big tech, but they never really went back to them afterward. I wonder if this could be the beginning of the end for purely remote interviewing; seems like in-person interviews would be less likely to be gamed in this way.
- weatherlite 2y ago
  Yep I agree I think its coming to an end unless some good and reliable counter cheating tools are developed.
- sien 2y ago
  This exactly. Before the pandemic the team I was in used to conduct interviews in person. We'd give people a sheet of paper with a bunch of questions that they had to answer. No leetcode type stuff. Just basics. With that we had one person cheat with a phone. Doing remote interviews from then on we saw at least one person who was very probably cheating. They would get an answer, pause, then give a response, but then when we asked follow up questions they would seem not to understand their previous answer. They also had 'Audio Visual Issues' at the start of the interview. We figured they had someone else listening in on the interview and giving them answers. No AI needed. One of the worrying things was that they did it fairly badly, had they done it well we probably could not have detected it. Perhaps the ideal interview now would be one in person with a computer that wasn't connected to the internet where you told a candidate exactly what tools and references they would have and then gave them a coding problem. For these jobs they were pretty heavily cloud related and also showed another probable problem, people are getting other people to get their certifications for them. If you have an AWS Associate Level cert you should be able to explain what S3 is without blinking.
  - anal_reactor 2y ago
    Fun story: I had two interviews one after another. The first interviewer asked me a basic question, and then kept drilling "ok but how does it work in more detail?". This was extremely exhausting but really refreshed my knowledge. Then the second interviewer asked me exactly the same question. I was able to give him perfect textbook answer with lots of details because of the previous interview. I also have a tendency not to look into the camera/someone's eyes when talking, I only do this when I'm listening. The interviewer accused me of cheating and ended the interview there and then.
throwawaysleep 2y ago
Mostly because it is pretty easy to use AI to cheat and if you aren’t leveraging new tools, you are falling behind. You should be using AI for interviews, AI for cover letters, bots to mass spam every remote job on LinkedIn (most of the jobs I have “applied” for in the past few weeks aren’t even dev jobs, but an application costs nothing so better safe than sorry), and all manner of other tools to play this game.
- arp242 2y ago
  This is horrible abusive behaviour people like you are the reason we can't have nice things.
  - throwawaysleep 2y ago
    It is the reason you can’t have nice things. I have nice things. If I stay ahead of the scamming curve, I will always have nice things. And that’s the problem. My action, your collective bill. Tragedy of the commons is very much an unsolved problem and the winning play has consistently been to destroy the commons through max exploitation.
    - ben_w 2y ago
      This time round it can also directly harm you, despite your self awareness of the most significant public issue with what you're doing, as shared black-lists for known spammers significantly predates LLMs.
      - throwawaysleep 2y ago
        Unless you become extremely prolific, blacklists work on matching some kind of resume info like a name or more likely an email or phone number. Those can be rotated. I specifically use a different email every job search wave just due to the tremendous volume so I would just need a new phone number and to use my middle name or something. But you are right, this is a risk.
serf 2y ago
I get the premise , but part of me asks : Well, the candidate was able to come up with an answer, why does it matter how they did it? Presumably these questions are rooted in some kind of developer reality, they're asked to gauge technical expertise and suitability.. what's the real difference between someone with a magic box that gives them the right answers versus someone who can derive the answers themselves if the questions suitably emulate real life scenarios? why should a company care about 'natural' problem solvers aside from the context of IP ownership and so on? It sounds like it really just turns application questions into a voight-kampf test for no real good reason when the real point is to ascertain whether or not a candidate can get the job done. I think this kind of stuff is just a gut reaction from a humanity that realizes that it's not the cleverness of the interview questions that are the problem, it's that machine tools are now at near human levels in the majority of mundane stuff a developer does every day -- the reactions are born from a panicked realization that it no longer makes sense to employ the lower end of the developer skill spectrum.
- throwawaysleep 2y ago
  Need to know that the dev knows when the AI is wrong.
  - cjbgkagh 2y ago
    A more direct solution would be to curate a list of mistakes made by the AI and see if the dev can correct them.
    - jachee 2y ago
      Even better would be a mixed bag of AI mistakes and correct code with no indication of which is which.
      - cjbgkagh 2y ago
        Could even have more than one mistake per question. As an aside, I really hate the 'art' of US essay writing. The AI has now learned to emulate this equivocating waffle that sits somewhere between not wrong and not even wrong. I do wonder now - with better tools available could we instead teach people to first act as an editor for other peoples content, perhaps even AI content. In effect have the people act as the discriminator in the GAN. I think it might be a faster and more thorough way of learning that embraces the help of AI while ditching the awful equivocating waffle that is currently being taught.
littlestymaar 2y ago
I did (as an interviewer) a lot of interviews last year, and I was always pretty explicit about allowing the candidate to use “whatever tools you'd use in real life, including copilot and ChatGPT”. This has two big advantages: - if someone not using them is less performant than someone using them, I should definitely hire the later, because mastering the use of these tools is an important skill, like knowing your key bindings or using a proper IDE and debugger. - it forces you to design the interview in a way that isn't trivially solved by an AI, and challenges the actual skills of the interviewees. Of course it means the interview cannot be given to brainless hiring managers or HR, but must be handled by software team leaders/managers themselves, or at least senior developers.
- Copenjin 2y ago
  You can do point 2 alone, skipping the nonsense of point 1. Please respect the time of the people that are interviewing and don't push your "innovative" ideas about interviewing on them. You are interviewing, not doing experiments.
  - littlestymaar 2y ago
    > skipping the nonsense of point 1. Please respect the time of the people that are interviewing and don't push your "innovative" ideas about interviewing on them. You are interviewing, not doing experiments. I have to admit I have no idea what you're complaining about. I've never treated the interviewees like they were experiment subjects and I don't see where you are reading that in “point 1”… All I'm saying in the first bullet point is that weather you like them or not, these tools are here to stay if they improve developers productivity and it would make no sense not to take the mastery of a tool as an asset when evaluating a candidate (like if someone is showing that they are proficient with git or a debugger during the interview question, that's a good thing, if they show they are proficient with Copilot or whatever AI-based tool then it's also a valuable skill that you should have in mind when making your evaluation. That's it. There's nothing here about making people lose their time, experimenting with them or whatever.
    - 2y ago
Copenjin 2y ago
If they interview for someone able to solve leetcode hards on the spot, that's what they will get. I hope for them that they they don't focus on this (j/k, we already know that some do).
janalsncm 2y ago
Can anyone confirm the premise of the article? Are interviews actually harder? And if so, maybe it’s because a lot of companies are “hiring” but not really. They’ll hire a senior for a junior position but otherwise just keep waiting for the “right fit”.
- Onewildgamer 2y ago
  I was hoping I'm not alone, I had mixed signals reading the article. OP talks about how the interviews are harder in 2024 and says candidates are using AI to crack impossible time bound programming tasks and instead recommend companies to do the interview online. Somewhere something is disconnected
- seanhunter 2y ago
  > Can anyone confirm the premise of the article? Are interviews actually harder? No interviewing is absolutely not harder (from the perspective of the hiring company). Have been interviewing candidates or hiring manager for 25+ years at this point. It's always been hard to run a good interview process or do a good interview [1] and hard to hire good candidates no matter how good the process (because good people are rare and always in demand). It doesn't appear to me to be harder to interview now than it used to be. If anything it's a bit easier than during the dotcom bubble for example but for different reasons. Now I get fewer candidates in total but more candidates who are at or close to the skill bar, so discernment is harder but the consequences of making a suboptimal choice are not quite as extreme (because a lot of candidates are pretty close to each other), whereas then I would be spammed with a deluge of candidates the vast majority of which were totally hopeless and extremely few of which were competent at all so the temptation to lower the bar was extremely high. [1] ie the task is a challenging one. It requires difficult tradeoffs, and time, skill and practise and application on the part of the interviewer. There is no one-size-fits all method which works for all roles, candidates and organizations. Interview processes tend also to degrade over time so any time when you think it's going well is a local maximum and it will slide from there and you'll need to rethink, change it up etc in a month or two.
fzeroracer 2y ago
If the enshittification of the Internet by the proliferation of AI results in the destruction of the awful pop quiz style coding interviews then at least it will have had some value.
nu2ycombinator 2y ago
Post seems more of a marketing for UltraCode than establishing the connection between AI and Interviews getting harder. Even before AI there were instances where Leetcode Hard questions were asked in interviews that are highly unlikely to solve in 30 mins, unless seen that Q before.
nu2ycombinator 2y ago
I think purpose of asking Leetcode hard questions in interviews is to reject if someone solves on the spot, and proceed with someone made reasonable progress and well thought out about the problem
trustno2 2y ago
I'm not a great programmer, but I think I'm quite good, I have contributions in quite a lot of foss projects, I have CS masters, yet I have no idea how would I even start these FAANG problems. In 30 minutes? Who is this optimizing for? I worked in one "unicorn" once and all I did was getting protobufs from one API, putting it to database and taking it back from database and putting it to different API; it was so boring. I solved my boredom by contributing to some in-house framework, after which I was told I am out of my line and should go back to copying data between APIs. Are people at Facebook actually solving these hard CS problems in daily life? Why are these interviews even a thing?
- jacknews 2y ago
  Exactly. The typical day-day is always something like 'munge the data from this input stream so it can be processed by this analysis service and then uploaded to snowflake' or whatever. Very rare you need any DSA (other than being aware of memory/performance implications of certain datastructures, etc) let alone DP.
- weatherlite 2y ago
  To be fair the ones who do succeed in those FAANG interviews are probably pretty smart, good under pressure and know algorithms / data structures very well. I'd have a hard time passing these interviews even with a lot of preparation - I don't excel under pressure nor do I have the will to prepare for those interviews for months (which is what it would take for me to reach 'FAANG' level interview taking and I'm being generous here, it could take me more than months). The vast majority of candidates are like me - some combination of time constraints and lack of superior intelligence will prevent them for excelling in those interviews or even trying. I'm just not in the top 2% of devs in terms of intelligence and am not particularly great in charming my way into an offer, nor am I a member of a protected group / DEI. So knowing my chances are low to begin with I'm pretty much fine with deciding FAANG is not very realistic for me.
  - DEADMINCE 2y ago
    > The vast majority of candidates are like me - some combination of time constraints and lack of superior intelligence will prevent them for excelling in those interviews or even trying.The vast majority of candidates are like me - some combination of time constraints and lack of superior intelligence will prevent them for excelling in those interviews or even trying. Surely that's an attitude thing more than a true limitation? Just because the hill is steeper for you doesn't mean you can't still get to the top.
koonsolo 2y ago
I conduct a lot of interviews for front-end developers. And I now settled on something that works well. Step 1 is an online HackerRank test that takes about 30 minutes. Its purpose is to filter out the really bad candidates (if you never did interviews, you can't imagine how bad the majority of candidates are). Step 2 is a remote video interview where I go over some code that I write step by step, and then have all these weird bugs that they need to solve. I basically test for "A junior has this strange problem, can you figure out what it is?". Practically it tests knowledge of the event loop, closures, event propagation, React peculiarities, ... . We do remote video interviews, and sometimes I have candidates that are obviously cheating. It's always funny how chaotic and incomprehensible their explanations are. And in the end, don't even come to a solution. I never liked homework tasks, because you can always cheat, for example your spouse might be an awesome coder. If any experienced interviewer has more tips, I'm always happy to hear them :).
doreenkevin825 2y ago
[dead]
dave333 2y ago
Fifty years ago there was a big debate about whether or not pocket electronic calculators should be allowed in exams since they would enable people with poor arithmetic skills. Slide rules were allowed since there is some skill using a slide rule. If someone is smart enough to harness AI to cheat on an interview, they are probably worth hiring, unless there is concern about dishonesty, eg in a bank.
test00192 2y ago
Try incruiter.com/incbot
test00192 2y ago
[dead]
mandarjog 2y ago
When you have a calculator, what is the point of testing people on how fast they can do complex math with paper and pencil. I think same thing applies here. The interview process must evolve to factor in gen ai. If a candidate is effective coding with gen-ai, does it matter that they are not effective without it ? Several people have mentioned code reviews, +1 to that.
nunez 2y ago
Honestly, I miss all-day _actually on-site_ on sites. Getting on site and meeting people definitely took the edge off of what were otherwise very hard interview rounds.
OutOfHere 2y ago
Instead of these interviews, consider this approach: Ask the applicant to provide links to their own open projects and/or to merged PRs they submitted to others open projects. Over each submitted relevant link, run a script that computes: `log(stars) * git_diff_length`. For one's own projects, git_diff_length is relative to an empty repo. If a link is not relevant to the position, do not consider it. For each applicant, use the sum of the top-scoring three links. This is assuming that the stars are not fake which is something you have to evaluate by seeing if the starring users have unique repos of their own or not.
ashleygeorge 2y ago
[flagged]

73 points by prakhar897 2y ago

It doesn't matter if AI makes interviews harder. An utterly broken system is not practically worse when it becomes more broken. We were already well past the point of futility before AI became part of the system. The only real way to get a job is by the equivalent of mass spamming and luck (which also means low chance of fit if hired) or by human connection and familiarity (the one true way for anything in life).

cranberryturkey 2y ago
I just landed a contract today after 9 months of not working. It was dumb luck.
ghaff 2y ago
>by human connection and familiarity (the one true way for anything in life) Every job I've had since grad school was through personal connections. (And luck.) I suspect the state of SWE (and adjacent roles) for the past decade+ leads to a lot of SWEs believing that sending out a few applications will result in a great offer in a week. I'm not sure how much AI has to do with making spray and pray less effective or enabling various cheating in interviews--presumably leading at some point to only allowing in-person interviews after an initial screen--but it probably decreases the time spent on the initial filter and you start relying more on heuristics like where they went to school.
- nicbou 2y ago
  The problem with connections is that some people have none. I help immigrants for a living, and the biggest underlying challenge for recent immigrants is the lack of a network. They truly start from scratch, so for a year or two they are often completely alone. Networking is absurdly effective, since all nodes stake their reputation every time they recommend another node. However it only helps those in the network. This leaves a lot of good candidates out. It can end up creating sort of exclusive club.
classified 2y ago
Agreed, it's a total gamble, but even in a game of luck there can be factors that make your odds even worse.

baron816 2y ago

I wish more companies would have interviewees conduct code reviews. Code reviews as an interview show a number of things you wouldn’t get from a typical interview—what opinions they have, what are the things they call out vs what they don’t waste time on, how they might communicate with another teammate, and more. And if we’re going to a world where AIs do much of the work and we just need to check that they implemented what we intended, those code review skill will still be highly relevant.

nunez 2y ago
I did a code review style interview at a startup a few years ago (loved them; got an offer, but my then-current employer countered surprisingly well). I really enjoyed it.
kragen 2y ago
gpt-4 is at least as good as i am at code reviews, so i don't think this solves the problem this post is about
- Copenjin 2y ago
  The objective is not finding someone good at reviews, or even that gets a specific review "right". It's having a chance to understand how someone thinks, better evaluate their actual knowledge and test for some important red flags.
  - kragen 2y ago
    gpt-4 is only mediocre at many things, and i've found mistakes in its code reviews (not, to be sure, mistakes i couldn't have made), but one thing it's absolutely superhuman at is avoiding red flags
    - godelski 2y ago
      > gpt-4 is only mediocre at many things, and i've found mistakes in its code reviews >> gpt-4 is at least as good as i am at code reviews, so i don't think this solves the problem this post is about These two comments don't seem consistent. Honestly, "i've found mistakes in its code reviews" vs "it's absolutely superhuman at is avoiding red flags" is possibly not self-consistent. But I think you mean glaring mistakes? Which if I understand you correctly, than I'm not sure how you get the first conclusion unless you are saying you're mediocre (which is fine). But really it just makes it seem like you and the machine complement one another, rather than compete, which makes me not understand the original comment in context. But I mostly agree with Copenjin. Interviews are less about on the spot skill checks so much as learning how someone thinks and problem solves. Honestly, you could play a boardgame/videogame/cardgame with them and it would be an effective interview (asking them relevant questions during the game, and maybe even better if it's a relatively unknown game so they can zero shot it). The reason for this is that in real world work you are more concerned with how someone adapts to changing environments and thinks through situations. To see when they'll ask for help, what they might get stuck on, and how they strategize. Your employees will always be gaining new skills. And honestly, it is easier to take a lower skilled person who's more adaptable and driven and turn them into a great employee than it is to take someone who's got skills but will stagnate. But ymmv depending on the job and requirements. Sometimes you just need to fill a seat.

infotainment 2y ago

Prior to the pandemic, in-person interviews were the norm at big tech, but they never really went back to them afterward. I wonder if this could be the beginning of the end for purely remote interviewing; seems like in-person interviews would be less likely to be gamed in this way.

weatherlite 2y ago
Yep I agree I think its coming to an end unless some good and reliable counter cheating tools are developed.
sien 2y ago
This exactly. Before the pandemic the team I was in used to conduct interviews in person. We'd give people a sheet of paper with a bunch of questions that they had to answer. No leetcode type stuff. Just basics. With that we had one person cheat with a phone. Doing remote interviews from then on we saw at least one person who was very probably cheating. They would get an answer, pause, then give a response, but then when we asked follow up questions they would seem not to understand their previous answer. They also had 'Audio Visual Issues' at the start of the interview. We figured they had someone else listening in on the interview and giving them answers. No AI needed. One of the worrying things was that they did it fairly badly, had they done it well we probably could not have detected it. Perhaps the ideal interview now would be one in person with a computer that wasn't connected to the internet where you told a candidate exactly what tools and references they would have and then gave them a coding problem. For these jobs they were pretty heavily cloud related and also showed another probable problem, people are getting other people to get their certifications for them. If you have an AWS Associate Level cert you should be able to explain what S3 is without blinking.
- anal_reactor 2y ago
  Fun story: I had two interviews one after another. The first interviewer asked me a basic question, and then kept drilling "ok but how does it work in more detail?". This was extremely exhausting but really refreshed my knowledge. Then the second interviewer asked me exactly the same question. I was able to give him perfect textbook answer with lots of details because of the previous interview. I also have a tendency not to look into the camera/someone's eyes when talking, I only do this when I'm listening. The interviewer accused me of cheating and ended the interview there and then.

throwawaysleep 2y ago

Mostly because it is pretty easy to use AI to cheat and if you aren’t leveraging new tools, you are falling behind. You should be using AI for interviews, AI for cover letters, bots to mass spam every remote job on LinkedIn (most of the jobs I have “applied” for in the past few weeks aren’t even dev jobs, but an application costs nothing so better safe than sorry), and all manner of other tools to play this game.

arp242 2y ago
This is horrible abusive behaviour people like you are the reason we can't have nice things.
- throwawaysleep 2y ago
  It is the reason you can’t have nice things. I have nice things. If I stay ahead of the scamming curve, I will always have nice things. And that’s the problem. My action, your collective bill. Tragedy of the commons is very much an unsolved problem and the winning play has consistently been to destroy the commons through max exploitation.
  - ben_w 2y ago
    This time round it can also directly harm you, despite your self awareness of the most significant public issue with what you're doing, as shared black-lists for known spammers significantly predates LLMs.
    - throwawaysleep 2y ago
      Unless you become extremely prolific, blacklists work on matching some kind of resume info like a name or more likely an email or phone number. Those can be rotated. I specifically use a different email every job search wave just due to the tremendous volume so I would just need a new phone number and to use my middle name or something. But you are right, this is a risk.

serf 2y ago

I get the premise , but part of me asks : Well, the candidate was able to come up with an answer, why does it matter how they did it? Presumably these questions are rooted in some kind of developer reality, they're asked to gauge technical expertise and suitability.. what's the real difference between someone with a magic box that gives them the right answers versus someone who can derive the answers themselves if the questions suitably emulate real life scenarios? why should a company care about 'natural' problem solvers aside from the context of IP ownership and so on? It sounds like it really just turns application questions into a voight-kampf test for no real good reason when the real point is to ascertain whether or not a candidate can get the job done. I think this kind of stuff is just a gut reaction from a humanity that realizes that it's not the cleverness of the interview questions that are the problem, it's that machine tools are now at near human levels in the majority of mundane stuff a developer does every day -- the reactions are born from a panicked realization that it no longer makes sense to employ the lower end of the developer skill spectrum.

throwawaysleep 2y ago
Need to know that the dev knows when the AI is wrong.
- cjbgkagh 2y ago
  A more direct solution would be to curate a list of mistakes made by the AI and see if the dev can correct them.
  - jachee 2y ago
    Even better would be a mixed bag of AI mistakes and correct code with no indication of which is which.
    - cjbgkagh 2y ago
      Could even have more than one mistake per question. As an aside, I really hate the 'art' of US essay writing. The AI has now learned to emulate this equivocating waffle that sits somewhere between not wrong and not even wrong. I do wonder now - with better tools available could we instead teach people to first act as an editor for other peoples content, perhaps even AI content. In effect have the people act as the discriminator in the GAN. I think it might be a faster and more thorough way of learning that embraces the help of AI while ditching the awful equivocating waffle that is currently being taught.

littlestymaar 2y ago

I did (as an interviewer) a lot of interviews last year, and I was always pretty explicit about allowing the candidate to use “whatever tools you'd use in real life, including copilot and ChatGPT”. This has two big advantages: - if someone not using them is less performant than someone using them, I should definitely hire the later, because mastering the use of these tools is an important skill, like knowing your key bindings or using a proper IDE and debugger. - it forces you to design the interview in a way that isn't trivially solved by an AI, and challenges the actual skills of the interviewees. Of course it means the interview cannot be given to brainless hiring managers or HR, but must be handled by software team leaders/managers themselves, or at least senior developers.

Copenjin 2y ago
You can do point 2 alone, skipping the nonsense of point 1. Please respect the time of the people that are interviewing and don't push your "innovative" ideas about interviewing on them. You are interviewing, not doing experiments.
- littlestymaar 2y ago
  > skipping the nonsense of point 1. Please respect the time of the people that are interviewing and don't push your "innovative" ideas about interviewing on them. You are interviewing, not doing experiments. I have to admit I have no idea what you're complaining about. I've never treated the interviewees like they were experiment subjects and I don't see where you are reading that in “point 1”… All I'm saying in the first bullet point is that weather you like them or not, these tools are here to stay if they improve developers productivity and it would make no sense not to take the mastery of a tool as an asset when evaluating a candidate (like if someone is showing that they are proficient with git or a debugger during the interview question, that's a good thing, if they show they are proficient with Copilot or whatever AI-based tool then it's also a valuable skill that you should have in mind when making your evaluation. That's it. There's nothing here about making people lose their time, experimenting with them or whatever.
  - 2y ago

Copenjin 2y ago

If they interview for someone able to solve leetcode hards on the spot, that's what they will get. I hope for them that they they don't focus on this (j/k, we already know that some do).

janalsncm 2y ago

Can anyone confirm the premise of the article? Are interviews actually harder? And if so, maybe it’s because a lot of companies are “hiring” but not really. They’ll hire a senior for a junior position but otherwise just keep waiting for the “right fit”.

Onewildgamer 2y ago
I was hoping I'm not alone, I had mixed signals reading the article. OP talks about how the interviews are harder in 2024 and says candidates are using AI to crack impossible time bound programming tasks and instead recommend companies to do the interview online. Somewhere something is disconnected
seanhunter 2y ago
> Can anyone confirm the premise of the article? Are interviews actually harder? No interviewing is absolutely not harder (from the perspective of the hiring company). Have been interviewing candidates or hiring manager for 25+ years at this point. It's always been hard to run a good interview process or do a good interview [1] and hard to hire good candidates no matter how good the process (because good people are rare and always in demand). It doesn't appear to me to be harder to interview now than it used to be. If anything it's a bit easier than during the dotcom bubble for example but for different reasons. Now I get fewer candidates in total but more candidates who are at or close to the skill bar, so discernment is harder but the consequences of making a suboptimal choice are not quite as extreme (because a lot of candidates are pretty close to each other), whereas then I would be spammed with a deluge of candidates the vast majority of which were totally hopeless and extremely few of which were competent at all so the temptation to lower the bar was extremely high. [1] ie the task is a challenging one. It requires difficult tradeoffs, and time, skill and practise and application on the part of the interviewer. There is no one-size-fits all method which works for all roles, candidates and organizations. Interview processes tend also to degrade over time so any time when you think it's going well is a local maximum and it will slide from there and you'll need to rethink, change it up etc in a month or two.

fzeroracer 2y ago

If the enshittification of the Internet by the proliferation of AI results in the destruction of the awful pop quiz style coding interviews then at least it will have had some value.

nu2ycombinator 2y ago

Post seems more of a marketing for UltraCode than establishing the connection between AI and Interviews getting harder. Even before AI there were instances where Leetcode Hard questions were asked in interviews that are highly unlikely to solve in 30 mins, unless seen that Q before.

nu2ycombinator 2y ago

I think purpose of asking Leetcode hard questions in interviews is to reject if someone solves on the spot, and proceed with someone made reasonable progress and well thought out about the problem

trustno2 2y ago

I'm not a great programmer, but I think I'm quite good, I have contributions in quite a lot of foss projects, I have CS masters, yet I have no idea how would I even start these FAANG problems. In 30 minutes? Who is this optimizing for? I worked in one "unicorn" once and all I did was getting protobufs from one API, putting it to database and taking it back from database and putting it to different API; it was so boring. I solved my boredom by contributing to some in-house framework, after which I was told I am out of my line and should go back to copying data between APIs. Are people at Facebook actually solving these hard CS problems in daily life? Why are these interviews even a thing?

jacknews 2y ago
Exactly. The typical day-day is always something like 'munge the data from this input stream so it can be processed by this analysis service and then uploaded to snowflake' or whatever. Very rare you need any DSA (other than being aware of memory/performance implications of certain datastructures, etc) let alone DP.
weatherlite 2y ago
To be fair the ones who do succeed in those FAANG interviews are probably pretty smart, good under pressure and know algorithms / data structures very well. I'd have a hard time passing these interviews even with a lot of preparation - I don't excel under pressure nor do I have the will to prepare for those interviews for months (which is what it would take for me to reach 'FAANG' level interview taking and I'm being generous here, it could take me more than months). The vast majority of candidates are like me - some combination of time constraints and lack of superior intelligence will prevent them for excelling in those interviews or even trying. I'm just not in the top 2% of devs in terms of intelligence and am not particularly great in charming my way into an offer, nor am I a member of a protected group / DEI. So knowing my chances are low to begin with I'm pretty much fine with deciding FAANG is not very realistic for me.
- DEADMINCE 2y ago
  > The vast majority of candidates are like me - some combination of time constraints and lack of superior intelligence will prevent them for excelling in those interviews or even trying.The vast majority of candidates are like me - some combination of time constraints and lack of superior intelligence will prevent them for excelling in those interviews or even trying. Surely that's an attitude thing more than a true limitation? Just because the hill is steeper for you doesn't mean you can't still get to the top.

koonsolo 2y ago

I conduct a lot of interviews for front-end developers. And I now settled on something that works well. Step 1 is an online HackerRank test that takes about 30 minutes. Its purpose is to filter out the really bad candidates (if you never did interviews, you can't imagine how bad the majority of candidates are). Step 2 is a remote video interview where I go over some code that I write step by step, and then have all these weird bugs that they need to solve. I basically test for "A junior has this strange problem, can you figure out what it is?". Practically it tests knowledge of the event loop, closures, event propagation, React peculiarities, ... . We do remote video interviews, and sometimes I have candidates that are obviously cheating. It's always funny how chaotic and incomprehensible their explanations are. And in the end, don't even come to a solution. I never liked homework tasks, because you can always cheat, for example your spouse might be an awesome coder. If any experienced interviewer has more tips, I'm always happy to hear them :).

doreenkevin825 2y ago

[dead]

dave333 2y ago

Fifty years ago there was a big debate about whether or not pocket electronic calculators should be allowed in exams since they would enable people with poor arithmetic skills. Slide rules were allowed since there is some skill using a slide rule. If someone is smart enough to harness AI to cheat on an interview, they are probably worth hiring, unless there is concern about dishonesty, eg in a bank.

test00192 2y ago

Try incruiter.com/incbot

test00192 2y ago

[dead]

mandarjog 2y ago

When you have a calculator, what is the point of testing people on how fast they can do complex math with paper and pencil. I think same thing applies here. The interview process must evolve to factor in gen ai. If a candidate is effective coding with gen-ai, does it matter that they are not effective without it ? Several people have mentioned code reviews, +1 to that.

nunez 2y ago

Honestly, I miss all-day _actually on-site_ on sites. Getting on site and meeting people definitely took the edge off of what were otherwise very hard interview rounds.

OutOfHere 2y ago

Instead of these interviews, consider this approach: Ask the applicant to provide links to their own open projects and/or to merged PRs they submitted to others open projects. Over each submitted relevant link, run a script that computes: `log(stars) * git_diff_length`. For one's own projects, git_diff_length is relative to an empty repo. If a link is not relevant to the position, do not consider it. For each applicant, use the sum of the top-scoring three links. This is assuming that the stars are not fake which is something you have to evaluate by seeing if the starring users have unique repos of their own or not.

ashleygeorge 2y ago

[flagged]