5 ms·

The key is having interviewers that know what they are talking about so in-depth meandering discussions can be had regarding personal and work projects which us

The key is having interviewers that know what they are talking about so in-depth meandering discussions can be had regarding personal and work projects which usually makes it clear whether the applicant knows what they are talking about. Leetcode was only ever a temporary interview technique, and this 'AI' prominence in the public domain has simply sped up it's demise.

danielbln 2y ago
This is the way. We do an intro call, an engineering chat (exactly as you describe), a coding challenge and 2 team chat sessions in person. At the end of that, we usually have a good feeling about how sharp the candidate is, of they like to learn and discover new things, what their work ethic is. It's not bullet proof, but it removes a lot of noise from the signal. The coding challenge is supposed to be solved with AI. We can no longer afford not to use LLMs for engineering, as it's that much of a productivity boost when used right, so candidates should show how they use LLMs. They need to be able to explain the code of course, and answer questions about it, but for us it's a negative mark of a candidate proclaims that they don't use LLMs.
- satvikpendem 2y ago
  > The coding challenge is supposed to be solved with AI. We can no longer afford not to use LLMs for engineering, as it's that much of a productivity boost when used right, so candidates should show how they use LLMs. They need to be able to explain the code of course, and answer questions about it, but for us it's a negative mark of a candidate proclaims that they don't use LLMs. Do you state this upfront or is it some hidden requirement? Generally I'd expect an interview coding exercise to not be done with AI, but if it's a hidden requirement that the interviewer does not disclose, it is unfair to be penalized for not reading their minds.
  - ktallett 2y ago
    I would say as long as it is stated you can complete the coding exercise using any tool available it is fine. I do agree, no task should be a trick. I am personally of the view you should be able to use search engines, AI, anything you want, as the task should be representative of doing the task in person. The key focus has to be the programmer's knowledge and why they did what they did.
    - trelliscoded 2y ago
      One client of mine has a couple repositories for non-mission critical things like their fork of an open source project, decommissioned microservices, a SVG generator for their web front-end, etc. They also take this approach of "whatever tool works," but their coding test is "here's some symptoms of the SVG generator misbehaving, figure out what happened and fix it," which requires digging into the commit history, issues, actually looking at the SVG output, etc. Once you've figured out how the system architecture works, and the most likely component to be causing the problem, you have to convert part of the code to use a newer, undocumented API exposed by a RPC server that speaks a serialization format that no LLM has ever seen before. Doing this is actually way faster and accurate using an AI, if you know how to centaur with it and make sure the output is tested to be correct. This is a much more representative test of how someone's going to handle doing actual work knocking issues out.
      - johnnyanmac 2y ago
        That's interesting and effective. But I do feel like "undocumented API" is an unnecessary trick in an interview setting.
    - CamperBob2 2y ago
      Reminds me of the old joke/story where the Caltech student asks, "Can we use Feynman in this open-book exam?"
  - danielbln 2y ago
    Well, the challenge involves using a python LLM framework to build a simple RAG system for recipes. It's not a hidden requirement per se to use LLM assistance, but the candidate should have a good answer ready why they didn't use an LLM to solve the challenge.
    - mvdtnz 2y ago
      Ah so you expect mind readers who can divine something from your brain that goes against 99.99% of interviewers' practices and would get them instantly disqualified from an overwhelming majority of interviews. Nice work good luck finding candidates.
      - satvikpendem 2y ago
        Indeed, looks like it is just an unspoken rule and an interview trick after all. I would not want to interview with this person, much less work with them.
    - what 2y ago
      Why is it a negative that the candidate can solve the challenge without using an LLM? I don’t really understand this. Also, what is a good answer for not using one? Will you provide access to one during the course of the interview? Or I am just expected to be paying for one?
      - danielbln 2y ago
        It's not negative that the candidate can solve it without an LLM, but it is positive if the candidate can use the LLM to speed up the solution. The code challenge is timeboxed. We are providing an API key for LLM inference, as implementing the challenge requires this as well. And I haven't heard a good answer yet for not using one, ideally the candidate knows how to mitigate the drawbacks of LLMs while benefiting from their utility regardless.
        what 2y ago
- crooked-v 2y ago
  > as it's that much of a productivity boost when used right Frankly, if an interviewer told me this, I would genuinely wonder why what they're building is such a simple toy product that an LLM can understand it well enough to be productive.
_puk 2y ago
This completely.. You ask a rote question and you'll get a rote answer while the interviewee is busy looking at a fixed point on the screen. You then ask a pointed question about something they know or care about, and suddenly their face lights up, they're animated, and they are looking around. It's a huge tell.
- crooked-v 2y ago
  You know, this makes me wonder if a viable remote interview technique, at least until real-time deepfaking gets better, would be to have people close their eyes while talking to them. For somebody who knows their stuff it'll have zero impact; for someone relying entirely on GPT, it will completely derail them.
  - deleted 2y ago
    [deleted]
  - ickelbawd 2y ago
    That’s an interesting idea. Sadly I think the next AI interviewing tool to be developed in response would make you look like your eyes are closed. But in the interim period it could be an interesting way to interview. Doesn’t really help for technical interviews where they kinda need to have their eyes open, but for pre-screens maybe…
  - steve_taylor 2y ago
    A filter could probably do it already. There are already filters to make you appear to be looking at the camera no matter where your eyes are pointing.
  - wiether 2y ago
    > For somebody who knows their stuff it'll have zero impact I just tried and it's: hard. It feels like being ask to keep one's breath like writing something. I need to focus too much on keeping my eyes closed, I don't have enough bandwith left to thing about anything relevant.
soheil 2y ago
its demise

danielbln 2y ago

This is the way. We do an intro call, an engineering chat (exactly as you describe), a coding challenge and 2 team chat sessions in person. At the end of that, we usually have a good feeling about how sharp the candidate is, of they like to learn and discover new things, what their work ethic is. It's not bullet proof, but it removes a lot of noise from the signal. The coding challenge is supposed to be solved with AI. We can no longer afford not to use LLMs for engineering, as it's that much of a productivity boost when used right, so candidates should show how they use LLMs. They need to be able to explain the code of course, and answer questions about it, but for us it's a negative mark of a candidate proclaims that they don't use LLMs.

satvikpendem 2y ago
> The coding challenge is supposed to be solved with AI. We can no longer afford not to use LLMs for engineering, as it's that much of a productivity boost when used right, so candidates should show how they use LLMs. They need to be able to explain the code of course, and answer questions about it, but for us it's a negative mark of a candidate proclaims that they don't use LLMs. Do you state this upfront or is it some hidden requirement? Generally I'd expect an interview coding exercise to not be done with AI, but if it's a hidden requirement that the interviewer does not disclose, it is unfair to be penalized for not reading their minds.
- ktallett 2y ago
  I would say as long as it is stated you can complete the coding exercise using any tool available it is fine. I do agree, no task should be a trick. I am personally of the view you should be able to use search engines, AI, anything you want, as the task should be representative of doing the task in person. The key focus has to be the programmer's knowledge and why they did what they did.
  - trelliscoded 2y ago
    One client of mine has a couple repositories for non-mission critical things like their fork of an open source project, decommissioned microservices, a SVG generator for their web front-end, etc. They also take this approach of "whatever tool works," but their coding test is "here's some symptoms of the SVG generator misbehaving, figure out what happened and fix it," which requires digging into the commit history, issues, actually looking at the SVG output, etc. Once you've figured out how the system architecture works, and the most likely component to be causing the problem, you have to convert part of the code to use a newer, undocumented API exposed by a RPC server that speaks a serialization format that no LLM has ever seen before. Doing this is actually way faster and accurate using an AI, if you know how to centaur with it and make sure the output is tested to be correct. This is a much more representative test of how someone's going to handle doing actual work knocking issues out.
    - johnnyanmac 2y ago
      That's interesting and effective. But I do feel like "undocumented API" is an unnecessary trick in an interview setting.
  - CamperBob2 2y ago
    Reminds me of the old joke/story where the Caltech student asks, "Can we use Feynman in this open-book exam?"
- danielbln 2y ago
  Well, the challenge involves using a python LLM framework to build a simple RAG system for recipes. It's not a hidden requirement per se to use LLM assistance, but the candidate should have a good answer ready why they didn't use an LLM to solve the challenge.
  - mvdtnz 2y ago
    Ah so you expect mind readers who can divine something from your brain that goes against 99.99% of interviewers' practices and would get them instantly disqualified from an overwhelming majority of interviews. Nice work good luck finding candidates.
    - satvikpendem 2y ago
      Indeed, looks like it is just an unspoken rule and an interview trick after all. I would not want to interview with this person, much less work with them.
  - what 2y ago
    Why is it a negative that the candidate can solve the challenge without using an LLM? I don’t really understand this. Also, what is a good answer for not using one? Will you provide access to one during the course of the interview? Or I am just expected to be paying for one?
    - danielbln 2y ago
      It's not negative that the candidate can solve it without an LLM, but it is positive if the candidate can use the LLM to speed up the solution. The code challenge is timeboxed. We are providing an API key for LLM inference, as implementing the challenge requires this as well. And I haven't heard a good answer yet for not using one, ideally the candidate knows how to mitigate the drawbacks of LLMs while benefiting from their utility regardless.
      - what 2y ago
crooked-v 2y ago
> as it's that much of a productivity boost when used right Frankly, if an interviewer told me this, I would genuinely wonder why what they're building is such a simple toy product that an LLM can understand it well enough to be productive.

_puk 2y ago

This completely.. You ask a rote question and you'll get a rote answer while the interviewee is busy looking at a fixed point on the screen. You then ask a pointed question about something they know or care about, and suddenly their face lights up, they're animated, and they are looking around. It's a huge tell.

crooked-v 2y ago
You know, this makes me wonder if a viable remote interview technique, at least until real-time deepfaking gets better, would be to have people close their eyes while talking to them. For somebody who knows their stuff it'll have zero impact; for someone relying entirely on GPT, it will completely derail them.
- deleted 2y ago
  [deleted]
- ickelbawd 2y ago
  That’s an interesting idea. Sadly I think the next AI interviewing tool to be developed in response would make you look like your eyes are closed. But in the interim period it could be an interesting way to interview. Doesn’t really help for technical interviews where they kinda need to have their eyes open, but for pre-screens maybe…
- steve_taylor 2y ago
  A filter could probably do it already. There are already filters to make you appear to be looking at the camera no matter where your eyes are pointing.
- wiether 2y ago
  > For somebody who knows their stuff it'll have zero impact I just tried and it's: hard. It feels like being ask to keep one's breath like writing something. I need to focus too much on keeping my eyes closed, I don't have enough bandwith left to thing about anything relevant.

soheil 2y ago

its demise