4 ms·

This is quite a conundrum. These AI companies thrive on the idea that very soon people will not be replaced by AI, but by people who can effectively use AI to b

This is quite a conundrum. These AI companies thrive on the idea that very soon people will not be replaced by AI, but by people who can effectively use AI to be 10x more productive. If AI turns a normal coder into a 10x dev, then why wouldn't you want to see that during an interview? Especially since cheating this whole interview system has become trivial in the past months. It's not the applicants that are the problem, it's the outdated way of doing interviews.

dailykoder 2y ago
Very very true! Give them a take home assignment first and if they have a good result on that, give them an easier task, without AI, in person. Then you will quickly figure out who actually understands their work
sifex 2y ago
Because as someone who’s interviewing, I know you can use AI — anyone can. It likely obscures me from judging the pitfalls, and design and architecture decisions that are required in proper engineering roles. Especially for senior and above applications, I want to make an assessment of how you think about problems, where it gives a chance for the candidate to show their experience, their technical understanding, and their communication skills. We don’t want to work with AI, we are going to pay the person for the persons time, and we want to employ someone who isn’t switching off half their cognition when a hard problem approaches.
- nejsjsjsbsb 2y ago
  Then they shouldn't use libraries, open source code or even existing compilers. They shouldn't search online (man pages is OK). They should use git plumbing commands and sh (not bash of zsh). They should not have potable water in there house but distill river water.
  - porridgeraisin 2y ago
    There is a balance to be struck. You obviously don't expect a SWE to begin by identifying rare earth metal mining spots on his first day. Where the line is drawn is context dependent, drawing the same single line for all possible situations is not possible and it's stupid to do so.
- lukan 2y ago
  No, not everyone can really use AI to deliver something that works. And ultimately, this is what this is about, right? Delivering working products.
  - miningape 2y ago
    > No, not everyone can really use AI to deliver something that works "That works" is doing a lot of heavy lifting here, and really depends more on the technical skills of the person. Because, shocker, AI doesn't magically make you good and isn't good itself. Anyone can prompt an AI for answers, it takes skill and knowledge to use those answers in something that works. By prompting AI for simple questions you don't train your skill/knowledge to answer the question yourself. Put simply, using AI makes you worse at your job - precisely when you need to be better.
    - wongarsu 2y ago
      There is also a skill in prompting the AI for the right things in the right way in the right situations. Just like everyone can use google and read documentation, but some people are a lot better at it than others. You absolutely can be a great developer who can't use AI effectively, or a mediocre developer who is very good with AI.
    - lukan 2y ago
      "Put simply, using AI makes you worse at your job - precisely when you need to be better." I don't follow. Usually jobs require deliver working things. The more efficient the worker knows his tools(like AI), the more he will deliver -> the better he is at his job. If he cannot deliver reliable working things, because he does not understand the LLM output, then he fails at delivering.
      - oneeyedpigeon 2y ago
        You cannot just reduce programming to "deliver working things", though. For some tasks, sure, "working" is all that matters. For many tasks, though, efficiency, maintainability, and other factors are important. You also need to take into account how to judge if something is "working" or not — that's not necessarily a trivial task.
- logicchains 2y ago
  >We don’t want to work with AI, we are going to pay the person for the persons time If your interview problems are representative of the work that you actually do, and an AI can do it as well as a qualified candidate, then that means that eventually you'll be out-competed by a competitor that does want to work with AI, because it's much cheaper to hire an AI. If an AI could do great at your interview problems but still suck at the job, that means your interview questions aren't very good/representative.
  - reshlo 2y ago
    Interview problems are never representative of the work that software developers do.
    - 63stack 2y ago
      Sounds like the interview process needs to be improved then?
  - sifex 2y ago
    It’s more like “can you design something with the JWT flow” and the candidates scramble to learn what JWT is in the interview to impress us, instead of asking us to remind them on the specifics on JWT. They then get it wrong, and waste the interviewers time.
- 63stack 2y ago
  I see this line of "I need to assess your thinking, not the AI's" thinking so often from people who claim they are interviewing, but they never recognize the elephant in the room for some reason. If people can AI their way into the position you are advertising, then at least one of the following two things have to be true: 1) the job you are advertising can be _literally_ solved by AI 2) you are not tailoring your interview process properly to the actual job that the candidate will need to do, hence the handwave-y "oh well harder problems will come up later that the AI will not be able to do". Focus the interview on the actual job that the AI can't do, and your worries will disappear. My impression is that the people who are crying about AI use in interviews are the same people who refuse to make an effort themselves. This is just the variation of the meme where you are asked to flip a red black tree on a whiteboard, but then you get the job, and your task is to center a button with CSS. Make an effort and focus your interview on the actual job, and if you are still worried people will AI their way into it, then what position are you even advertising? Either use the AI to solve the problem then, or admit that the AI can't solve this and stop worrying about people using it.
  - sifex 2y ago
    Right now there’s a lot of engineering that falls outside SWE — think datacenter architecture or integrations, or security operations, or designing automotive assemblies. There are also dozens of components of a job that require context windows measured in the _years_ of experience — knowing what will work at scale and what won’t, client interfacing, communicating decisions down inside organisations through to customer support networks about how they are required to operate. When we’re hiring for my role, Security Operations, I can’t have someone googling or asking AI what to do during an cyber security incident, but they can certainly use AI as much as they want when writing automations. I reject candidates at all stages for all sorts of reasons, but more and more candidates believe the job can be done with AI. If we wanted AI, we will probably go wholesale and not include the person asking for the job to do the typing for us. We’re not crying due to AI, we’re crying over the dozens of lost hours of interviews we’re having to conduct where it’s business critical that people know their stuff — engineering positions with consequences (banks, infrastructure, automotive). There isn’t space for “well I didn’t write the code”.
pydry 2y ago
If an AI can do your test better than a human in 2025 it reflects not much better on your test than if a pocket calculator could do your test better than a human in 1970. That did happen and the result from the test creators was the same back then: "we're not the problem, the machines are the problem. ban them!" In the long run it turned out that if you could cheat with a calculator though, it was just a bad test.... I think there is an unwillingness to admit that there is a skill issue here with the test creators and that if they got better at their job they wouldnt need to ban candidates from using AI. It's surprising to hear this from anthropic though.
a2128 2y ago
If the interview consists of the interviewer asking "Write (xyz)", the interviewee opening copilot and asking "Write (xyz)", and accepting the code. What was the point of the interview? Is the interviewee a genius productive 10x programmer because by using AI he just spent 1/10 the time to write the code? Sure, maybe you can say that the tasks should be complex enough that AI can't do it, but AI systems are constantly changing, collecting user prompts and training to improve on them. And sometimes the candidates aren't deep enough in the hiring process yet to justify spending significant time giving a complex task. It's just easier and more effective to just say no AI please
otabdeveloper4 2y ago
It's not a conundrum, they're selling snake oil. (Come on people, we've been through this many times already.)

dailykoder 2y ago

Very very true! Give them a take home assignment first and if they have a good result on that, give them an easier task, without AI, in person. Then you will quickly figure out who actually understands their work

sifex 2y ago

Because as someone who’s interviewing, I know you can use AI — anyone can. It likely obscures me from judging the pitfalls, and design and architecture decisions that are required in proper engineering roles. Especially for senior and above applications, I want to make an assessment of how you think about problems, where it gives a chance for the candidate to show their experience, their technical understanding, and their communication skills. We don’t want to work with AI, we are going to pay the person for the persons time, and we want to employ someone who isn’t switching off half their cognition when a hard problem approaches.

nejsjsjsbsb 2y ago
Then they shouldn't use libraries, open source code or even existing compilers. They shouldn't search online (man pages is OK). They should use git plumbing commands and sh (not bash of zsh). They should not have potable water in there house but distill river water.
- porridgeraisin 2y ago
  There is a balance to be struck. You obviously don't expect a SWE to begin by identifying rare earth metal mining spots on his first day. Where the line is drawn is context dependent, drawing the same single line for all possible situations is not possible and it's stupid to do so.
lukan 2y ago
No, not everyone can really use AI to deliver something that works. And ultimately, this is what this is about, right? Delivering working products.
- miningape 2y ago
  > No, not everyone can really use AI to deliver something that works "That works" is doing a lot of heavy lifting here, and really depends more on the technical skills of the person. Because, shocker, AI doesn't magically make you good and isn't good itself. Anyone can prompt an AI for answers, it takes skill and knowledge to use those answers in something that works. By prompting AI for simple questions you don't train your skill/knowledge to answer the question yourself. Put simply, using AI makes you worse at your job - precisely when you need to be better.
  - wongarsu 2y ago
    There is also a skill in prompting the AI for the right things in the right way in the right situations. Just like everyone can use google and read documentation, but some people are a lot better at it than others. You absolutely can be a great developer who can't use AI effectively, or a mediocre developer who is very good with AI.
  - lukan 2y ago
    "Put simply, using AI makes you worse at your job - precisely when you need to be better." I don't follow. Usually jobs require deliver working things. The more efficient the worker knows his tools(like AI), the more he will deliver -> the better he is at his job. If he cannot deliver reliable working things, because he does not understand the LLM output, then he fails at delivering.
    - oneeyedpigeon 2y ago
      You cannot just reduce programming to "deliver working things", though. For some tasks, sure, "working" is all that matters. For many tasks, though, efficiency, maintainability, and other factors are important. You also need to take into account how to judge if something is "working" or not — that's not necessarily a trivial task.
logicchains 2y ago
>We don’t want to work with AI, we are going to pay the person for the persons time If your interview problems are representative of the work that you actually do, and an AI can do it as well as a qualified candidate, then that means that eventually you'll be out-competed by a competitor that does want to work with AI, because it's much cheaper to hire an AI. If an AI could do great at your interview problems but still suck at the job, that means your interview questions aren't very good/representative.
- reshlo 2y ago
  Interview problems are never representative of the work that software developers do.
  - 63stack 2y ago
    Sounds like the interview process needs to be improved then?
- sifex 2y ago
  It’s more like “can you design something with the JWT flow” and the candidates scramble to learn what JWT is in the interview to impress us, instead of asking us to remind them on the specifics on JWT. They then get it wrong, and waste the interviewers time.
63stack 2y ago
I see this line of "I need to assess your thinking, not the AI's" thinking so often from people who claim they are interviewing, but they never recognize the elephant in the room for some reason. If people can AI their way into the position you are advertising, then at least one of the following two things have to be true: 1) the job you are advertising can be _literally_ solved by AI 2) you are not tailoring your interview process properly to the actual job that the candidate will need to do, hence the handwave-y "oh well harder problems will come up later that the AI will not be able to do". Focus the interview on the actual job that the AI can't do, and your worries will disappear. My impression is that the people who are crying about AI use in interviews are the same people who refuse to make an effort themselves. This is just the variation of the meme where you are asked to flip a red black tree on a whiteboard, but then you get the job, and your task is to center a button with CSS. Make an effort and focus your interview on the actual job, and if you are still worried people will AI their way into it, then what position are you even advertising? Either use the AI to solve the problem then, or admit that the AI can't solve this and stop worrying about people using it.
- sifex 2y ago
  Right now there’s a lot of engineering that falls outside SWE — think datacenter architecture or integrations, or security operations, or designing automotive assemblies. There are also dozens of components of a job that require context windows measured in the _years_ of experience — knowing what will work at scale and what won’t, client interfacing, communicating decisions down inside organisations through to customer support networks about how they are required to operate. When we’re hiring for my role, Security Operations, I can’t have someone googling or asking AI what to do during an cyber security incident, but they can certainly use AI as much as they want when writing automations. I reject candidates at all stages for all sorts of reasons, but more and more candidates believe the job can be done with AI. If we wanted AI, we will probably go wholesale and not include the person asking for the job to do the typing for us. We’re not crying due to AI, we’re crying over the dozens of lost hours of interviews we’re having to conduct where it’s business critical that people know their stuff — engineering positions with consequences (banks, infrastructure, automotive). There isn’t space for “well I didn’t write the code”.

pydry 2y ago

If an AI can do your test better than a human in 2025 it reflects not much better on your test than if a pocket calculator could do your test better than a human in 1970. That did happen and the result from the test creators was the same back then: "we're not the problem, the machines are the problem. ban them!" In the long run it turned out that if you could cheat with a calculator though, it was just a bad test.... I think there is an unwillingness to admit that there is a skill issue here with the test creators and that if they got better at their job they wouldnt need to ban candidates from using AI. It's surprising to hear this from anthropic though.

a2128 2y ago

If the interview consists of the interviewer asking "Write (xyz)", the interviewee opening copilot and asking "Write (xyz)", and accepting the code. What was the point of the interview? Is the interviewee a genius productive 10x programmer because by using AI he just spent 1/10 the time to write the code? Sure, maybe you can say that the tasks should be complex enough that AI can't do it, but AI systems are constantly changing, collecting user prompts and training to improve on them. And sometimes the candidates aren't deep enough in the hiring process yet to justify spending significant time giving a complex task. It's just easier and more effective to just say no AI please

otabdeveloper4 2y ago

It's not a conundrum, they're selling snake oil. (Come on people, we've been through this many times already.)