5 ms·

I'm currently running the interview process for the company I work for and I allow AI on the coding tests for the same reason I allow googling/etc: I want to s

I'm currently running the interview process for the company I work for and I allow AI on the coding tests for the same reason I allow googling/etc: I want to simulate as close to the real environment you will be coding in if you come and work here. My "rules" on AI/LLMs at work is that you can use whatever tools you want (much like SO before) but any code you commit, you are responsible for (also unchanged from before LLMs). I never want to hear "the LLM wrote that" when asked about how something works. If you can explain, modify, build on top of the code that an LLM spits out then fine by me. We don't require LLM usage (no token quotas or silly things like that) nor do we disallow LLMs. At the end of the day they are all tools. I've always found leet coding, whiteboarding, memorizing algorithms, etc to just be a silly form of hazing and a bad indicator of how well the employee will perform if you hire them. In the same way I still think my college was stupid for making us write, on paper, a C program with all the correct headers/syntax/etc for an exam or how I got a 90/100 on my final SQL exam because I didn't end my, otherwise perfect, queries with a semicolon.

freedomben 1y ago
I run our interviewing process as well and fully agree with you. An interview process that most closely mirrors "real work" gets much better results. Some of the best people we've had were really terrible at algorithms and leet code, but wrote quality code (that they tested!), had strong work ethics, were willing to be around off-hours for the occasional production outage, took ownership/accountability for their tasks, and were enjoyable people to work with/around. It's good to have at least one person on the team who is good with algorithms and stuff for those times when that comes up, but having that be a requirement for everyone on the team is an anti-pattern IMHO
- joshstrange 1y ago
  I completely agree. We've had almost every candidate (even some we _didn't_ hire) thank us for the interview process and how we went out of our way to make it stress-free and representative of the job. I'm not interested in tricking people or trying to show off "look how smart we are and how hard our test was"... puke. I want to work with solid performers who don't play games, why would I start that relationship off by playing a bunch of stupid games (silly interview tests/process)? And here's the thing, even with our somewhat easier/less-stress testing process, we still got high quality hires and have had very few cases where we have later had to let someone go due to not being able to do the job. This isn't a "hire fast, fire fast" situation.
jghn 1y ago
I agree with you that the real rubric is that I'd expect the candidate to be able to explain everything they did and why. For the most part I don't care how they got there. Was thinking about this recently in a conversation about whether or not the candidate should have a full screen share so that interviewers could see what they're looking up, etc. I realized that I'd reached the point where I now trust my fellow interviewers less than I do the candidate at these bits. Personally I think it's nice to see how people are leveraging tools, but too often I see interviewers dig candidates anyways on the specifics. I've now found this across multiple companies now has been that when coding exercises are "open book" like this that other interviewers *are* actively judging people based on what they're googling, and these days LLMing. If you're going to allow google but then fall back to "I can't believe they had to look *that* up!", that's not how it works.
- sokoloff 1y ago
  I think “how to write for loop in $LANGUAGE” is something that it’s fair to judge a candidate on if they claim (or the job requires) recent experience in $LANGUAGE.
  - jghn 1y ago
    Maybe. I've been coding professionally for a few decades now. I have always looked stuff up constantly. It's a lot more seamless now due to first search engines and later the ever improving autocomplete. To be honest, I think half the time I just do it to do it. But for instance I bounce between languages so often that I can never remember the fine details of syntax. To use your example, things like for loops, matches, and the like are things that I tend to double check. And that's the point. Everyone has their quirks, and if you're going to claim you want them to work they way they normally work, who cares? If they're able to turn out quality code at a reasonable clip, who cares what they need to look up.

freedomben 1y ago

I run our interviewing process as well and fully agree with you. An interview process that most closely mirrors "real work" gets much better results. Some of the best people we've had were really terrible at algorithms and leet code, but wrote quality code (that they tested!), had strong work ethics, were willing to be around off-hours for the occasional production outage, took ownership/accountability for their tasks, and were enjoyable people to work with/around. It's good to have at least one person on the team who is good with algorithms and stuff for those times when that comes up, but having that be a requirement for everyone on the team is an anti-pattern IMHO

joshstrange 1y ago
I completely agree. We've had almost every candidate (even some we _didn't_ hire) thank us for the interview process and how we went out of our way to make it stress-free and representative of the job. I'm not interested in tricking people or trying to show off "look how smart we are and how hard our test was"... puke. I want to work with solid performers who don't play games, why would I start that relationship off by playing a bunch of stupid games (silly interview tests/process)? And here's the thing, even with our somewhat easier/less-stress testing process, we still got high quality hires and have had very few cases where we have later had to let someone go due to not being able to do the job. This isn't a "hire fast, fire fast" situation.

jghn 1y ago

I agree with you that the real rubric is that I'd expect the candidate to be able to explain everything they did and why. For the most part I don't care how they got there. Was thinking about this recently in a conversation about whether or not the candidate should have a full screen share so that interviewers could see what they're looking up, etc. I realized that I'd reached the point where I now trust my fellow interviewers less than I do the candidate at these bits. Personally I think it's nice to see how people are leveraging tools, but too often I see interviewers dig candidates anyways on the specifics. I've now found this across multiple companies now has been that when coding exercises are "open book" like this that other interviewers *are* actively judging people based on what they're googling, and these days LLMing. If you're going to allow google but then fall back to "I can't believe they had to look *that* up!", that's not how it works.

sokoloff 1y ago
I think “how to write for loop in $LANGUAGE” is something that it’s fair to judge a candidate on if they claim (or the job requires) recent experience in $LANGUAGE.
- jghn 1y ago
  Maybe. I've been coding professionally for a few decades now. I have always looked stuff up constantly. It's a lot more seamless now due to first search engines and later the ever improving autocomplete. To be honest, I think half the time I just do it to do it. But for instance I bounce between languages so often that I can never remember the fine details of syntax. To use your example, things like for loops, matches, and the like are things that I tend to double check. And that's the point. Everyone has their quirks, and if you're going to claim you want them to work they way they normally work, who cares? If they're able to turn out quality code at a reasonable clip, who cares what they need to look up.