6 ms·

I don't think those explanations are mutually exclusive. Yes, there's a large cohort of "senior" software engineers who can't actually code. They bullshit thei

I don't think those explanations are mutually exclusive. Yes, there's a large cohort of "senior" software engineers who can't actually code. They bullshit their way into jobs until they're fired and then apply for the next one. These are the people you want to filter out with live coding. But also, someone can fail a live coding interview for reasons other than belonging to that group.

TinkersW 1y ago
You could filter then out much more effectively by letting them sit in a room by themself to write the code, that way you aren't missing out on good candidates who can't function when under abnormal stress(that has nothing in common with the actual job).
- dnissley 1y ago
  We live in a remote world where a hiring process like this is less of an option in most cases
  - ghaff 1y ago
    Leaving aside that many companies have pulled back from remote to at least some degree, I'd always push for an in-person day for a variety of reasons. In general, the cost is nothing for a late-stage/end-stage confirmation. And, honestly, a candidate that just doesn't want to do that is a red flag.
    - dangus 1y ago
      While I don’t disagree with you I find it to be a slippery slope to some extent. Would you screen out Linus Torvalds because he hypothetically doesn’t want to come in to a physical office for an interview? Hiring managers should think long and hard in a data-driven way about whether the office presence is so necessary that you are willing to miss out on the best candidates who have the luxury of being picky. Is it true scientifically that an in-person interview day results in better candidate quality or is that just a vibe? I think eliminating top talent who refuse to step foot in an office and are rare enough to be able to maintain that demand is a lot of quality people being left out of your talent pool. I thought during the pandemic we already proved by numerous studies that in-office workers are less productive. My company philosophy would be more like, put the burden of identifying quality talent on the employer rather than the employee. Put the candidate through the minimum effort required to screen them and identify standout talent. Then when you find that standout talent you roll out the red carpet and focus on convincing them to work at your company.
      - ghaff 1y ago
        You can come up with outlier examples of course--though I'm not sure how relevant they are unless you're looking at hiring a "name" for some reason. But I'd still default to an in-person visit of some sort. I've never seen any data but then in-person was just assumed in most cases until a few years ago.
        dangus 1y ago
        Read your last two sentences over again. That’s exactly my point: it’s all an old habit that isn’t based on outcomes. I think it’s a human social instincts thing and not a quantitative thing. There might be a better way but we default to the social ritual because ape brain is most comfortable with it.
    - marcosdumay 1y ago
      > In general, the cost is nothing for a late-stage/end-stage confirmation. One in-person day costs a nearby candidate about 3 days, and a more remote candidate anything from a week to a couple of months depending mostly on where you are. And yeah, it also costs some money that maybe you will reimburse. It doesn't cost you much, that's for sure. But if it's for a full-remote position, it's absolutely not a "the cost is nothing" situation and the candidate refusing it for some random company in a random stage of the interview is absolutely reasonable.
- deleted 1y ago
  [deleted]
- sigmoid10 1y ago
  I've had take home problems for job interviews that were given a few days before and during the actual interview I only had to explain my code. But I wouldn't be sure this still works as a useful candidate filter today, given how much coding agents have advanced. In fact, if you were a sr dev and had a bunch of guys to bounce this problem back and forth, it wouldn't even have filtered out the bad ones back in the old days. There is very little that is more telling than seeing a person work out a problem live, even if that sucks for smart people who can't handle stress.
  - gitremote 1y ago
    How would someone explain code that was vibe-coded?
    - sigmoid10 1y ago
      Are you asking how they would get that info they didn't have / couldn't come up with? Because you can literally have a chatbot explain every line to you and why it is there and even ask it about things you don't know like a teacher. And for simple problems this will probably work better than with actual humans.
      - gitremote 1y ago
        I assume questions to explain the code would be extremely specific about why you did something on a specific line, or why you chose to design this part that way instead of another, to detect plagiarism and vibe coding, not a request for a prepared monologue.
    - squeaky-clean 1y ago
      You can have the AI explain it to you. There's also a middle ground between vibe coding and "I can code some things but never could have coded this without an AI". Doesn't even have to be AI. Give me some random file from the Linux kernel and I could probably explain everything to you if you gave me a few hours. But that doesn't mean I would ever be able to write that code.
- JonChesterfield 1y ago
  I don't know about that. Long ago I interviewed with someone that wanted some trivial C++ thing written on their laptop. I hadn't seen a Windows dev machine before and had no Internet access. I think I'd worked out the compiler was called visual studio and how to compile hello world by the time limit. Not sure that told either of us much.
Paul-Craft 1y ago
Why do you need arbitrary (and very short) deadlines, and for someone to stand up at a whiteboard while simultaneously trying to solve a problem and "walk you through their thought process" to filter out people who can't write code on the job?
- ndriscoll 1y ago
  The short deadlines are because neither the company nor the candidate wants to spend a month on an extended interview. Solving a problem and walking through the thought process are because that's what "coding" is.
  - Paul-Craft 1y ago
    I don't know about you, but I've never had to live code a PR and explain to my reviewer what I was thinking while writing the code. By "deadlines" I'm referring to the length of the interview. Take home problems theoretically solve both these issues, but they need to be properly scoped and specified to be valid assessments.
    - ndriscoll 1y ago
      I sit down with juniors and sketch out designs or code for them while talking through the thought process at least once a week, and even when solo coding, I expect everyone produces work that explains itself. For particularly complex/nuanced changes, people do hop on a call to talk through it. Like I said the deadlines work for both sides. If a company wants to give homework instead of having their own senior engineers spend time talking to me, that tells me what I need to know about how they value my time.
      - Paul-Craft 1y ago
        > I sit down with juniors and sketch out designs or code for them while talking through the thought process at least once a week, and even when solo coding, I expect everyone produces work that explains itself. For particularly complex/nuanced changes, people do hop on a call to talk through it. That's not equivalent to what I said, nor is it live coding. Again, those deadlines are artificially short compared to real world scenarios, and completely arbitrary. They are so short, in fact, that they render the interview an invalid test of real working ability. A work sample has been proven time and again to be the most valid measure of whether a candidate can actually perform the job, but the conditions under which a live coded "work sample" is performed in an interview render it invalid.
onion2k 1y ago
I think there's a lot of developers who can ace a live-coding interview but who lack the understanding of engineering systems at scale so they'll make your whole codebase worse over time by introducing tech debt, anti-patterns, and inconsistencies. These are the people you really want to avoid, but very few interview processes are set up to filter them out. There's an assumption that the company's existing senior architects and developers will stop a new person from making the code worse, but also devs at every company thinks their codebase is terrible so it obviously isn't working.
- zwnow 1y ago
  I've seen lots of devs who think their codebase is the only correct way to do things. Lots of overconfident people out there. Inconsistencies are fine as long as there's file level consistency. All that really matters is if you can relatively quickly understand what you are working with. What you really want to avoid is having functions doing 20 different things from 5 different contexts.
- whstl 1y ago
  I agree. Live coding always has a much smaller scope than real software, and after a few interviews it is easy to learn to read the room, even for the worst developers. I think we can leave companies who don't care about quality out of the discussion, but for those who do, the time to detect those developers is in a probational period, which is not something that most companies really use on their favor. The problem is this requires a good management that is able to actively paying attention to the work of the developer. Which is often not in place, even in companies who want to prioritize quality :/
  - rockemsockem 1y ago
    Have you interviewed people recently? The "worst developers" absolutely cannot solve basic problems.
    - whstl 1y ago
      Yes, I agree. Good point. A fizz buzz or something similar is often already too much for these.
bugjoey 1y ago
I share your point of view, but live coding these days are just beyond that testing programming skills. You must know by heart the most common algorithms out there and design solutions that might involve two or three of them to solve a problem in 30 minutes. Sometimes you spend the whole time trying to figure out how to solve the puzzle that don't even have time to show that you can - actually - code.
- CBLT 1y ago
  > You must know by heart the most common algorithms out there and design solutions that might involve two or three of them to solve a problem in 30 minutes. You're not going to pass every interview. Some of them are truly outlandish, and require all of the above. What you need is the technical knowledge to translate requirements into a loose pattern like "This looks like a search problem", then have the charisma (or more accurately, practice) to walk the interviewer through how each search algorithm you know could apply there. Then of course be able to actually write some code. I've passed interviews where I had never heard of the datastructure they wanted me to solve it with; I just walked them through the tradeoffs of every data structure that I knew applied to it instead.
paxys 1y ago
There are two ways to interview: 1. Make sure you pick every good candidate, but some bad candidates will slip through as well. 2. Make sure you reject every bad candidate, but some good candidates will fail as well. Candidates want process #1, but companies have no reason to push for it. The cost of accidentally hiring a bad employee is simply too high, way more than rejecting a good employee. The current system in place prioritizes #2. Yes they are rejecting great candidates, and they are aware of it.
- sceptic123 1y ago
  The article is suggesting that #2 will end up rejecting LOTS of good candidates (and potentially ALL female candidates)
  - rightbyte 1y ago
    Also I don't think being artificially picky is a better filter than just going with some gut feeling after weeding out candidates with fake credentials. Being picky gives the illusion of choosing when you in practice are bound down by the process.
sarchertech 1y ago
I’ve been doing this for 20 years. I’ve never worked with a single one of those people. I don’t think I’ve ever even interviewed one where I couldn’t have screened them out based on their resume and a 15 minute conversation. I’ve worked with plenty of people who passed a whiteboard interview and then spent years actively reducing aggregate productivity.
bradlys 1y ago
> Yes, there's a large cohort of "senior" software engineers who can't actually code. They bullshit their way into jobs until they're fired and then apply for the next one. These are the people you want to filter out with live coding. Genuinely, are there any amount of these at any significant scale in a place like Silicon Valley? I'm not sure I've ever met someone who couldn't code at any of the places I've worked. Senior engineers are heavily evaluated for their ability to pump out code. If you're not coding, what the hell are you doing at a startup that needs features built to generate any revenue?
- deleted 1y ago
  [deleted]

TinkersW 1y ago

You could filter then out much more effectively by letting them sit in a room by themself to write the code, that way you aren't missing out on good candidates who can't function when under abnormal stress(that has nothing in common with the actual job).

dnissley 1y ago
We live in a remote world where a hiring process like this is less of an option in most cases
- ghaff 1y ago
  Leaving aside that many companies have pulled back from remote to at least some degree, I'd always push for an in-person day for a variety of reasons. In general, the cost is nothing for a late-stage/end-stage confirmation. And, honestly, a candidate that just doesn't want to do that is a red flag.
  - dangus 1y ago
    While I don’t disagree with you I find it to be a slippery slope to some extent. Would you screen out Linus Torvalds because he hypothetically doesn’t want to come in to a physical office for an interview? Hiring managers should think long and hard in a data-driven way about whether the office presence is so necessary that you are willing to miss out on the best candidates who have the luxury of being picky. Is it true scientifically that an in-person interview day results in better candidate quality or is that just a vibe? I think eliminating top talent who refuse to step foot in an office and are rare enough to be able to maintain that demand is a lot of quality people being left out of your talent pool. I thought during the pandemic we already proved by numerous studies that in-office workers are less productive. My company philosophy would be more like, put the burden of identifying quality talent on the employer rather than the employee. Put the candidate through the minimum effort required to screen them and identify standout talent. Then when you find that standout talent you roll out the red carpet and focus on convincing them to work at your company.
    - ghaff 1y ago
      You can come up with outlier examples of course--though I'm not sure how relevant they are unless you're looking at hiring a "name" for some reason. But I'd still default to an in-person visit of some sort. I've never seen any data but then in-person was just assumed in most cases until a few years ago.
      - dangus 1y ago
        Read your last two sentences over again. That’s exactly my point: it’s all an old habit that isn’t based on outcomes. I think it’s a human social instincts thing and not a quantitative thing. There might be a better way but we default to the social ritual because ape brain is most comfortable with it.
  - marcosdumay 1y ago
    > In general, the cost is nothing for a late-stage/end-stage confirmation. One in-person day costs a nearby candidate about 3 days, and a more remote candidate anything from a week to a couple of months depending mostly on where you are. And yeah, it also costs some money that maybe you will reimburse. It doesn't cost you much, that's for sure. But if it's for a full-remote position, it's absolutely not a "the cost is nothing" situation and the candidate refusing it for some random company in a random stage of the interview is absolutely reasonable.
deleted 1y ago
[deleted]
sigmoid10 1y ago
I've had take home problems for job interviews that were given a few days before and during the actual interview I only had to explain my code. But I wouldn't be sure this still works as a useful candidate filter today, given how much coding agents have advanced. In fact, if you were a sr dev and had a bunch of guys to bounce this problem back and forth, it wouldn't even have filtered out the bad ones back in the old days. There is very little that is more telling than seeing a person work out a problem live, even if that sucks for smart people who can't handle stress.
- gitremote 1y ago
  How would someone explain code that was vibe-coded?
  - sigmoid10 1y ago
    Are you asking how they would get that info they didn't have / couldn't come up with? Because you can literally have a chatbot explain every line to you and why it is there and even ask it about things you don't know like a teacher. And for simple problems this will probably work better than with actual humans.
    - gitremote 1y ago
      I assume questions to explain the code would be extremely specific about why you did something on a specific line, or why you chose to design this part that way instead of another, to detect plagiarism and vibe coding, not a request for a prepared monologue.
  - squeaky-clean 1y ago
    You can have the AI explain it to you. There's also a middle ground between vibe coding and "I can code some things but never could have coded this without an AI". Doesn't even have to be AI. Give me some random file from the Linux kernel and I could probably explain everything to you if you gave me a few hours. But that doesn't mean I would ever be able to write that code.
JonChesterfield 1y ago
I don't know about that. Long ago I interviewed with someone that wanted some trivial C++ thing written on their laptop. I hadn't seen a Windows dev machine before and had no Internet access. I think I'd worked out the compiler was called visual studio and how to compile hello world by the time limit. Not sure that told either of us much.

Paul-Craft 1y ago

Why do you need arbitrary (and very short) deadlines, and for someone to stand up at a whiteboard while simultaneously trying to solve a problem and "walk you through their thought process" to filter out people who can't write code on the job?

ndriscoll 1y ago
The short deadlines are because neither the company nor the candidate wants to spend a month on an extended interview. Solving a problem and walking through the thought process are because that's what "coding" is.
- Paul-Craft 1y ago
  I don't know about you, but I've never had to live code a PR and explain to my reviewer what I was thinking while writing the code. By "deadlines" I'm referring to the length of the interview. Take home problems theoretically solve both these issues, but they need to be properly scoped and specified to be valid assessments.
  - ndriscoll 1y ago
    I sit down with juniors and sketch out designs or code for them while talking through the thought process at least once a week, and even when solo coding, I expect everyone produces work that explains itself. For particularly complex/nuanced changes, people do hop on a call to talk through it. Like I said the deadlines work for both sides. If a company wants to give homework instead of having their own senior engineers spend time talking to me, that tells me what I need to know about how they value my time.
    - Paul-Craft 1y ago
      > I sit down with juniors and sketch out designs or code for them while talking through the thought process at least once a week, and even when solo coding, I expect everyone produces work that explains itself. For particularly complex/nuanced changes, people do hop on a call to talk through it. That's not equivalent to what I said, nor is it live coding. Again, those deadlines are artificially short compared to real world scenarios, and completely arbitrary. They are so short, in fact, that they render the interview an invalid test of real working ability. A work sample has been proven time and again to be the most valid measure of whether a candidate can actually perform the job, but the conditions under which a live coded "work sample" is performed in an interview render it invalid.

onion2k 1y ago

I think there's a lot of developers who can ace a live-coding interview but who lack the understanding of engineering systems at scale so they'll make your whole codebase worse over time by introducing tech debt, anti-patterns, and inconsistencies. These are the people you really want to avoid, but very few interview processes are set up to filter them out. There's an assumption that the company's existing senior architects and developers will stop a new person from making the code worse, but also devs at every company thinks their codebase is terrible so it obviously isn't working.

zwnow 1y ago
I've seen lots of devs who think their codebase is the only correct way to do things. Lots of overconfident people out there. Inconsistencies are fine as long as there's file level consistency. All that really matters is if you can relatively quickly understand what you are working with. What you really want to avoid is having functions doing 20 different things from 5 different contexts.
whstl 1y ago
I agree. Live coding always has a much smaller scope than real software, and after a few interviews it is easy to learn to read the room, even for the worst developers. I think we can leave companies who don't care about quality out of the discussion, but for those who do, the time to detect those developers is in a probational period, which is not something that most companies really use on their favor. The problem is this requires a good management that is able to actively paying attention to the work of the developer. Which is often not in place, even in companies who want to prioritize quality :/
- rockemsockem 1y ago
  Have you interviewed people recently? The "worst developers" absolutely cannot solve basic problems.
  - whstl 1y ago
    Yes, I agree. Good point. A fizz buzz or something similar is often already too much for these.

bugjoey 1y ago

I share your point of view, but live coding these days are just beyond that testing programming skills. You must know by heart the most common algorithms out there and design solutions that might involve two or three of them to solve a problem in 30 minutes. Sometimes you spend the whole time trying to figure out how to solve the puzzle that don't even have time to show that you can - actually - code.

CBLT 1y ago
> You must know by heart the most common algorithms out there and design solutions that might involve two or three of them to solve a problem in 30 minutes. You're not going to pass every interview. Some of them are truly outlandish, and require all of the above. What you need is the technical knowledge to translate requirements into a loose pattern like "This looks like a search problem", then have the charisma (or more accurately, practice) to walk the interviewer through how each search algorithm you know could apply there. Then of course be able to actually write some code. I've passed interviews where I had never heard of the datastructure they wanted me to solve it with; I just walked them through the tradeoffs of every data structure that I knew applied to it instead.

paxys 1y ago

There are two ways to interview: 1. Make sure you pick every good candidate, but some bad candidates will slip through as well. 2. Make sure you reject every bad candidate, but some good candidates will fail as well. Candidates want process #1, but companies have no reason to push for it. The cost of accidentally hiring a bad employee is simply too high, way more than rejecting a good employee. The current system in place prioritizes #2. Yes they are rejecting great candidates, and they are aware of it.

sceptic123 1y ago
The article is suggesting that #2 will end up rejecting LOTS of good candidates (and potentially ALL female candidates)
- rightbyte 1y ago
  Also I don't think being artificially picky is a better filter than just going with some gut feeling after weeding out candidates with fake credentials. Being picky gives the illusion of choosing when you in practice are bound down by the process.

sarchertech 1y ago

I’ve been doing this for 20 years. I’ve never worked with a single one of those people. I don’t think I’ve ever even interviewed one where I couldn’t have screened them out based on their resume and a 15 minute conversation. I’ve worked with plenty of people who passed a whiteboard interview and then spent years actively reducing aggregate productivity.

bradlys 1y ago

> Yes, there's a large cohort of "senior" software engineers who can't actually code. They bullshit their way into jobs until they're fired and then apply for the next one. These are the people you want to filter out with live coding. Genuinely, are there any amount of these at any significant scale in a place like Silicon Valley? I'm not sure I've ever met someone who couldn't code at any of the places I've worked. Senior engineers are heavily evaluated for their ability to pump out code. If you're not coding, what the hell are you doing at a startup that needs features built to generate any revenue?

deleted 1y ago
[deleted]