6 ms·

> What would you rather have? Ideally? Not to assume my resume/education experience is a lie and to instead assume that I've done enough technical work by this

> What would you rather have? Ideally? Not to assume my resume/education experience is a lie and to instead assume that I've done enough technical work by this point to merit respect that I don't need to prove some "technical merit" via any sort of whiteboarding exercise that is neither anything like day-to-day technical work nor it is it much more than time wasting busy work to give interviewers plenty of room for excuses on why they don't like people. But, I'm in a minority opinion in general that I'd rather see the industry mature and if it needs to build a standardized test like the MCAT or the Bar Exam or PE certification to stop asking trivia questions in interviews it should just build a standardized test and stop wasting everyone's time.

asimpletune 5y ago
I mean we could have like a bar but that would severely limit opportunities of smart people who can code. It’s one of the best/worst things about our industry. Personally, I wouldn’t suffer, but I think as an industry we're unique in this way and it’s kind of overall maybe slightly better than not.
- HeyLaughingBoy 5y ago
  > that would severely limit opportunities of smart people who can code Whether or not that's true, is it really important? I guess it depends on your perspective. Do you think the industry exists to provide job opportunities, or to solve real business problems?
  - asimpletune 5y ago
    I think by serving an alternative way of doing things we're a nice position to imagine making that comparison. So, which industries are more innovative software/tech or those that have some kind of standards/gate-keeping? Admittedly it's not like a super fair comparison because we're still in the a very nascent phase of software as an industry, but I think as a concept the question still poses merit. Maybe when the industry matures more we'll finally establish a bar, but in the meanwhile I think most of the money people feel that there's just too much work to do for that to happen. Personally, I don't really know what's the optimal way to do things. Mainly my point is that it's interesting that we even exist. Like, we are one of the only "professionals" who don't have any standards to adhere... and this is precisely why interviews are so hard in our field. Sometimes they're asinine, as has been discussed here. (FWIW I try really hard to be a good interviewer and to give the candidate a pleasant experience and to design everything so they feel fairly judged and I get good signal)
- WorldMaker 5y ago
  A lot of professions split into separate job categories for white collar/blue collar or "formally educated"/"on the job learning". Lawyer/paralegal, Doctor/RN/Nurse Practitioner, etc. I don't know if that is something that software could use, but it certainly is a known option, and might help. In some ways software already has bespoke versions of those career paths, but there's no standardization from company to company and that's part of what makes interviewing hard because even career paths mean different things to different companies.
  - asimpletune 5y ago
    Yeah, apprenticeship would be a cool route to take.
kube-system 5y ago
I give take-homes because they’re a better representation of people’s actual work output. But I’m not giving them because I think people are lying. I can tell if people are lying about their experience in the first interview. The assignment part of the interview is to assess the quality of their work. Some people do awesome work straight out of school, and some have been doing awful work for 20 years. We do have standardized testing: certs. But they're of limited usefulness in determining someone's job performance.
- WorldMaker 5y ago
  But it is about "lying": you don't trust the quality of people's "actual work output" despite what their resume states. You have a "guilty until proven innocent" attitude with respect to people's professional work. You aren't trusting that other companies develop talent or prune people "doing awful work". Sure, a resume isn't about "job performance" but how much do you distrust the labor market in your own industry that you think people can do awful work for 20 years and still remain meaningfully employed enough to have resume experience across all 20 of those years? I understand where you are coming from and appreciate it is a common attitude in this industry, I just think it is a wild attitude, and an attitude related directly to the immaturity of our industry that everyone assumes the labor market is inefficient and doesn't reflect "quality"/"performance".
  - kube-system 5y ago
    > But it is about "lying": you don't trust the quality of people's "actual work output" despite what their resume states. No, I’m saying that resumes simply don’t convey quality of work. They convey the duration of work and who it was for. Duration isn’t a proxy for quality. Beyond the first couple years of experience, I don’t think I’ve seen much of a reliable pattern between experience and ability. I can’t really look at some random project at some random company on a resume and tell you whether participation in that project amounted to brilliant engineering work, or whether it was changing ‘if’ statements three times a week in a legacy system. > how much do you distrust the labor market in your own industry that you think people can do awful work for 20 years and still remain meaningfully employed enough to have resume experience across all 20 of those years? I’ve been on the wrong end of hiring mistakes where that was exactly the situation. There are definitely certain types of jobs that are “known for being cushy” in this industry. Although, that’s nothing unique to tech. People coast in tons of white-collar jobs.
drenvuk 5y ago
A standardized test might actually be ok but I doubt it could encompass all of the technologies that appear over the years which one would use at a company if the company is looking for a specific skillset. That's why we go with resumes. I would be willing to bet that you have at least one thing on your resume that is embellished ever so slightly. Some people take it a bit further than others, which is what these interviews are meant to ferret out.
- WorldMaker 5y ago
  > A standardized test might actually be ok but I doubt it could encompass all of the technologies that appear over the years which one would use at a company if the company is looking for a specific skillset. I think the problem here is the company never knows if it is looking for a specific skillset or not, and "how specific" to hire. Then using that as a crutch to avoid asking hard questions of itself if it can hire-to-train. Most specific skillsets can be trained with a broad enough education. That's what most other industries/career paths rely on. Software uses the excuse that technologies move "too quickly" to avoid hiring to train or hiring generalists that would be fine with a specific skillset if given the right guidance/time/budget. > I would be willing to bet that you have at least one thing on your resume that is embellished ever so slightly. I don't. Why would I? This is exactly what I'm complaining about. As an industry we've taken a "guilty until proven innocent" mentality and interviews aren't about respect but presuming lies and always about "ferreting out" the liars. It's awful and just about no other industry works that way in hiring decisions. I've had interviews find lies that a recruiter edited into my resume, but that's a problem with the industry's relationship with bad recruiters and doesn't seem a good reason to call me a liar, when I'm not the one doing that.
  - drenvuk 5y ago
    Then you're a paragon of honesty. There are many others who aren't, sadly. Rather than assume people are liars I use the live coding test more as another filter. It verifies the resume, sure, but it also showcases or sometimes exposes other aspects of the candidate that a resume or maybe just their resume cannot.
amf12 5y ago
> Ideally? Not to assume my resume/education experience is a lie and to instead assume that I've done enough technical work by this point to merit respect that I don't need to prove some "technical merit" Unfortunately, I have interviewed many candidates who have excellent resume, relevant experience, but who can't write basic string manipulation code which does not even require any advanced algorithms or data structures.
- WorldMaker 5y ago
  I think a lot of this is a variant of "survivor bias" ("failure bias"?): a whiteboard interview tests unrelated things like nerves/anxiety/stress/context. You have no idea how many of those candidates you interviewed were just having a bad day, or were too focused on trying to solve for "traps" in the interview they stumbled over basics, or had to do so much advanced string manipulation that got trapped in trying to re-simplify their concerns from real world practical work (forest) to "basic string manipulation" (trees). I think a lot of it gets down to what are we actually testing for here?
  - amf12 5y ago
    > You have no idea how many of those candidates you interviewed were just having a bad day Sure, which is why we have multiple interviews and not just one. > or were too focused on trying to solve for "traps" in the interview they stumbled over basics I disagree to an extent. Its not a "here is the problem, you have N minutes, go". I work with the candidate - guiding them, explaining, understanding their rationale. > or had to do so much advanced string manipulation that got trapped in trying to re-simplify their concerns from real world practical work (forest) to "basic string manipulation" (trees). I am talking about simple string manipulation which can be done with just loops. > I think a lot of it gets down to what are we actually testing for here? That they can code independently, think about the problem, reason about the potential edge cases and communicate their rationale. Its not a binary decision, but how they are overall.
    - WorldMaker 5y ago
      > That they can code independently You are testing that they can code independently by forcing them to code in a group setting? "Guiding them, explaining, understanding their rationale" is about the exact opposite of testing for "can they code independently". Not to mention all the usual issues that whiteboard "coding" resembles real world coding not at all. Especially if you are just talking simple string manipulation which can be done with just loops; either you are asking the candidate to reinvent the wheel of a library function they use all the time and should never write by hand in a real world codebase or your example is contrived in other ways in which the real answer is way more complicated due to performance issues (because low level string manipulation likely is a performance issue). This is exactly where it becomes a trap, too: in a real world application I'd need to evaluate any and every string manipulation written by hand to test it for performance issues and determine the appropriate string manipulation tools for the job. In C# today that includes knowing when to use a StringBuilder versus knowing a lot of gory details of Span<T> and String.Create and a deep rabbit hole of memory management issues (.NET strings are immutable, so this pretty much accounts for all real world string manipulation in .NET today that it is never "just" string manipulation). Other languages have related memory management concerns, but different dialects and mental overhead. Also, it's 2021 and no one in the real world should be doing any string manipulation at all without accounting for Unicode, so in a real world application you need to make sure you are using your platform's correct APIs for "rune" manipulation rather than raw codepoint manipulation for Unicode safety. In a whiteboard exercise you are probably going to tell me "don't worry about that" or "just keep it simple", but that's the biggest "reasoning" behind potential edge cases and after a number of years of professional work I can't shut off that firehose of practical concerns and it will take me a while to get to the "simple exercise" because my brain has lots of in-grained habits at this point and you are asking for "coding insight" like the real world, but in no way like an actual practical programming issue and the real programming issues still get in the way because I "know too much" at this point to react well in any "simple" problem. The other trap here is "soft-skills", again deeply contrary to "coding independently": while "coding" you want the candidate to express themselves out loud and communicate their rationale. While many of the pop psychology ideas of "right brain, left brain" are mostly wrong (or at least wildly over-generalized), the basic idea applies here well: you are asking candidates to light up two very different sections of their brain all at once. Maybe for someone far more used to pair programming that's a bit like walking and chewing bubble gum at the same time, but for someone used to "coding independently" (as you are asking to test) that's a lot more like those stupid pat your head and rub your tummy at the same time "tests". You can do it, it takes a lot more energy and conscious thought, and it doesn't "feel natural" for "coding independently" at all. I don't know about anyone else, but I find that not just incredibly draining but a migraine trigger and it is very hard for me not to end nearly every interview with a nasty migraine. > think about the problem, reason about the potential edge cases You'd test for these better with creative problem solving exercises. Everyone likes to make fun of those "silly Microsoft-style questions" like "describe all the functions of Vending machine to me like I've never used one" or "how many spherical elephants can you fly in a Boeing if the Boeing were made out of Legos you had to assemble from scratch but assume the fully assembled plane could still fly for some reason" or silly things like that, but they actually do a stronger job of exploring someone's creative problem solving skillset than any equivalent "coding" exercise, if the person truly has never encountered that specific problem example before (which is the hard part Microsoft found that people started to collect them and prep for them). That's what I mean by "what are we actually testing for here?" If we are testing for creative problem solving, coding is rarely the best way to test for that. (Especially coding for "simple" things like string manipulation. That may be a domain someone has explored to considerable depths, and come out the other side where that is no longer a creative problem solving exercise but a boring, practical "construction" project with existing boring blueprints. Most algorithms questions test if a candidate can follow blueprints, not if they can problem solve.) > communicate their rationale Here's where we see the true crux of my question "what are we actually testing for here?" If we are testing for soft-skills, test for soft-skills. "Coding" isn't a great time to test people used to "coding independently" for communication soft-skills (see above). The HR world has centuries of knowledge on how to run soft-skills interviews (and how not to run them to avoid discrimination). "Coding" veers towards that "how not to run them" side because of that pat your head and rub your tummy effect. At best, it's not a great test of real world soft-skills performance. At worst, it's potentially discriminatory versus neuro-diversity.

asimpletune 5y ago

I mean we could have like a bar but that would severely limit opportunities of smart people who can code. It’s one of the best/worst things about our industry. Personally, I wouldn’t suffer, but I think as an industry we're unique in this way and it’s kind of overall maybe slightly better than not.

HeyLaughingBoy 5y ago
> that would severely limit opportunities of smart people who can code Whether or not that's true, is it really important? I guess it depends on your perspective. Do you think the industry exists to provide job opportunities, or to solve real business problems?
- asimpletune 5y ago
  I think by serving an alternative way of doing things we're a nice position to imagine making that comparison. So, which industries are more innovative software/tech or those that have some kind of standards/gate-keeping? Admittedly it's not like a super fair comparison because we're still in the a very nascent phase of software as an industry, but I think as a concept the question still poses merit. Maybe when the industry matures more we'll finally establish a bar, but in the meanwhile I think most of the money people feel that there's just too much work to do for that to happen. Personally, I don't really know what's the optimal way to do things. Mainly my point is that it's interesting that we even exist. Like, we are one of the only "professionals" who don't have any standards to adhere... and this is precisely why interviews are so hard in our field. Sometimes they're asinine, as has been discussed here. (FWIW I try really hard to be a good interviewer and to give the candidate a pleasant experience and to design everything so they feel fairly judged and I get good signal)
WorldMaker 5y ago
A lot of professions split into separate job categories for white collar/blue collar or "formally educated"/"on the job learning". Lawyer/paralegal, Doctor/RN/Nurse Practitioner, etc. I don't know if that is something that software could use, but it certainly is a known option, and might help. In some ways software already has bespoke versions of those career paths, but there's no standardization from company to company and that's part of what makes interviewing hard because even career paths mean different things to different companies.
- asimpletune 5y ago
  Yeah, apprenticeship would be a cool route to take.

kube-system 5y ago

I give take-homes because they’re a better representation of people’s actual work output. But I’m not giving them because I think people are lying. I can tell if people are lying about their experience in the first interview. The assignment part of the interview is to assess the quality of their work. Some people do awesome work straight out of school, and some have been doing awful work for 20 years. We do have standardized testing: certs. But they're of limited usefulness in determining someone's job performance.

WorldMaker 5y ago
But it is about "lying": you don't trust the quality of people's "actual work output" despite what their resume states. You have a "guilty until proven innocent" attitude with respect to people's professional work. You aren't trusting that other companies develop talent or prune people "doing awful work". Sure, a resume isn't about "job performance" but how much do you distrust the labor market in your own industry that you think people can do awful work for 20 years and still remain meaningfully employed enough to have resume experience across all 20 of those years? I understand where you are coming from and appreciate it is a common attitude in this industry, I just think it is a wild attitude, and an attitude related directly to the immaturity of our industry that everyone assumes the labor market is inefficient and doesn't reflect "quality"/"performance".
- kube-system 5y ago
  > But it is about "lying": you don't trust the quality of people's "actual work output" despite what their resume states. No, I’m saying that resumes simply don’t convey quality of work. They convey the duration of work and who it was for. Duration isn’t a proxy for quality. Beyond the first couple years of experience, I don’t think I’ve seen much of a reliable pattern between experience and ability. I can’t really look at some random project at some random company on a resume and tell you whether participation in that project amounted to brilliant engineering work, or whether it was changing ‘if’ statements three times a week in a legacy system. > how much do you distrust the labor market in your own industry that you think people can do awful work for 20 years and still remain meaningfully employed enough to have resume experience across all 20 of those years? I’ve been on the wrong end of hiring mistakes where that was exactly the situation. There are definitely certain types of jobs that are “known for being cushy” in this industry. Although, that’s nothing unique to tech. People coast in tons of white-collar jobs.

drenvuk 5y ago

A standardized test might actually be ok but I doubt it could encompass all of the technologies that appear over the years which one would use at a company if the company is looking for a specific skillset. That's why we go with resumes. I would be willing to bet that you have at least one thing on your resume that is embellished ever so slightly. Some people take it a bit further than others, which is what these interviews are meant to ferret out.

WorldMaker 5y ago
> A standardized test might actually be ok but I doubt it could encompass all of the technologies that appear over the years which one would use at a company if the company is looking for a specific skillset. I think the problem here is the company never knows if it is looking for a specific skillset or not, and "how specific" to hire. Then using that as a crutch to avoid asking hard questions of itself if it can hire-to-train. Most specific skillsets can be trained with a broad enough education. That's what most other industries/career paths rely on. Software uses the excuse that technologies move "too quickly" to avoid hiring to train or hiring generalists that would be fine with a specific skillset if given the right guidance/time/budget. > I would be willing to bet that you have at least one thing on your resume that is embellished ever so slightly. I don't. Why would I? This is exactly what I'm complaining about. As an industry we've taken a "guilty until proven innocent" mentality and interviews aren't about respect but presuming lies and always about "ferreting out" the liars. It's awful and just about no other industry works that way in hiring decisions. I've had interviews find lies that a recruiter edited into my resume, but that's a problem with the industry's relationship with bad recruiters and doesn't seem a good reason to call me a liar, when I'm not the one doing that.
- drenvuk 5y ago
  Then you're a paragon of honesty. There are many others who aren't, sadly. Rather than assume people are liars I use the live coding test more as another filter. It verifies the resume, sure, but it also showcases or sometimes exposes other aspects of the candidate that a resume or maybe just their resume cannot.

amf12 5y ago

> Ideally? Not to assume my resume/education experience is a lie and to instead assume that I've done enough technical work by this point to merit respect that I don't need to prove some "technical merit" Unfortunately, I have interviewed many candidates who have excellent resume, relevant experience, but who can't write basic string manipulation code which does not even require any advanced algorithms or data structures.

WorldMaker 5y ago
I think a lot of this is a variant of "survivor bias" ("failure bias"?): a whiteboard interview tests unrelated things like nerves/anxiety/stress/context. You have no idea how many of those candidates you interviewed were just having a bad day, or were too focused on trying to solve for "traps" in the interview they stumbled over basics, or had to do so much advanced string manipulation that got trapped in trying to re-simplify their concerns from real world practical work (forest) to "basic string manipulation" (trees). I think a lot of it gets down to what are we actually testing for here?
- amf12 5y ago
  > You have no idea how many of those candidates you interviewed were just having a bad day Sure, which is why we have multiple interviews and not just one. > or were too focused on trying to solve for "traps" in the interview they stumbled over basics I disagree to an extent. Its not a "here is the problem, you have N minutes, go". I work with the candidate - guiding them, explaining, understanding their rationale. > or had to do so much advanced string manipulation that got trapped in trying to re-simplify their concerns from real world practical work (forest) to "basic string manipulation" (trees). I am talking about simple string manipulation which can be done with just loops. > I think a lot of it gets down to what are we actually testing for here? That they can code independently, think about the problem, reason about the potential edge cases and communicate their rationale. Its not a binary decision, but how they are overall.
  - WorldMaker 5y ago
    > That they can code independently You are testing that they can code independently by forcing them to code in a group setting? "Guiding them, explaining, understanding their rationale" is about the exact opposite of testing for "can they code independently". Not to mention all the usual issues that whiteboard "coding" resembles real world coding not at all. Especially if you are just talking simple string manipulation which can be done with just loops; either you are asking the candidate to reinvent the wheel of a library function they use all the time and should never write by hand in a real world codebase or your example is contrived in other ways in which the real answer is way more complicated due to performance issues (because low level string manipulation likely is a performance issue). This is exactly where it becomes a trap, too: in a real world application I'd need to evaluate any and every string manipulation written by hand to test it for performance issues and determine the appropriate string manipulation tools for the job. In C# today that includes knowing when to use a StringBuilder versus knowing a lot of gory details of Span<T> and String.Create and a deep rabbit hole of memory management issues (.NET strings are immutable, so this pretty much accounts for all real world string manipulation in .NET today that it is never "just" string manipulation). Other languages have related memory management concerns, but different dialects and mental overhead. Also, it's 2021 and no one in the real world should be doing any string manipulation at all without accounting for Unicode, so in a real world application you need to make sure you are using your platform's correct APIs for "rune" manipulation rather than raw codepoint manipulation for Unicode safety. In a whiteboard exercise you are probably going to tell me "don't worry about that" or "just keep it simple", but that's the biggest "reasoning" behind potential edge cases and after a number of years of professional work I can't shut off that firehose of practical concerns and it will take me a while to get to the "simple exercise" because my brain has lots of in-grained habits at this point and you are asking for "coding insight" like the real world, but in no way like an actual practical programming issue and the real programming issues still get in the way because I "know too much" at this point to react well in any "simple" problem. The other trap here is "soft-skills", again deeply contrary to "coding independently": while "coding" you want the candidate to express themselves out loud and communicate their rationale. While many of the pop psychology ideas of "right brain, left brain" are mostly wrong (or at least wildly over-generalized), the basic idea applies here well: you are asking candidates to light up two very different sections of their brain all at once. Maybe for someone far more used to pair programming that's a bit like walking and chewing bubble gum at the same time, but for someone used to "coding independently" (as you are asking to test) that's a lot more like those stupid pat your head and rub your tummy at the same time "tests". You can do it, it takes a lot more energy and conscious thought, and it doesn't "feel natural" for "coding independently" at all. I don't know about anyone else, but I find that not just incredibly draining but a migraine trigger and it is very hard for me not to end nearly every interview with a nasty migraine. > think about the problem, reason about the potential edge cases You'd test for these better with creative problem solving exercises. Everyone likes to make fun of those "silly Microsoft-style questions" like "describe all the functions of Vending machine to me like I've never used one" or "how many spherical elephants can you fly in a Boeing if the Boeing were made out of Legos you had to assemble from scratch but assume the fully assembled plane could still fly for some reason" or silly things like that, but they actually do a stronger job of exploring someone's creative problem solving skillset than any equivalent "coding" exercise, if the person truly has never encountered that specific problem example before (which is the hard part Microsoft found that people started to collect them and prep for them). That's what I mean by "what are we actually testing for here?" If we are testing for creative problem solving, coding is rarely the best way to test for that. (Especially coding for "simple" things like string manipulation. That may be a domain someone has explored to considerable depths, and come out the other side where that is no longer a creative problem solving exercise but a boring, practical "construction" project with existing boring blueprints. Most algorithms questions test if a candidate can follow blueprints, not if they can problem solve.) > communicate their rationale Here's where we see the true crux of my question "what are we actually testing for here?" If we are testing for soft-skills, test for soft-skills. "Coding" isn't a great time to test people used to "coding independently" for communication soft-skills (see above). The HR world has centuries of knowledge on how to run soft-skills interviews (and how not to run them to avoid discrimination). "Coding" veers towards that "how not to run them" side because of that pat your head and rub your tummy effect. At best, it's not a great test of real world soft-skills performance. At worst, it's potentially discriminatory versus neuro-diversity.