9 ms·

It's because in software engineering your work product is confidential/proprietary and work samples take a very long time to produce. If you could review someo

It's because in software engineering your work product is confidential/proprietary and work samples take a very long time to produce. If you could review someone's code from their previous jobs, I think it would be quite easy to vet applicants. And it would be much easier to move from job to job. Instead we have to simulate work samples through whiteboard coding, pair programming sessions, take home coding projects, coding knowledge tests, etc. We know that none of these methods are particularly good, so we err on the side of failing candidates when we are on the fence. They also take a long time to complete, so software engineers are reluctant to change jobs because they know how much of a pain it will be to go through interviews again. It is much easier for product designers and product managers to pass interviews. You can see their work product. At my company we're working on a way to create standardized coding projects that can't be faked so you can use them across job applications. If you are interested in this space, shoot me a message: robert at facet dot net.

UncleOxidant 5y ago
Do medical doctors face this? Their work is confidential as well - patient data is highly confidential.
- 6gvONxR4sf7o 5y ago
  Medical doctors have licenses which include exams and all sort of other barriers before they’re a ‘real’ independent doctor.
  - adamhp 5y ago
    I'd be curious to hear peoples thoughts if they would subject themselves to this sort of professional board and licensing for software. There's clearly a pre-determined set of information and skills that are fundamental enough to be tested, but it is also fairly creative and subjective at the end of the day.
    - rhacker 5y ago
      With medicine there are probably like 4 ways to treat a bullet wound. If you go too far outside of that you're probably not a good doctor. With software you can write a TODO app with a million lines of code or 7.
      - dleslie 5y ago
        ... Every serious trauma is unique and challenging in its own way.
    - hkt 5y ago
      Realistically, I think I'd not be allowed to practice with that kind of system. I suspect there's an entire class of programmer (maybe this is the dev vs eng distinction the article refers to) which would be excluded by a system like this. Fundamentally though I think that system of high priests and laypeople who are barely allowed to code at all can't be sustained. I strongly suspect all professions will eventually require a little programming, and certification will be hard for those cases.
    - smolder 5y ago
      I would likely take issue with whatever criteria people chose for judging someone's software engineering ability. Creating the best performing system is rarely anyone's goal (not that performance should be discounted,) they want to make something with the hard to quantify qualities of robustness, scalability, maintainability and expandability, to different degrees. They also often want to use languages or frameworks (or supporting software like databases) based on what boils down to fashion. Often the most desirable quality for candidates is an ability to learn and leverage new tools, given varied and shifting standards of good design, which again is hard to quantify.
    - 6gvONxR4sf7o 5y ago
      Given the amount of people with CS degrees who fail the most basic interviews, it would be hard to set something good up. I assume universities are trying to solve the same problems as the hypothetical board setting up this test. That, and the field moving so quickly and being so broad. I interview for ML. I assume interviews for front end are extremely different.
  - UncleOxidant 5y ago
    Sure, but that doesn't mean they're a good doctor. Most of us have had experiences with bad doctors. The exams they took could have been decades ago, but what does that tell us about how they've performed recently?
    - 6gvONxR4sf7o 5y ago
      Bad doctor is relative. I expect that most doctors who have pass through that gauntlet can pass the doctor equivalent of fizzbuzz, which already is a lot different than our industry.
      - dopylitty 5y ago
        A doctor equivalent of fizzbuzz could be “what do you prescribe for a viral infection with no indication of a secondary infection” and from what I’ve heard there are a lot of doctors failing that one during the COVID pandemic by prescribing antibiotics.
        sjburt 5y ago
        They are highly effective at making patients stop begging for antibiotics they don't need. Patient satisfaction with their doctor has nothing to do with medical outcomes.
        cratermoon 5y ago
        > prescribing antibiotics. I'd hire that one over anyone pushing horse de-wormer, anti-malarial drugs, or homeopathic remedies.
        rsynnott 5y ago
        Well, it's less dangerous than the first two, generally, but roughly equivalent to the third, for covid.
    - Ginden 5y ago
      It's quite hard to be really bad physician - because Pareto principle applies to medicine. As far as I remember, 5 autoimmune diseases (of >120 described; ~4%) make up for 80% of total autoimmune diseases. My relative did research on kidney diseases and results were similar.
- jaaron 5y ago
  No. As a software engineer for 20+ years, I've talked with my sister, who is an ER doctor about this often. Most folks outside of engineering think our interview processes are crazy. Keep in mind, there's significantly more training that goes into the medical field, so there's more emphasis on the institutions. Where did you go to school? What hospitals or facilities did you work with? Your resume and references speak more strongly. Other hurdles in software engineering hiring: - It's both a technical and (often) a creative field. - Most work requires teams to complete and there's very little insight into who contributed what into a project. - You don't have many of the signals from other fields, such as published papers, law cases, or hard numbers such as sales, certifications, etc. Long story short: we don't have good mechanisms within companies to measure engineering productivity, let alone measure programmers outside of an institution. I'm reminded of Steve McConnell's intro to "Software Estimation": "Meanwhile, the typical software organization is not struggling to improve its estimates from +/-10% to +/-5% accuracy. The typical software organization is struggling to avoid estimates that are incorrect by 100% or more."
  - bonesinger 5y ago
    One key difference is licensing. As a physician you have to take the USMLE Step 1,2,3 and then potentially other board certifications. We engineers don't have any of that.
    - Bilal_io 5y ago
      I imagine that amazon would still grill you in an interview even if you have one of their advance AWS certificates. I know those certificates are not the same, but I don't think it's about certifications.
    - jaaron 5y ago
      Yes and depending on the field, you may have to do mandatory additional certification and training throughout your career. We have certs in software engineering, but in my experience, they've largely failed as an indicator of qualifications. Some hiring managers see them as nice to have, but I've also met hiring managers who are actively skeptical of certifications. We also don't have differentiation of software levels or skills. If you're a programmer, you could, in theory, go program anything from web to AI to robotics. You don't have to be certified to work in nearly any industry. And we don't have formal hierarchy of skilled work, such as EMTs, nurses and doctors. We're a relatively young industry that just hasn't worked things out yet and, to be honest, in the United States, there isn't pressure by employers to get our act together. The engineers themselves are in charge often and we're skeptical of boards and authority. We have too many stories of folks making it big with no formal education. Those with the money aren't demanding change because the current system "works." It's possible software engineering will someday evolve into a boring respectable profession, but right now it's still a creative field of fads, trends, rock stars and super stars (let alone ninjas and wizards).
- 908B64B197 5y ago
  No, but they have a legalized cartel to make sure there's never an over-supply (that would lower the price for the consumer, we don't want that!) that also gatekeep credentialing. Meanwhile in Software there are initiatives for "Gender X of Minority Y can Code" and "Everyone can be an Engineer after 3 months of BootCamp".
JohnBooty 5y ago
Instead we have to simulate work samples through whiteboard coding, pair programming sessions, take home coding projects, coding knowledge tests, etc. Very small sample size but we've had good results skipping coding exams entirely, and simply talking through some scenarios with candidates. - give them a hypothetical problem ("the page is loading slowly") and role play how they'd troubleshoot it - do they understand how different bits of the stack fit together? have they ruled out network issues before digging into the code? etc. - "tell us about a tricky technical challenge you solved" - ask them what makes a for good code review on a pull request / what makes good code review culture in general - ask them about things the team did well at their previous role(s) and things they'd improve, and how they'd improve upon them. (not looking for "correct" answers or opinions - but they should have some opinions) - what are some technologies/frameworks/languages you'd like to learn next? again there's no "right" answer. but if they don't have any - red flag, right? - etc This of course presumes that some minimal screening has been done to make sure they're actually a software developer; an initial Fizzbuzz screen etc. At each step of the way we discussed relevant technical bits in a natural and conversational way. For example if they mention that they "optimized" or "refactored" or "fixed" something at a previous job then naturally we ask them for the details and usually wind up swapping some stories as well. If they passed this conversational interview, we usually extended an offer to them within 24 hours or less. Sometimes we waited longer and they got snatched up by another company. Of course this sort of interview doesn't tell you about their work ethic and how reliable they are and so on. But, neither does a lengthy coding assignment-based process either. And the "conversational" method gives you a sense of how they'll fit in with the team. Considering how quickly good candidates are snapped up, doing "fast" interviews like this can be a big advantage for your hiring efforts. If worst comes to worst, and they don't work out, you can part ways. It's not like either side is locked into the employment agreement. Depends on your specific locale a bit, of course -- and at my current employer we're helped by being a remote team, so it's not like we're asking folks to relocate or anything.
- vlovich123 5y ago
  > - give them a hypothetical problem ("the page is loading slowly") and role play how they'd troubleshoot it - do they understand how different bits of the stack fit together? have they ruled out network issues before digging into the code? etc. I don't know. I think I'm pretty good at troubleshooting on the job (since I can be the one to troubleshoot & solve the trickier bugs) but thinking about doing it in an interview setting makes me nervous. It feels too fickle & up to the biases of the interviewer that day, not to mention that any serious production bug relies on knowing how the software stack operates & which things are relevant, all things that are ambiguous in an interview the interviewee would just be stuck grasping in the dark to try to figure out which scenario you might be describing. The rest seems fine, but I'm curious about your experience with the live debugging & if you've considered cutting it.
mywittyname 5y ago
> If you could review someone's code from their previous jobs, I think it would be quite easy to vet applicants. I disagree, somewhat. In positions where I work with a medium or large team, any code of mine you reviewed would give you an idea of how the team leads want code to be written, not really how I as a candidate write code. I've often disagreed on design/architecture but was forced to go through with it because I really didn't feel like arguing and getting on the shitlist of a high-ranking engineer with influence in the company. If you want an idea of how I would design something, it's probably best to have me write something specific to the job. I'm not talking a whiteboard red-black tree implementation, but something that is actually indicative of the work I'll be doing.
WalterBright 5y ago
That's one advantage to working on open source. Your work is all just a click away.
- dangwu 5y ago
  Have you ever gotten a job from just showing off your open source work, though?
  - WalterBright 5y ago
    Other people I know have. Quite a few have leveraged their contributions to the D language into a much higher paying job than they had before.
  - donretag 5y ago
    I was a committer to Elasticsearch. My name is thanked in the (very old) official book. I will still get asked questions. No one looks at open source work, just as no one cares about your resume.
roberthahn 5y ago
I have had good luck hiring developers by asking them to do a code review of something WE wrote (but framed as if a junior developer wrote it) This helps us look for good signal around their communication skills, mentoring skills, empathy, reasoning skills and software development skills. We used this review as a diagnostic - there are no wrong answers since we know they’re not aware of our style guide and conventions. It worked for us. Maybe it could work for others.
- lytefm 5y ago
  I'm using the same technique. Taking MRs from Junior Devs together with the Ticket description, reducing the scope a bit and sprinkling different errors or incomplete implementations here and there. In the interview, I'm checking of problems that the applicant notices and see how he approaches the problem. So far I've been satisfied with all hired who passed that test. And they liked not having to solve stupid whiteboard problems.
- gwittel 5y ago
  I do something very similar and it works quite well for anyone except fresh grads (I've a different variant for them). I pose a situation and introduce the v1 code. They look for bugs and propose fixes. Then I add on a new feature, and they look for more bugs. I've maybe 4-5 levels overall each having different classes of bugs, some of more or less subtlety. Fits on one whiteboard / single piece of paper in the end. There are no syntax errors, just logic. Errors are in categories like: robustness, failure handling (internal or external), data/thread races, unforseen side effects, etc. The more experienced you are, the more problems I expect to be found. A "I don't know" is just fine (and often the following discussion is MORE informative).
  - roberthahn 5y ago
    I _really_ like this variant! In my experience new grads never seemed to have too much trouble with the exercise but then I do explain that it’s diagnostic in nature and use the exercise to see how coachable they are.
- 5y ago
Matthias247 5y ago
I disagree on 2 points: > It is much easier for product designers and product managers to pass interviews. You can see their work product. You don't see any more work of a product manager than of an engineer. What you see is always the finished product, and you have no idea who contributed what parts. If the overall product that can be publicly experienced is good, then some engineers in that team seem to have been doing a good job. You won't easily know which ones - and you especially don't know whether a product manager on the team really a lot of influence on this or whether the products vision and execution was mostly driven by engineers. And the second point: > If you could review someone's code from their previous jobs, I think it would be quite easy to vet applicants. And it would be much easier to move from job to job. Sometimes employers can - because candidates have open source code. But experience has shown that these datapoints are not used, and employers still fall back to the default process of assuming nothing and doing whiteboard/leetcode style interviews.
- cutemonster 5y ago
  And, about reviewing code from someone's previous job -- they might have had to write sloppy code because of time pressure

UncleOxidant 5y ago

Do medical doctors face this? Their work is confidential as well - patient data is highly confidential.

6gvONxR4sf7o 5y ago
Medical doctors have licenses which include exams and all sort of other barriers before they’re a ‘real’ independent doctor.
- adamhp 5y ago
  I'd be curious to hear peoples thoughts if they would subject themselves to this sort of professional board and licensing for software. There's clearly a pre-determined set of information and skills that are fundamental enough to be tested, but it is also fairly creative and subjective at the end of the day.
  - rhacker 5y ago
    With medicine there are probably like 4 ways to treat a bullet wound. If you go too far outside of that you're probably not a good doctor. With software you can write a TODO app with a million lines of code or 7.
    - dleslie 5y ago
      ... Every serious trauma is unique and challenging in its own way.
  - hkt 5y ago
    Realistically, I think I'd not be allowed to practice with that kind of system. I suspect there's an entire class of programmer (maybe this is the dev vs eng distinction the article refers to) which would be excluded by a system like this. Fundamentally though I think that system of high priests and laypeople who are barely allowed to code at all can't be sustained. I strongly suspect all professions will eventually require a little programming, and certification will be hard for those cases.
  - smolder 5y ago
    I would likely take issue with whatever criteria people chose for judging someone's software engineering ability. Creating the best performing system is rarely anyone's goal (not that performance should be discounted,) they want to make something with the hard to quantify qualities of robustness, scalability, maintainability and expandability, to different degrees. They also often want to use languages or frameworks (or supporting software like databases) based on what boils down to fashion. Often the most desirable quality for candidates is an ability to learn and leverage new tools, given varied and shifting standards of good design, which again is hard to quantify.
  - 6gvONxR4sf7o 5y ago
    Given the amount of people with CS degrees who fail the most basic interviews, it would be hard to set something good up. I assume universities are trying to solve the same problems as the hypothetical board setting up this test. That, and the field moving so quickly and being so broad. I interview for ML. I assume interviews for front end are extremely different.
- UncleOxidant 5y ago
  Sure, but that doesn't mean they're a good doctor. Most of us have had experiences with bad doctors. The exams they took could have been decades ago, but what does that tell us about how they've performed recently?
  - 6gvONxR4sf7o 5y ago
    Bad doctor is relative. I expect that most doctors who have pass through that gauntlet can pass the doctor equivalent of fizzbuzz, which already is a lot different than our industry.
    - dopylitty 5y ago
      A doctor equivalent of fizzbuzz could be “what do you prescribe for a viral infection with no indication of a secondary infection” and from what I’ve heard there are a lot of doctors failing that one during the COVID pandemic by prescribing antibiotics.
      - sjburt 5y ago
        They are highly effective at making patients stop begging for antibiotics they don't need. Patient satisfaction with their doctor has nothing to do with medical outcomes.
      - cratermoon 5y ago
        > prescribing antibiotics. I'd hire that one over anyone pushing horse de-wormer, anti-malarial drugs, or homeopathic remedies.
        rsynnott 5y ago
        Well, it's less dangerous than the first two, generally, but roughly equivalent to the third, for covid.
  - Ginden 5y ago
    It's quite hard to be really bad physician - because Pareto principle applies to medicine. As far as I remember, 5 autoimmune diseases (of >120 described; ~4%) make up for 80% of total autoimmune diseases. My relative did research on kidney diseases and results were similar.
jaaron 5y ago
No. As a software engineer for 20+ years, I've talked with my sister, who is an ER doctor about this often. Most folks outside of engineering think our interview processes are crazy. Keep in mind, there's significantly more training that goes into the medical field, so there's more emphasis on the institutions. Where did you go to school? What hospitals or facilities did you work with? Your resume and references speak more strongly. Other hurdles in software engineering hiring: - It's both a technical and (often) a creative field. - Most work requires teams to complete and there's very little insight into who contributed what into a project. - You don't have many of the signals from other fields, such as published papers, law cases, or hard numbers such as sales, certifications, etc. Long story short: we don't have good mechanisms within companies to measure engineering productivity, let alone measure programmers outside of an institution. I'm reminded of Steve McConnell's intro to "Software Estimation": "Meanwhile, the typical software organization is not struggling to improve its estimates from +/-10% to +/-5% accuracy. The typical software organization is struggling to avoid estimates that are incorrect by 100% or more."
- bonesinger 5y ago
  One key difference is licensing. As a physician you have to take the USMLE Step 1,2,3 and then potentially other board certifications. We engineers don't have any of that.
  - Bilal_io 5y ago
    I imagine that amazon would still grill you in an interview even if you have one of their advance AWS certificates. I know those certificates are not the same, but I don't think it's about certifications.
  - jaaron 5y ago
    Yes and depending on the field, you may have to do mandatory additional certification and training throughout your career. We have certs in software engineering, but in my experience, they've largely failed as an indicator of qualifications. Some hiring managers see them as nice to have, but I've also met hiring managers who are actively skeptical of certifications. We also don't have differentiation of software levels or skills. If you're a programmer, you could, in theory, go program anything from web to AI to robotics. You don't have to be certified to work in nearly any industry. And we don't have formal hierarchy of skilled work, such as EMTs, nurses and doctors. We're a relatively young industry that just hasn't worked things out yet and, to be honest, in the United States, there isn't pressure by employers to get our act together. The engineers themselves are in charge often and we're skeptical of boards and authority. We have too many stories of folks making it big with no formal education. Those with the money aren't demanding change because the current system "works." It's possible software engineering will someday evolve into a boring respectable profession, but right now it's still a creative field of fads, trends, rock stars and super stars (let alone ninjas and wizards).
908B64B197 5y ago
No, but they have a legalized cartel to make sure there's never an over-supply (that would lower the price for the consumer, we don't want that!) that also gatekeep credentialing. Meanwhile in Software there are initiatives for "Gender X of Minority Y can Code" and "Everyone can be an Engineer after 3 months of BootCamp".

JohnBooty 5y ago

Instead we have to simulate work samples through whiteboard coding, pair programming sessions, take home coding projects, coding knowledge tests, etc. Very small sample size but we've had good results skipping coding exams entirely, and simply talking through some scenarios with candidates. - give them a hypothetical problem ("the page is loading slowly") and role play how they'd troubleshoot it - do they understand how different bits of the stack fit together? have they ruled out network issues before digging into the code? etc. - "tell us about a tricky technical challenge you solved" - ask them what makes a for good code review on a pull request / what makes good code review culture in general - ask them about things the team did well at their previous role(s) and things they'd improve, and how they'd improve upon them. (not looking for "correct" answers or opinions - but they should have some opinions) - what are some technologies/frameworks/languages you'd like to learn next? again there's no "right" answer. but if they don't have any - red flag, right? - etc This of course presumes that some minimal screening has been done to make sure they're actually a software developer; an initial Fizzbuzz screen etc. At each step of the way we discussed relevant technical bits in a natural and conversational way. For example if they mention that they "optimized" or "refactored" or "fixed" something at a previous job then naturally we ask them for the details and usually wind up swapping some stories as well. If they passed this conversational interview, we usually extended an offer to them within 24 hours or less. Sometimes we waited longer and they got snatched up by another company. Of course this sort of interview doesn't tell you about their work ethic and how reliable they are and so on. But, neither does a lengthy coding assignment-based process either. And the "conversational" method gives you a sense of how they'll fit in with the team. Considering how quickly good candidates are snapped up, doing "fast" interviews like this can be a big advantage for your hiring efforts. If worst comes to worst, and they don't work out, you can part ways. It's not like either side is locked into the employment agreement. Depends on your specific locale a bit, of course -- and at my current employer we're helped by being a remote team, so it's not like we're asking folks to relocate or anything.

vlovich123 5y ago
> - give them a hypothetical problem ("the page is loading slowly") and role play how they'd troubleshoot it - do they understand how different bits of the stack fit together? have they ruled out network issues before digging into the code? etc. I don't know. I think I'm pretty good at troubleshooting on the job (since I can be the one to troubleshoot & solve the trickier bugs) but thinking about doing it in an interview setting makes me nervous. It feels too fickle & up to the biases of the interviewer that day, not to mention that any serious production bug relies on knowing how the software stack operates & which things are relevant, all things that are ambiguous in an interview the interviewee would just be stuck grasping in the dark to try to figure out which scenario you might be describing. The rest seems fine, but I'm curious about your experience with the live debugging & if you've considered cutting it.

mywittyname 5y ago

> If you could review someone's code from their previous jobs, I think it would be quite easy to vet applicants. I disagree, somewhat. In positions where I work with a medium or large team, any code of mine you reviewed would give you an idea of how the team leads want code to be written, not really how I as a candidate write code. I've often disagreed on design/architecture but was forced to go through with it because I really didn't feel like arguing and getting on the shitlist of a high-ranking engineer with influence in the company. If you want an idea of how I would design something, it's probably best to have me write something specific to the job. I'm not talking a whiteboard red-black tree implementation, but something that is actually indicative of the work I'll be doing.

WalterBright 5y ago

That's one advantage to working on open source. Your work is all just a click away.

dangwu 5y ago
Have you ever gotten a job from just showing off your open source work, though?
- WalterBright 5y ago
  Other people I know have. Quite a few have leveraged their contributions to the D language into a much higher paying job than they had before.
- donretag 5y ago
  I was a committer to Elasticsearch. My name is thanked in the (very old) official book. I will still get asked questions. No one looks at open source work, just as no one cares about your resume.

roberthahn 5y ago

I have had good luck hiring developers by asking them to do a code review of something WE wrote (but framed as if a junior developer wrote it) This helps us look for good signal around their communication skills, mentoring skills, empathy, reasoning skills and software development skills. We used this review as a diagnostic - there are no wrong answers since we know they’re not aware of our style guide and conventions. It worked for us. Maybe it could work for others.

lytefm 5y ago
I'm using the same technique. Taking MRs from Junior Devs together with the Ticket description, reducing the scope a bit and sprinkling different errors or incomplete implementations here and there. In the interview, I'm checking of problems that the applicant notices and see how he approaches the problem. So far I've been satisfied with all hired who passed that test. And they liked not having to solve stupid whiteboard problems.
gwittel 5y ago
I do something very similar and it works quite well for anyone except fresh grads (I've a different variant for them). I pose a situation and introduce the v1 code. They look for bugs and propose fixes. Then I add on a new feature, and they look for more bugs. I've maybe 4-5 levels overall each having different classes of bugs, some of more or less subtlety. Fits on one whiteboard / single piece of paper in the end. There are no syntax errors, just logic. Errors are in categories like: robustness, failure handling (internal or external), data/thread races, unforseen side effects, etc. The more experienced you are, the more problems I expect to be found. A "I don't know" is just fine (and often the following discussion is MORE informative).
- roberthahn 5y ago
  I _really_ like this variant! In my experience new grads never seemed to have too much trouble with the exercise but then I do explain that it’s diagnostic in nature and use the exercise to see how coachable they are.
5y ago

Matthias247 5y ago

I disagree on 2 points: > It is much easier for product designers and product managers to pass interviews. You can see their work product. You don't see any more work of a product manager than of an engineer. What you see is always the finished product, and you have no idea who contributed what parts. If the overall product that can be publicly experienced is good, then some engineers in that team seem to have been doing a good job. You won't easily know which ones - and you especially don't know whether a product manager on the team really a lot of influence on this or whether the products vision and execution was mostly driven by engineers. And the second point: > If you could review someone's code from their previous jobs, I think it would be quite easy to vet applicants. And it would be much easier to move from job to job. Sometimes employers can - because candidates have open source code. But experience has shown that these datapoints are not used, and employers still fall back to the default process of assuming nothing and doing whiteboard/leetcode style interviews.

cutemonster 5y ago
And, about reviewing code from someone's previous job -- they might have had to write sloppy code because of time pressure