4 ms·

This could have also been entitled, "A Story of an Incompetent Screener". I have interviewed over 2500 programmers and have never looked at their previous work

by edw519 10y ago

This could have also been entitled, "A Story of an Incompetent Screener". I have interviewed over 2500 programmers and have never looked at their previous work, on github or anywhere else. Never. - I don't know if it's their own work. - I don't know if they were part of a team that did the work. - I don't know their role on that team. - I don't know how long it took to build it. - I don't know what else it connects to. - I don't understand the context that the work fits into. - It's usually far too much for me to review to get a feel of their ability. I only need to see a very small sample of work done just for me. I usually have them do something small in 15 to 30 minutes, on white board, pencil and paper, or environment of their choice. OP mentions fizzbuzz. This is often a good choice (and weeds out 90% right away). Then I review what they've done with them. I'm interested in their "noun" (what they've produced), but more interested in their "verb", the discussion thay have with me about what they've done. That discussion can get very interesting... - Why did you do it this way? - How else could you have done it? - What could go wrong? - If you had more time, what else would you have done? - What additional data could you have used to make your decisions? That is how you find you what a programmer can do, not by looking at some repository without context. Believe me, I know there are lots of Fraudulent Bryans out there. They don't get past me. They shouldn't get past anyone who doesn't have a lazy dependence on stuff like github.

joshwcomeau 10y ago
What's frustrating for me is I tend to choke in high-stress situations like this. Obviously you can't trust a generic todo app in someone's GitHub, but some people have a very unique set of projects and it's clear that it isn't Stack Overflow copy and paste. By not looking at their GitHub at all, you're ignoring a huge signal as to the applicant's coding ability, as well as organizational skills, git fluency, etc. EDIT: that said, even on my worst day I can handle basic fizz-buzz problems. As long as the screener is basic enough, and only step 1 in a larger process, I have no complaints.
- edw519 10y ago
  What's frustrating for me is I tend to choke in high-stress situations like this. I'd rather find this out in an interview than during a critical sprint. That's part of the reason I do it this way. I'm interested in your work habits and personality as much as your code. I make my interviews as "low stress" as possible. A coder who can't code for me for 15 to 30 minutes is a big red flag. you're invalidating months/years of open-source work I'm not "invalidating" anything. I'm just doing my job as efficiently as possible, which is to assess your ability to contribute to my team.
  - Cpoll 10y ago
    > I'd rather find this out in an interview than during a critical sprint. I think that's a false equivalence. Crunch isn't like an evaluation. There's already a joke about this, where the interviewer is verbally abusive to the interviewee to "test if they work well under stress." (Not to say that you're doing anything wrong, just that I don't think your gleaning the correct information from such a thing).
  - joshwcomeau 10y ago
    Yeah, as another commenter said, an interview is not the same as a critical sprint. I don't mean that I find coding stressful, I mean that I find the social aspect of being interviewed stressful, and it's hard for me to get in the problem-solving state of mind when in that situation. I don't think you can argue that you're not invalidating someone's experience (within the context of their suitability for the position) if you don't look at any of the code they've written outside an interview process, because it's self-evident that there are candidates that are talented programmers who cannot demonstrate those skills effectively during an interview. You can argue about how insignificant of a minority that is, though. I'm a senior developer who has been offered most of the jobs he's applied for because I _have_ been able to prove basic code literacy during an interview (and have typically surprised employers by how effective I can be, when not in an interview situation). So if the _screening_ bar is low and if you're taking steps to ensure that the handicap of social anxiety is diminished, it may be a negligible percentage of candidates.
  - wott 10y ago
    > I'd rather find this out in an interview than during a critical sprint. There is no relation between those.
- segmondy 10y ago
  Practice, he just said fizzbuzz is a good example, if you choke up implementing fizzbuzz and consider it high stress situation. You don't deserve to a job, not even as a junior. I can understand if you were told to implement a red-black tree or some esoteric algorithm.
  - joshwcomeau 10y ago
    You misunderstand. I don't mean that fizzbuzz is so challenging that it causes stress. I mean that I have social anxiety that causes my brain to not work during interviews. Doesn't matter how good you are at coding if you're having a panic attack.
- rootlocus 10y ago
  > By not looking at their GitHub at all, you're invalidating months/years of open-source work I disagree. Nobody is invalidating anything. Those contributions still exist, people still benefit from them and you still retain your experience. That being said, while they don't directly contribute to your employment chances in this case, your experience does.
  - joshwcomeau 10y ago
    Yeah, I mean that they're invalidated in the scope of "material used to evaluate whether a candidate is suitable for the position". Definitely no arguments that open-source work has merit far beyond its impact on hireability :)
brogrammernot 10y ago
That's rather interesting. I've actually found that reading code on someone's github or checking out their personal website is great for ruling in/out candidates. When they share their github or website, I'll view the source on the website to see how they organized stuff or if they did literally copy/paste a template. With GitHub, it gives some talking points. So for example, if they have some Angular code on their GitHub and also list it as a competency on their resume I'll ask them about the project. Granted, I agree that this screener really did a poor job at seeing through this candidate's smoke screen but still, I find GitHub and personal websites useful in the interview process. It does not take away the actual parts you mentioned about walking through a problem during the interview, or talking about real situations on the job and how they would approach finding a solution.
- segmondy 10y ago
  do you search for code theft? 80% of the code submitted for my interviews is 100% copy and pasted. When they put in effort, they remove the comments and rename the variable.
  - brogrammernot 10y ago
    If the code is "beyond" what I would expect to see at the candidate's level, yes. It's merely a talking point, it becomes super clear if someone has copied/pasted code.
- cr0sh 10y ago
  Here's a question - to you and anyone else who cares to respond: I like to treat my github in a certain manner, and maybe I should change that. Currently, what I have on my github is some repos that are all mine - but I also have a bunch more repos that are forks of other repos. I do this (fork other repos) because I like the repo, and some of them might be really old (that is, they haven't been changes in months or years - low commit frequency or such). So - a fork for me is kinda like "save it for later", in case the original repo goes away. Plus, IIRC, if the original repo does get updated, I'll be notified and I can pull down the changes into my repo (my only other option would be to download a zip of the master branch - but then if it did change, I wouldn't know). Should I not do this? I mean, those forked repos are clearly labeled as forks from the original repo, and I have made it clear in the past that "only these repos are mine exclusively" - but should I not do this? Should I instead have a separate github just for those kinds of forks, and keep my personal repo for only my own stuff? I'm just wondering how confusing this really is for prospective employers or others - I don't want them thinking that repos x/y/z are mine when they aren't, and miss the repos that actually are mine. It's too bad that there isn't an option to segregate this better on github, and only show your repos by default. Maybe I need to review my settings...
lj3 10y ago
This type of interview in video form courtesy of Casey Muratori: https://www.youtube.com/watch?v=cfyWvJdsDRI https://www.youtube.com/watch?v=cfyWvJdsDRI
vanadium 10y ago
This is exactly the same format I've used, with two exceptions: I _do_ look at previous work, and the "custom project" isn't done in-person to keep it low-stress. This allows me to draw parallels in terms of patterns used and other commonalities I can then further ask about and reference while going over the "custom project" for the interview, without giving an opportunity to overwhelm them with anxiety. If the code style is completely out of bounds between the two projects, it either helps me dig deeper and ask about previous environments, etc. Adaptability. It also has the rather interesting side-effect of calling out frauds, especially if the "custom project" was completed and the explanation scripted into rote memory for the interview. Asking why a pattern was used in 'X' for a similar function in 'Y' project versus the pattern used in said "custom project" usually does the trick, separating rather efficiently the people that know what they're talking about from those who are just trying to skate by.
79d697i6fdif 10y ago
You would never hire somebody like me :). I write code in IDEs usually, and never basic console apps. Always inside of a framework or a bigger project. Good luck creating a maven project archetype and getting it to work during a whiteboard session. I would choke so hard on a basic "make a console app to display these numbers". I don't even remember the calls to string split and console display in some of my favorite languages. Why? You'd be an idiot to use those over Log4net or whatever you use in real life. You're asking a guy that might know how to design rockets how to put together a bicycle. A does not imply B in this case. I've always thought whiteboard interviews were biased towards those straight out of school, since that's the only place you would write toy apps that use basal language features. On the other hand, I've got a ton of my own libraries on my Github and some of them are pretty popular. It's abundantly clear that I wrote all of these since they're all published in official package managers with my private keys. I see things from the other direction. An employer too lazy to check my github and verify my credentials probably sucks.
- cr0sh 10y ago
  > I don't even remember the calls to string split and console display in some of my favorite languages. Provided you understand what is being asked, you shouldn't even need to remember a "favorite language". Most of the time, the interviewer doesn't care if you know something specific in a particular language, but rather you understand the process. Heck - do it with a flowchart/process diagram, or write it in a "custom" pseudocode - provided that it looks like it will produce the proper output based on the input and the detailed process, it should be considered "correct" enough to get you at least past the first stage. There are tons of people who claim to be programmers/coders/developers who can't even logically diagram a process or pseudocode a function/class/whatever. If they can't do that, they have some fundamental issues in their skill set that they need to remedy stat.
- flukus 10y ago
  > I don't even remember the calls to string split and console display in some of my favorite languages. On a whiteboard you don't have to, pseudo code with "print fizz" is sufficient. Aside from that, it does sound like you are too framework/tooling dependent if you can't work without maven and an IDE.
mightybyte 10y ago
Yes, github can be falsified, but that doesn't mean it's useless. It's really not that hard to look at a repo and figure out if the things there are likely to be real or not. Is it an absolute guarantee? No. Should an extensive github history be an absolute requirement for jobs? No. But it can be a tremendous source of signal if you spent a few minutes actually looking at commits to find some evidence that the person has committed non-trivial code for a project that is not something generic that could be found on StackOverflow.
- paulryanrogers 10y ago
  "But it can be a tremendous source of signal..." Does relying upon such a signal risk putting minorites--who typically have less free time--at a disadvantage?
  - mightybyte 10y ago
    I don't consider github to be a requirement and am completely open to hiring people who don't have any work on github. I view github code as a sanity check that helps me reduce false negatives from the other steps in the hiring process.

by edw519 10y ago

joshwcomeau 10y ago

What's frustrating for me is I tend to choke in high-stress situations like this. Obviously you can't trust a generic todo app in someone's GitHub, but some people have a very unique set of projects and it's clear that it isn't Stack Overflow copy and paste. By not looking at their GitHub at all, you're ignoring a huge signal as to the applicant's coding ability, as well as organizational skills, git fluency, etc. EDIT: that said, even on my worst day I can handle basic fizz-buzz problems. As long as the screener is basic enough, and only step 1 in a larger process, I have no complaints.

edw519 10y ago
What's frustrating for me is I tend to choke in high-stress situations like this. I'd rather find this out in an interview than during a critical sprint. That's part of the reason I do it this way. I'm interested in your work habits and personality as much as your code. I make my interviews as "low stress" as possible. A coder who can't code for me for 15 to 30 minutes is a big red flag. you're invalidating months/years of open-source work I'm not "invalidating" anything. I'm just doing my job as efficiently as possible, which is to assess your ability to contribute to my team.
- Cpoll 10y ago
  > I'd rather find this out in an interview than during a critical sprint. I think that's a false equivalence. Crunch isn't like an evaluation. There's already a joke about this, where the interviewer is verbally abusive to the interviewee to "test if they work well under stress." (Not to say that you're doing anything wrong, just that I don't think your gleaning the correct information from such a thing).
- joshwcomeau 10y ago
  Yeah, as another commenter said, an interview is not the same as a critical sprint. I don't mean that I find coding stressful, I mean that I find the social aspect of being interviewed stressful, and it's hard for me to get in the problem-solving state of mind when in that situation. I don't think you can argue that you're not invalidating someone's experience (within the context of their suitability for the position) if you don't look at any of the code they've written outside an interview process, because it's self-evident that there are candidates that are talented programmers who cannot demonstrate those skills effectively during an interview. You can argue about how insignificant of a minority that is, though. I'm a senior developer who has been offered most of the jobs he's applied for because I _have_ been able to prove basic code literacy during an interview (and have typically surprised employers by how effective I can be, when not in an interview situation). So if the _screening_ bar is low and if you're taking steps to ensure that the handicap of social anxiety is diminished, it may be a negligible percentage of candidates.
- wott 10y ago
  > I'd rather find this out in an interview than during a critical sprint. There is no relation between those.
segmondy 10y ago
Practice, he just said fizzbuzz is a good example, if you choke up implementing fizzbuzz and consider it high stress situation. You don't deserve to a job, not even as a junior. I can understand if you were told to implement a red-black tree or some esoteric algorithm.
- joshwcomeau 10y ago
  You misunderstand. I don't mean that fizzbuzz is so challenging that it causes stress. I mean that I have social anxiety that causes my brain to not work during interviews. Doesn't matter how good you are at coding if you're having a panic attack.
rootlocus 10y ago
> By not looking at their GitHub at all, you're invalidating months/years of open-source work I disagree. Nobody is invalidating anything. Those contributions still exist, people still benefit from them and you still retain your experience. That being said, while they don't directly contribute to your employment chances in this case, your experience does.
- joshwcomeau 10y ago
  Yeah, I mean that they're invalidated in the scope of "material used to evaluate whether a candidate is suitable for the position". Definitely no arguments that open-source work has merit far beyond its impact on hireability :)

brogrammernot 10y ago

That's rather interesting. I've actually found that reading code on someone's github or checking out their personal website is great for ruling in/out candidates. When they share their github or website, I'll view the source on the website to see how they organized stuff or if they did literally copy/paste a template. With GitHub, it gives some talking points. So for example, if they have some Angular code on their GitHub and also list it as a competency on their resume I'll ask them about the project. Granted, I agree that this screener really did a poor job at seeing through this candidate's smoke screen but still, I find GitHub and personal websites useful in the interview process. It does not take away the actual parts you mentioned about walking through a problem during the interview, or talking about real situations on the job and how they would approach finding a solution.

segmondy 10y ago
do you search for code theft? 80% of the code submitted for my interviews is 100% copy and pasted. When they put in effort, they remove the comments and rename the variable.
- brogrammernot 10y ago
  If the code is "beyond" what I would expect to see at the candidate's level, yes. It's merely a talking point, it becomes super clear if someone has copied/pasted code.
cr0sh 10y ago
Here's a question - to you and anyone else who cares to respond: I like to treat my github in a certain manner, and maybe I should change that. Currently, what I have on my github is some repos that are all mine - but I also have a bunch more repos that are forks of other repos. I do this (fork other repos) because I like the repo, and some of them might be really old (that is, they haven't been changes in months or years - low commit frequency or such). So - a fork for me is kinda like "save it for later", in case the original repo goes away. Plus, IIRC, if the original repo does get updated, I'll be notified and I can pull down the changes into my repo (my only other option would be to download a zip of the master branch - but then if it did change, I wouldn't know). Should I not do this? I mean, those forked repos are clearly labeled as forks from the original repo, and I have made it clear in the past that "only these repos are mine exclusively" - but should I not do this? Should I instead have a separate github just for those kinds of forks, and keep my personal repo for only my own stuff? I'm just wondering how confusing this really is for prospective employers or others - I don't want them thinking that repos x/y/z are mine when they aren't, and miss the repos that actually are mine. It's too bad that there isn't an option to segregate this better on github, and only show your repos by default. Maybe I need to review my settings...

lj3 10y ago

This type of interview in video form courtesy of Casey Muratori: https://www.youtube.com/watch?v=cfyWvJdsDRI https://www.youtube.com/watch?v=cfyWvJdsDRI

vanadium 10y ago

This is exactly the same format I've used, with two exceptions: I _do_ look at previous work, and the "custom project" isn't done in-person to keep it low-stress. This allows me to draw parallels in terms of patterns used and other commonalities I can then further ask about and reference while going over the "custom project" for the interview, without giving an opportunity to overwhelm them with anxiety. If the code style is completely out of bounds between the two projects, it either helps me dig deeper and ask about previous environments, etc. Adaptability. It also has the rather interesting side-effect of calling out frauds, especially if the "custom project" was completed and the explanation scripted into rote memory for the interview. Asking why a pattern was used in 'X' for a similar function in 'Y' project versus the pattern used in said "custom project" usually does the trick, separating rather efficiently the people that know what they're talking about from those who are just trying to skate by.

79d697i6fdif 10y ago

You would never hire somebody like me :). I write code in IDEs usually, and never basic console apps. Always inside of a framework or a bigger project. Good luck creating a maven project archetype and getting it to work during a whiteboard session. I would choke so hard on a basic "make a console app to display these numbers". I don't even remember the calls to string split and console display in some of my favorite languages. Why? You'd be an idiot to use those over Log4net or whatever you use in real life. You're asking a guy that might know how to design rockets how to put together a bicycle. A does not imply B in this case. I've always thought whiteboard interviews were biased towards those straight out of school, since that's the only place you would write toy apps that use basal language features. On the other hand, I've got a ton of my own libraries on my Github and some of them are pretty popular. It's abundantly clear that I wrote all of these since they're all published in official package managers with my private keys. I see things from the other direction. An employer too lazy to check my github and verify my credentials probably sucks.

cr0sh 10y ago
> I don't even remember the calls to string split and console display in some of my favorite languages. Provided you understand what is being asked, you shouldn't even need to remember a "favorite language". Most of the time, the interviewer doesn't care if you know something specific in a particular language, but rather you understand the process. Heck - do it with a flowchart/process diagram, or write it in a "custom" pseudocode - provided that it looks like it will produce the proper output based on the input and the detailed process, it should be considered "correct" enough to get you at least past the first stage. There are tons of people who claim to be programmers/coders/developers who can't even logically diagram a process or pseudocode a function/class/whatever. If they can't do that, they have some fundamental issues in their skill set that they need to remedy stat.
flukus 10y ago
> I don't even remember the calls to string split and console display in some of my favorite languages. On a whiteboard you don't have to, pseudo code with "print fizz" is sufficient. Aside from that, it does sound like you are too framework/tooling dependent if you can't work without maven and an IDE.

mightybyte 10y ago

Yes, github can be falsified, but that doesn't mean it's useless. It's really not that hard to look at a repo and figure out if the things there are likely to be real or not. Is it an absolute guarantee? No. Should an extensive github history be an absolute requirement for jobs? No. But it can be a tremendous source of signal if you spent a few minutes actually looking at commits to find some evidence that the person has committed non-trivial code for a project that is not something generic that could be found on StackOverflow.

paulryanrogers 10y ago
"But it can be a tremendous source of signal..." Does relying upon such a signal risk putting minorites--who typically have less free time--at a disadvantage?
- mightybyte 10y ago
  I don't consider github to be a requirement and am completely open to hiring people who don't have any work on github. I view github code as a sanity check that helps me reduce false negatives from the other steps in the hiring process.