5 ms·

I’ve evaluated GitHub profiles and contributions in the past. As long as there’s a body of work with reasonable complexity and no big red flags like a nasty arg

I’ve evaluated GitHub profiles and contributions in the past. As long as there’s a body of work with reasonable complexity and no big red flags like a nasty argument or a bug-riddled PR, I consider it a positive signal that they can code. I specifically look at their opened issues and PR’s, rather than their commits, as a way to gauge their ability to code in a team setting. I won’t try to guess how good they are at coding from GitHub alone. Most of my own public contributions are cases of “four days studying, two hours coding.” So as an exercise in tempering my interviewer ego, I remind myself that by just looking their work, I am very likely missing a lot of the picture.

andrewstuart 6y ago
I'm pretty cynical about how any code evaluation is done. Amongst the many many ways code evaluations fail, the worst and most typical IMO is that the evaluator has an air of superiority, who marks down things they don't understand, and thinks their own coding is that of an artistic genius, approaches the tasks with zero science or rigor and is unable to articulate anything hard to back up their vague assertions coming out of the assessment "process".
- Aeolun 6y ago
  Honestly, since we hire everyone on as a contractor first, I generally tend to believe that they can code if it’s on their resume and they’re otherwise a decent person and able to work in a team. If someone doesn’t work out we just let them go after half a year. So far that hasn’t actually been necessary.
- tastroder 6y ago
  I like giving a cursory glance at candidates github profiles mostly to see what they're interested in, if they participate in open source, things like that. The particular "quality" of code is imho relatively irrelevant. Besides normal open source contributions and semi-maintained side projects my own github for example is full of one off scripts and weekend project repos that are interesting for an average of N<10 people per year, I still put those up because that N is larger than 0. You won't find the code quality your previous post asked for, or even any test code in some of them. That doesn't mean I don't have any. Just like there's great developers with zero presence on GitHub out there. It's just one of the many signals in the hiring process, as long as I don't see a major red flag in the profile it mostly informs "this person works on interesting stuff in domain X / similar interests like my other devs / extensive experience with platform Y". I don't have time to look at fancy coverage badges in all 30+ repos of a profile, there's plenty of time to talk about specifics later on. I treat github profile links like any other, say to a candidate's personal tech blog. The mere existence is nice and I'll give it a glance but that's about it. In most domains I've worked in other parts of the CV and public presence are far more informative about a candidate, shoehorning pseudo-scientific metrics about code or test quality on candidate's github profiles seems counterproductive. People have other things to do in their life than maintaining recruitment-friendly github profiles, expecting one is not much different to the odd practice of unpaid take home exercises imho.
random32840 6y ago
How do you differentiate between "reasonable developer, can deal with complexity" and "extremely good developer, will dramatically improve the codebase and produce less technical debt"? Does the distinction matter at your company, or is the primary metric "good enough"? Is this industry standard, or does it vary by company?
- munchbunny 6y ago
  How do you differentiate between "reasonable developer, can deal with complexity" and "extremely good developer, will dramatically improve the codebase and produce less technical debt"? I definitely don’t rely on their GitHub info for that. In the interview process in general, I don’t expect this with junior developers, only senior+ developers, and then I look for how deliberate the developer seems to be with design trade-offs in their coding exercises and their answers to “tell be about a time when...” questions. Does the distinction matter at your company, or is the primary metric "good enough"? The distinction matters, but in my experience interviews produce very noisy signals for this trait. (1) How often do you design API’s in half an hour? In practice the decisions that affect long term maintainability happen over days, not hours. (2) Evaluating this requires a lot of subjective judgement from interviewers, and experience tells me that no interviewer is as objective or insightful as they think, myself included. So instead of trying to answer that question, I just look for a deliberate process of enumerating and deciding trade-offs, with answers that are in the ballpark of sane. I wouldn’t call good enough a metric, more like a philosophy for hiring. I look at many unrelated traits in an interview, but I explicitly reject the idea that the interview is accurate enough to conclude something like “this developer is a net reducer of technical debt”. Is this industry standard, or does it vary by company? The interview rubrics I’ve seen all use holistic definitions of “good enough to hire”, if that’s what you’re asking, but based on what I’ve seen as an interviewee I highly doubt there’s an industry standard the way you’re asking.
  - random32840 6y ago
    Thanks for the answer. What I'm really trying to fish out is: if a developer is significantly better than the competition relative to his years of experience, how is he supposed to communicate that? How does he actually get paid commensurate to that extra effort? In your experience is it even possible? As far as I can tell the only way to really do that reliably is either work on a respected product & basically make an impression outside your company (crapshoot) or show what you can do via open source. I can't see how I could communicate it otherwise.

andrewstuart 6y ago

I'm pretty cynical about how any code evaluation is done. Amongst the many many ways code evaluations fail, the worst and most typical IMO is that the evaluator has an air of superiority, who marks down things they don't understand, and thinks their own coding is that of an artistic genius, approaches the tasks with zero science or rigor and is unable to articulate anything hard to back up their vague assertions coming out of the assessment "process".

Aeolun 6y ago
Honestly, since we hire everyone on as a contractor first, I generally tend to believe that they can code if it’s on their resume and they’re otherwise a decent person and able to work in a team. If someone doesn’t work out we just let them go after half a year. So far that hasn’t actually been necessary.
tastroder 6y ago
I like giving a cursory glance at candidates github profiles mostly to see what they're interested in, if they participate in open source, things like that. The particular "quality" of code is imho relatively irrelevant. Besides normal open source contributions and semi-maintained side projects my own github for example is full of one off scripts and weekend project repos that are interesting for an average of N<10 people per year, I still put those up because that N is larger than 0. You won't find the code quality your previous post asked for, or even any test code in some of them. That doesn't mean I don't have any. Just like there's great developers with zero presence on GitHub out there. It's just one of the many signals in the hiring process, as long as I don't see a major red flag in the profile it mostly informs "this person works on interesting stuff in domain X / similar interests like my other devs / extensive experience with platform Y". I don't have time to look at fancy coverage badges in all 30+ repos of a profile, there's plenty of time to talk about specifics later on. I treat github profile links like any other, say to a candidate's personal tech blog. The mere existence is nice and I'll give it a glance but that's about it. In most domains I've worked in other parts of the CV and public presence are far more informative about a candidate, shoehorning pseudo-scientific metrics about code or test quality on candidate's github profiles seems counterproductive. People have other things to do in their life than maintaining recruitment-friendly github profiles, expecting one is not much different to the odd practice of unpaid take home exercises imho.

random32840 6y ago

How do you differentiate between "reasonable developer, can deal with complexity" and "extremely good developer, will dramatically improve the codebase and produce less technical debt"? Does the distinction matter at your company, or is the primary metric "good enough"? Is this industry standard, or does it vary by company?

munchbunny 6y ago
How do you differentiate between "reasonable developer, can deal with complexity" and "extremely good developer, will dramatically improve the codebase and produce less technical debt"? I definitely don’t rely on their GitHub info for that. In the interview process in general, I don’t expect this with junior developers, only senior+ developers, and then I look for how deliberate the developer seems to be with design trade-offs in their coding exercises and their answers to “tell be about a time when...” questions. Does the distinction matter at your company, or is the primary metric "good enough"? The distinction matters, but in my experience interviews produce very noisy signals for this trait. (1) How often do you design API’s in half an hour? In practice the decisions that affect long term maintainability happen over days, not hours. (2) Evaluating this requires a lot of subjective judgement from interviewers, and experience tells me that no interviewer is as objective or insightful as they think, myself included. So instead of trying to answer that question, I just look for a deliberate process of enumerating and deciding trade-offs, with answers that are in the ballpark of sane. I wouldn’t call good enough a metric, more like a philosophy for hiring. I look at many unrelated traits in an interview, but I explicitly reject the idea that the interview is accurate enough to conclude something like “this developer is a net reducer of technical debt”. Is this industry standard, or does it vary by company? The interview rubrics I’ve seen all use holistic definitions of “good enough to hire”, if that’s what you’re asking, but based on what I’ve seen as an interviewee I highly doubt there’s an industry standard the way you’re asking.
- random32840 6y ago
  Thanks for the answer. What I'm really trying to fish out is: if a developer is significantly better than the competition relative to his years of experience, how is he supposed to communicate that? How does he actually get paid commensurate to that extra effort? In your experience is it even possible? As far as I can tell the only way to really do that reliably is either work on a respected product & basically make an impression outside your company (crapshoot) or show what you can do via open source. I can't see how I could communicate it otherwise.