11 ms·

We used this a couple times at Sandstorm back in the day. At the end of the interview the candidate would play Factorio cooperatively with the team for a while.

We used this a couple times at Sandstorm back in the day. At the end of the interview the candidate would play Factorio cooperatively with the team for a while. I think it is remarkably effective at identifying the kinds of skills and personality traits that a software engineer actually needs to have in day-to-day work. You can find out if someone is self-directed, how fast they work, whether they produce clean designs or spaghetti code, whether they are good at cooperating or tend to go off on their own, etc. Some people will just sit and watch and do nothing unless instructed... that's bad. Some people will build stuff, but with obvious efficiency flaws and "bugs"... also bad. Some people identify what needs doing and get it done effectively but without trying to be perfect... that's good. Factorio essentially compresses real-world work patterns into a shorter time period, giving you the chance to see how someone works in the space of a couple hours. I don't know anything else that does that. But, obviously, it's problematic. A person who has played the game before will have a big advantage. A person who doesn't play video games at all will have a big disadvantage. For these reasons, I don't think I could seriously recommend this approach be adopted more widely. But then, I don't know of any approach to interviewing that I think is fair. Everything has problems: * Regular interviews bias towards people that are charismatic, not effective. * Coding interviews bias towards people who can code under pressure with someone looking over their shoulder, which is almost nothing like real-world coding. * Puzzle-y algorithms questions identify people good at clever solutions but that's hardly what most people need to do day-to-day when coding. * Take-home assignments bias towards people who have lots of free time, and still don't really tell you how that person works with others. * Looking at GitHub history biases against people whose lives are too busy to code in their free time (e.g. maybe they have kids) and who weren't lucky enough to have a previous job that touched a lot of open source. * Looking at past work experience misses brilliant junior programmers coming out of college. * Don't even think about using ML for this, that doesn't solve biases, it just hides them in a black box. It seems like there's no right answer here.

rhacker 6y ago
There really isn't. At the end of the day every company I have been in has: the asshole, the smart guy, the idiot, the guy everyone is scared of, the guy that can't be fired ("the originals"). Then you just kind of multiply them randomly depending on team size. These hiring articles are a huge pointless waste of time.
- collyw 6y ago
  My company had all of those rolled into one. Thankfully he burned himself out with the technical debt that he had created and quit. He was smart and could probably beat me at hacker rank type challenges. I know to focus on maintainability over cleverness.
- SamBam 6y ago
  Makes me feel lucky to be in my job, and a jolt of gratitude every-so-often is a welcome thing, so thank you. I work with 8 other senior or senior-ish engineers, we are all basically taking a slight paycut (not too much, maybe 80-90% of rates we might get elsewhere) to work for a non-profit making interesting stuff for kids, and everyone is genuinely nice and fairly hard-working. I wonder if the "taking a paycut to do something actually interesting and worthwhile" is part of it.
  - iAmAPencilYo 6y ago
    Hi, could you share a little of what you do? It didn't even cross my mind I can work for a non-profit, still get paid (not top dollar) and have a good community impact (kids in your case). Thanks!
    - SamBam 6y ago
      I work for a grant-funded educational software company that makes science simulations for K through college, but mostly middle school and high school.
plank_time 6y ago
Hire quickly and fire quickly with a generous severance. That’s the only method that I’ve seen works effectively. Netflix does more with 2000 engineers that companies with 10-20x that. This also lets you take chances on engineers that weren’t “perfect” but had a lot of positives to offer.
- inetknght 6y ago
  > hire quickly and fire quickly This kind of mentality is not conducive to a well-functioning nation with people who have dependents and need stability.
  - plank_time 6y ago
    Saying that it isn’t conducive to an entire nation is a bit of an over-dramatization, don’t you think? I think a generous severance makes up for that. Plus people would know what they are getting into, just like Netflix. If you don’t want the risk/return you don’t have to apply or accept a job there.
    - inetknght 6y ago
      > Saying that it isn’t conducive to an entire nation is a bit of an over-dramatization, don’t you think? I don't think so. > I think a generous severance makes up for that. That's your opinion. But what you think is generous isn't necessarily what others think is generous. > Plus people would know what they are getting into, just like Netflix. I disagree. > If you don’t want the risk/return you don’t have to apply or accept a job there. The risk/return needs to be clear though and hirefast/firefast isn't always clear.
  - namdnay 6y ago
    If the nation is well-functioning, getting fired isn’t the end of the world. If your job is independant of your healthcare and you have several years of allowance to seek another one it’s fine to try out a few jobs before finding what sticks
  - dillondoyle 6y ago
billfruit 6y ago
While I get some of this sentiment, wont some sort of social deduction game like 'Throne of Lies' be better at identifying potentially good team members?.
- kentonv 6y ago
  I'm not familiar with that game, but the point of using Factorio is it is a game where you have to build complex systems and maintain them, which has many parallels with real-world software engineering. "Throne of Lies" doesn't look like that kind of game.
  - billfruit 6y ago
    Throne of lies and social deduction games like them are about finding out peoples real intentions from what they say and do, also aspects like using and understanding subtle signalling to collaborate with ones team.
    - Shish2k 6y ago
      In my software engineering work I do a lot more “writing documentation to be as clear as possible” than “social deduction and subtle signalling” ^^;;
      - liaukovv 6y ago
        In fact i would hate to work at a place where you have to do social deduction
prepend 6y ago
> Some people will just sit and watch and do nothing unless instructed... that's bad. Some people will build stuff, but with obvious efficiency flaws and "bugs"... also bad. Some people identify what needs doing and get it done effectively but without trying to be perfect... that's good. Some personalities are better suited for programming, although I think hard work can overcome anything. I remember an old blog post [0] of mine that used Hammerstein-Equord personality quadrants of smart/industrious, smart/lazy, stupid/industrious, stupid/lazy and describing it for how that effects tech team planning. [0] http://www.prepend.com/2009/05/programmer-classifications-by-quadrant.html http://www.prepend.com/2009/05/programmer-classifications-by...
- collyw 6y ago
  > Some personalities are better suited for programming, although I think hard work can overcome anything. I am not sure about the hard work part. Some people don't seem to think the right way for programming. I studied a one year IT masters conversion course back in the dot com boom, and chose the best / hardest course I could find. The first 3 weeks of the course was an intensive Java programming course. There was an exam at the end of this. They suggested that anyone who didn't pass this exam, don't do the rest of the course, but they didn't actually force them off. I was friends with at least two people that failed that exam, but continued anyway. They struggled and I don't think they passed the final exam despite plenty of hard work.
  - prepend 6y ago
    I think hard work and studying is probably better than natural aptitude and no hard work. I’ve worked with people who didn’t care at all, it was a job, but they studied and worked hard and were pretty good. Of course, I’d rather work with someone who had both.
collyw 6y ago
I had a pleasant interview experience one time, that avoided a number of those problems. I was asked to bring in my laptop with some code I had written. I did, we talked about it. The interviewer asked me to add a simple feature. It's code I was familiar with (as you would be after working for a company for a bit). So no stress. No gotchas. No time wasted with yet another technical test with a throwaway app. Setting up a new project is always a bit of a faff. Admittedly not everyone will have some code ready, but then a normal take home test would be the fallback.
CamperBob2 6y ago
It seems like there's no right answer here. I think you've done a good job identifying the most common options. Since none has proven decisively better than any other at aiding hiring decisions, how about letting the candidate decide which sort of interview to participate in?
johncessna 6y ago
> But then, I don't know of any approach to interviewing that I think is fair The companies that think deeply about interviews are aware that their interview is biased, and they do so intentionally. At a certain point watching 5 good engineers not meet the bar is better than having 10 bad engineers get through. Maybe the 'right answer' is to let your needs drive the bias of your interview. Do you have a position that needs to deal with immense pressure with management breathing down their neck? Bias your selection with whiteboard interviews. Are you looking for a principal engineer that can drive adaptation and change in a large org? Bias your selection process to get that. tl;dr Maybe the issue is the one size fits all thinking to the tech interview.
kilroy123 6y ago
You really hit the nail on the head here. I couldn't agree more. However, I think playing this game would be a fun and interesting interview.
jackietreehorn 6y ago
We used it at Cloudflare a few times too (as suggested by former Sandstorm people). I still remember getting the purchase of the game approved by engineering leadership. Was pretty humorous where it went from wanting to understand to whatever approved in like 30 seconds. I agree with your assessment of the pros/cons of the various interview methods. I agree that there is no right answer.
- kentonv 6y ago
  Huh I actually had no idea it was tried at Cloudflare!
BurningFrog 6y ago
A reminder that the goal of an interview process is not to be fair. This is not a sports event or a court case. Mostly your goals align with "being fair", but not always, and that's OK.
- jtwaleson 6y ago
  I agree, so upvoted. The most important question is: "Would I like to work with this person?". That is unfortunately unfair.
- mwcampbell 6y ago
  > A reminder that the goal of an interview process is not to be fair. This is not a sports event or a court case. Given the power that companies have over individual workers, I'd say that a job interview should be treated as seriously as a court case. So yes, making it fair should be an explicit goal.
  - thereisnospork 6y ago
    A company's goal isn't to hire every good potential employee, it is to hire a number of good employees, and avoid hiring bad ones. If Factorio wizards make good engineers, who cares if only 1/10 good engineers have even played?
    - RogerL 6y ago
      Who cares if they are all white 23yo frat bros? who cares if no woman can get hired so long as the company makes hires it likes. Or gay, or Jewish, or whatever busy is in your selection. I'm not asserting there's a specific bias in this particular interview choice, but I am asserting that if you were taking a fairly narrow part of the population you better be working damned hard that you don't have hiring biases that are not relevant to job performance
deleted 6y ago
[deleted]
mwcampbell 6y ago
> A person who doesn't play video games at all will have a big disadvantage. Even more so for a person who can't play video games, e.g. blind people. Many of the other interview techniques, for all their faults, at least don't (inadvertently) discriminate against people with disabilities.
gtirloni 6y ago
> person who doesn't play video games at all will have a big disadvantage My thoughts exactly while reading the article.
brightball 6y ago
I've honestly simplified my approach to interviewing over the last few years and I'm pretty happy with the results. I just have a conversation and ask them to tell me about a favorite code project they've worked on. Could be school project, previous job, whatever. And just let them talk about it. I'll prod for more detail if I need to give a little nudge. Ask about problems they solved when doing it, etc. The _only_ thing that I'm looking for in this process is whether or not the person is truly interested what they're doing. This comes through when talking about something they're excited to have worked on, seemingly small problems that they had to really work to solve, etc. Geeking out, if you will. The single most important trait I've seen in programmers that I've worked with over the years is "level of interest". There's too much that you have to learn on a daily basis to be successful in this field. If you're not interested in learning new things and finding solutions to the problems that you encounter, you're not going to make it very far. Beyond that and basic competence in the core skills that the job requires, I don't look for much more. Thus far, everybody I've ever hired who has passed this little conversational test has been an excellent team member.
- ChrisMarshallNY 6y ago
  This. I was a hiring manager for 25 years, and this was exactly what I did. I made very, very few mistakes, and actually had some pleasant surprises.
  - thaumasiotes 6y ago
    > I made very, very few mistakes How do you know? How often did you follow up on rejections to see whether they should have been hires?
    - ChrisMarshallNY 6y ago
      Actually, for the company that I worked at, and the level of folks we hired, I often heard about how they went. The only one that made me sad, was a chap who used to show up on the Adobe Photoshop splash page. We offered, he declined. In the aggregate, I think things turned out OK for all of us. There are some folks that I have to admit, probably did best by not accepting us.
cesarb 6y ago
> A person who has played the game before will have a big advantage. A person who doesn't play video games at all will have a big disadvantage. There's also a "similar game" advantage. For instance, a person who played games like OpenTTD before will have a much easier time with Factorio rail signals, since the two kinds of rail signals in Factorio work very similarly to two of the several kinds of rail signals found in OpenTTD.
trhway 6y ago
so, you've pointed 7 flawed existing approaches as the reason to come up with and use an 8th one. At least those 7 have actual connection to programming. Your 8th one has only imaginary indirect connection only through your internal interpretation. It is just like those box-building/etc. games they play at Scrum training which have nothing in common with reality. And many flaws in the 7 aren't intrinsic flaws, instead it is flaws of the wrong "one size fits all" application: > Looking at past work experience misses brilliant junior programmers coming out of college. yeah, don't use that as negative for junior programmers coming out of college. >Take-home assignments bias towards people who have lots of free time, and still don't really tell you how that person works with others. the same, if candidate prefers that way - let them, if not - use other approach etc.
KingEllis 6y ago
> * Don't even think about using ML for this, that doesn't solve biases, it just hides them in a black box. Can you clarify: what is "ML" here? "Machine learning", ML the language, MatLab, "machine language" (a.k.a. assembly), Matt LeBlanc? I'm out of ideas...
- spartanatreyu 6y ago
  Machine learning
ilaksh 6y ago
I think the answer is that it takes time to get to know someone and their skills. There is no quick simple test. You can give them multiple tests over time if you want. I think that if you have money for recruiting then you can screen with an initial test, then pay them for a few hours or days of additional tests. And then ideally there is a contracting period that is an additional test but on a real project.
WorldMaker 6y ago
> Regular interviews bias towards people that are charismatic, not effective. A cooperative multiplayer game is also going to be a similar soft skills check that favors people that can communicate not just well, but also charismatically. That said, a possible advantage to doing that in a game, as opposed to a regular soft skills interview (or worse trying to force such soft skill checks into the pressure cooker of a whiteboard interview or puzzle questions) is that you maybe provide an environment where the interviewee can be dopamine hacked into believing they can relax and you maybe get a glimpse into how the person's soft skills check out on the other side of that boundary where interviewee knows that they need to be "on" and acting. I'm not actually sure if that is good or bad, but a lot of my takeaway from this article and other comments in these threads is it sort of feels like there's a lot of excuses for "pick a game that seems technically similar", but most of the "good feedback" that seems to be coming back from these experiences resembles what history tells us about regular interviews versus technical interviews: soft skills matter a lot more than what you can gauge technically in an interview. Most technical interviews are just soft skills interviews under the pressure cooker of taking a non-standardized pop quiz your professor decided you need to take because you showed up to class in only your underwear (and also you are flying for some reason in this nightmare). Cooperative games possibly present the opposite of the pressure cooker: dopamine hacking the interviewee an excuse to let the pressure seem like it is off while you are still doing that same sort of soft skills check. (Aside: problem solving is classically a "soft skill". "Behavioral interviews" were as much about problem solving decades before the software industry invented the "technical" interview.) (Further aside: This sort of dopamine hacking is very familiar to anyone that's seen Fraternity/Sorority rushes. Soft skills are hugely important for "group fit"/"team fit" in college social groups as they are in employment groups, and seeing how potential new members interact in "fun"/"game" situations often tells you more about the person than if you sat down and interviewed them one-on-one in a suit and tie or equivalent and they know they have to be "on". Rushes include both of those things for many reasons.)
johnsmith4739 6y ago
> I don't know of any approach to interviewing that I think is fair. That is true, because it was my motivation to build a wizard for behavioural interviews designed to remove biases and be objective. In practice, most of the time people are not even sure what they are looking for - I know it when I see it - and default to gut hiring. This leads to teams made of people that think like you, so you're basically paying 2 salaries for one brain. My formula for removing biases is in 2 steps: first define explicitly the behavioural traits you desire, and second, use falsifiable questions to determine the truth. Because we're here to help each other, if this is relevant to you, I'd be happy to share this solution with you.
- geoduck14 6y ago
  > if this is relevant to you, I'd be happy to share this solution with you Yes, please!
  - johnsmith4739 6y ago
    I hope it is ok to share this here: https://freyasense.com/recruiting/ https://freyasense.com/recruiting/ Give me a sign and I would be happy to give you a tour
jackbravo 6y ago
> Coding interviews bias towards people who can code under pressure with someone looking over their shoulder, which is almost nothing like real-world coding. I don't understand this. Why instead play a game, which, as you said, could have disadvantages because there are people that do play and people who've never played. I guess the thinking goes that by easing into a game you can see clear behaviors of people. But you can also try to ease them into coding while you are looking over their shoulder. The interviews I do, and I've seen done at my previous company and the one I'm currently at, it is similar to pair programming, and yes, there is people that is super nervous at the beginning, and that seem to need a lot of hand holding at the beginning, but after coding and having a few small wins start to ease at the test and then they can surprise you with very nice coding or interactions on solutions.

rhacker 6y ago

There really isn't. At the end of the day every company I have been in has: the asshole, the smart guy, the idiot, the guy everyone is scared of, the guy that can't be fired ("the originals"). Then you just kind of multiply them randomly depending on team size. These hiring articles are a huge pointless waste of time.

collyw 6y ago
My company had all of those rolled into one. Thankfully he burned himself out with the technical debt that he had created and quit. He was smart and could probably beat me at hacker rank type challenges. I know to focus on maintainability over cleverness.
SamBam 6y ago
Makes me feel lucky to be in my job, and a jolt of gratitude every-so-often is a welcome thing, so thank you. I work with 8 other senior or senior-ish engineers, we are all basically taking a slight paycut (not too much, maybe 80-90% of rates we might get elsewhere) to work for a non-profit making interesting stuff for kids, and everyone is genuinely nice and fairly hard-working. I wonder if the "taking a paycut to do something actually interesting and worthwhile" is part of it.
- iAmAPencilYo 6y ago
  Hi, could you share a little of what you do? It didn't even cross my mind I can work for a non-profit, still get paid (not top dollar) and have a good community impact (kids in your case). Thanks!
  - SamBam 6y ago
    I work for a grant-funded educational software company that makes science simulations for K through college, but mostly middle school and high school.

plank_time 6y ago

Hire quickly and fire quickly with a generous severance. That’s the only method that I’ve seen works effectively. Netflix does more with 2000 engineers that companies with 10-20x that. This also lets you take chances on engineers that weren’t “perfect” but had a lot of positives to offer.

inetknght 6y ago
> hire quickly and fire quickly This kind of mentality is not conducive to a well-functioning nation with people who have dependents and need stability.
- plank_time 6y ago
  Saying that it isn’t conducive to an entire nation is a bit of an over-dramatization, don’t you think? I think a generous severance makes up for that. Plus people would know what they are getting into, just like Netflix. If you don’t want the risk/return you don’t have to apply or accept a job there.
  - inetknght 6y ago
    > Saying that it isn’t conducive to an entire nation is a bit of an over-dramatization, don’t you think? I don't think so. > I think a generous severance makes up for that. That's your opinion. But what you think is generous isn't necessarily what others think is generous. > Plus people would know what they are getting into, just like Netflix. I disagree. > If you don’t want the risk/return you don’t have to apply or accept a job there. The risk/return needs to be clear though and hirefast/firefast isn't always clear.
- namdnay 6y ago
  If the nation is well-functioning, getting fired isn’t the end of the world. If your job is independant of your healthcare and you have several years of allowance to seek another one it’s fine to try out a few jobs before finding what sticks
- dillondoyle 6y ago

billfruit 6y ago

While I get some of this sentiment, wont some sort of social deduction game like 'Throne of Lies' be better at identifying potentially good team members?.

kentonv 6y ago
I'm not familiar with that game, but the point of using Factorio is it is a game where you have to build complex systems and maintain them, which has many parallels with real-world software engineering. "Throne of Lies" doesn't look like that kind of game.
- billfruit 6y ago
  Throne of lies and social deduction games like them are about finding out peoples real intentions from what they say and do, also aspects like using and understanding subtle signalling to collaborate with ones team.
  - Shish2k 6y ago
    In my software engineering work I do a lot more “writing documentation to be as clear as possible” than “social deduction and subtle signalling” ^^;;
    - liaukovv 6y ago
      In fact i would hate to work at a place where you have to do social deduction

prepend 6y ago

> Some people will just sit and watch and do nothing unless instructed... that's bad. Some people will build stuff, but with obvious efficiency flaws and "bugs"... also bad. Some people identify what needs doing and get it done effectively but without trying to be perfect... that's good. Some personalities are better suited for programming, although I think hard work can overcome anything. I remember an old blog post [0] of mine that used Hammerstein-Equord personality quadrants of smart/industrious, smart/lazy, stupid/industrious, stupid/lazy and describing it for how that effects tech team planning. [0] http://www.prepend.com/2009/05/programmer-classifications-by-quadrant.html http://www.prepend.com/2009/05/programmer-classifications-by...

collyw 6y ago
> Some personalities are better suited for programming, although I think hard work can overcome anything. I am not sure about the hard work part. Some people don't seem to think the right way for programming. I studied a one year IT masters conversion course back in the dot com boom, and chose the best / hardest course I could find. The first 3 weeks of the course was an intensive Java programming course. There was an exam at the end of this. They suggested that anyone who didn't pass this exam, don't do the rest of the course, but they didn't actually force them off. I was friends with at least two people that failed that exam, but continued anyway. They struggled and I don't think they passed the final exam despite plenty of hard work.
- prepend 6y ago
  I think hard work and studying is probably better than natural aptitude and no hard work. I’ve worked with people who didn’t care at all, it was a job, but they studied and worked hard and were pretty good. Of course, I’d rather work with someone who had both.

collyw 6y ago

I had a pleasant interview experience one time, that avoided a number of those problems. I was asked to bring in my laptop with some code I had written. I did, we talked about it. The interviewer asked me to add a simple feature. It's code I was familiar with (as you would be after working for a company for a bit). So no stress. No gotchas. No time wasted with yet another technical test with a throwaway app. Setting up a new project is always a bit of a faff. Admittedly not everyone will have some code ready, but then a normal take home test would be the fallback.

CamperBob2 6y ago

It seems like there's no right answer here. I think you've done a good job identifying the most common options. Since none has proven decisively better than any other at aiding hiring decisions, how about letting the candidate decide which sort of interview to participate in?

johncessna 6y ago

> But then, I don't know of any approach to interviewing that I think is fair The companies that think deeply about interviews are aware that their interview is biased, and they do so intentionally. At a certain point watching 5 good engineers not meet the bar is better than having 10 bad engineers get through. Maybe the 'right answer' is to let your needs drive the bias of your interview. Do you have a position that needs to deal with immense pressure with management breathing down their neck? Bias your selection with whiteboard interviews. Are you looking for a principal engineer that can drive adaptation and change in a large org? Bias your selection process to get that. tl;dr Maybe the issue is the one size fits all thinking to the tech interview.

kilroy123 6y ago

You really hit the nail on the head here. I couldn't agree more. However, I think playing this game would be a fun and interesting interview.

jackietreehorn 6y ago

We used it at Cloudflare a few times too (as suggested by former Sandstorm people). I still remember getting the purchase of the game approved by engineering leadership. Was pretty humorous where it went from wanting to understand to whatever approved in like 30 seconds. I agree with your assessment of the pros/cons of the various interview methods. I agree that there is no right answer.

kentonv 6y ago
Huh I actually had no idea it was tried at Cloudflare!

BurningFrog 6y ago

A reminder that the goal of an interview process is not to be fair. This is not a sports event or a court case. Mostly your goals align with "being fair", but not always, and that's OK.

jtwaleson 6y ago
I agree, so upvoted. The most important question is: "Would I like to work with this person?". That is unfortunately unfair.
mwcampbell 6y ago
> A reminder that the goal of an interview process is not to be fair. This is not a sports event or a court case. Given the power that companies have over individual workers, I'd say that a job interview should be treated as seriously as a court case. So yes, making it fair should be an explicit goal.
- thereisnospork 6y ago
  A company's goal isn't to hire every good potential employee, it is to hire a number of good employees, and avoid hiring bad ones. If Factorio wizards make good engineers, who cares if only 1/10 good engineers have even played?
  - RogerL 6y ago
    Who cares if they are all white 23yo frat bros? who cares if no woman can get hired so long as the company makes hires it likes. Or gay, or Jewish, or whatever busy is in your selection. I'm not asserting there's a specific bias in this particular interview choice, but I am asserting that if you were taking a fairly narrow part of the population you better be working damned hard that you don't have hiring biases that are not relevant to job performance

deleted 6y ago

[deleted]

mwcampbell 6y ago

> A person who doesn't play video games at all will have a big disadvantage. Even more so for a person who can't play video games, e.g. blind people. Many of the other interview techniques, for all their faults, at least don't (inadvertently) discriminate against people with disabilities.

gtirloni 6y ago

> person who doesn't play video games at all will have a big disadvantage My thoughts exactly while reading the article.

brightball 6y ago

I've honestly simplified my approach to interviewing over the last few years and I'm pretty happy with the results. I just have a conversation and ask them to tell me about a favorite code project they've worked on. Could be school project, previous job, whatever. And just let them talk about it. I'll prod for more detail if I need to give a little nudge. Ask about problems they solved when doing it, etc. The _only_ thing that I'm looking for in this process is whether or not the person is truly interested what they're doing. This comes through when talking about something they're excited to have worked on, seemingly small problems that they had to really work to solve, etc. Geeking out, if you will. The single most important trait I've seen in programmers that I've worked with over the years is "level of interest". There's too much that you have to learn on a daily basis to be successful in this field. If you're not interested in learning new things and finding solutions to the problems that you encounter, you're not going to make it very far. Beyond that and basic competence in the core skills that the job requires, I don't look for much more. Thus far, everybody I've ever hired who has passed this little conversational test has been an excellent team member.

ChrisMarshallNY 6y ago
This. I was a hiring manager for 25 years, and this was exactly what I did. I made very, very few mistakes, and actually had some pleasant surprises.
- thaumasiotes 6y ago
  > I made very, very few mistakes How do you know? How often did you follow up on rejections to see whether they should have been hires?
  - ChrisMarshallNY 6y ago
    Actually, for the company that I worked at, and the level of folks we hired, I often heard about how they went. The only one that made me sad, was a chap who used to show up on the Adobe Photoshop splash page. We offered, he declined. In the aggregate, I think things turned out OK for all of us. There are some folks that I have to admit, probably did best by not accepting us.

cesarb 6y ago

> A person who has played the game before will have a big advantage. A person who doesn't play video games at all will have a big disadvantage. There's also a "similar game" advantage. For instance, a person who played games like OpenTTD before will have a much easier time with Factorio rail signals, since the two kinds of rail signals in Factorio work very similarly to two of the several kinds of rail signals found in OpenTTD.

trhway 6y ago

so, you've pointed 7 flawed existing approaches as the reason to come up with and use an 8th one. At least those 7 have actual connection to programming. Your 8th one has only imaginary indirect connection only through your internal interpretation. It is just like those box-building/etc. games they play at Scrum training which have nothing in common with reality. And many flaws in the 7 aren't intrinsic flaws, instead it is flaws of the wrong "one size fits all" application: > Looking at past work experience misses brilliant junior programmers coming out of college. yeah, don't use that as negative for junior programmers coming out of college. >Take-home assignments bias towards people who have lots of free time, and still don't really tell you how that person works with others. the same, if candidate prefers that way - let them, if not - use other approach etc.

KingEllis 6y ago

> * Don't even think about using ML for this, that doesn't solve biases, it just hides them in a black box. Can you clarify: what is "ML" here? "Machine learning", ML the language, MatLab, "machine language" (a.k.a. assembly), Matt LeBlanc? I'm out of ideas...

spartanatreyu 6y ago
Machine learning

ilaksh 6y ago

I think the answer is that it takes time to get to know someone and their skills. There is no quick simple test. You can give them multiple tests over time if you want. I think that if you have money for recruiting then you can screen with an initial test, then pay them for a few hours or days of additional tests. And then ideally there is a contracting period that is an additional test but on a real project.

WorldMaker 6y ago

> Regular interviews bias towards people that are charismatic, not effective. A cooperative multiplayer game is also going to be a similar soft skills check that favors people that can communicate not just well, but also charismatically. That said, a possible advantage to doing that in a game, as opposed to a regular soft skills interview (or worse trying to force such soft skill checks into the pressure cooker of a whiteboard interview or puzzle questions) is that you maybe provide an environment where the interviewee can be dopamine hacked into believing they can relax and you maybe get a glimpse into how the person's soft skills check out on the other side of that boundary where interviewee knows that they need to be "on" and acting. I'm not actually sure if that is good or bad, but a lot of my takeaway from this article and other comments in these threads is it sort of feels like there's a lot of excuses for "pick a game that seems technically similar", but most of the "good feedback" that seems to be coming back from these experiences resembles what history tells us about regular interviews versus technical interviews: soft skills matter a lot more than what you can gauge technically in an interview. Most technical interviews are just soft skills interviews under the pressure cooker of taking a non-standardized pop quiz your professor decided you need to take because you showed up to class in only your underwear (and also you are flying for some reason in this nightmare). Cooperative games possibly present the opposite of the pressure cooker: dopamine hacking the interviewee an excuse to let the pressure seem like it is off while you are still doing that same sort of soft skills check. (Aside: problem solving is classically a "soft skill". "Behavioral interviews" were as much about problem solving decades before the software industry invented the "technical" interview.) (Further aside: This sort of dopamine hacking is very familiar to anyone that's seen Fraternity/Sorority rushes. Soft skills are hugely important for "group fit"/"team fit" in college social groups as they are in employment groups, and seeing how potential new members interact in "fun"/"game" situations often tells you more about the person than if you sat down and interviewed them one-on-one in a suit and tie or equivalent and they know they have to be "on". Rushes include both of those things for many reasons.)

johnsmith4739 6y ago

> I don't know of any approach to interviewing that I think is fair. That is true, because it was my motivation to build a wizard for behavioural interviews designed to remove biases and be objective. In practice, most of the time people are not even sure what they are looking for - I know it when I see it - and default to gut hiring. This leads to teams made of people that think like you, so you're basically paying 2 salaries for one brain. My formula for removing biases is in 2 steps: first define explicitly the behavioural traits you desire, and second, use falsifiable questions to determine the truth. Because we're here to help each other, if this is relevant to you, I'd be happy to share this solution with you.

geoduck14 6y ago
> if this is relevant to you, I'd be happy to share this solution with you Yes, please!
- johnsmith4739 6y ago
  I hope it is ok to share this here: https://freyasense.com/recruiting/ https://freyasense.com/recruiting/ Give me a sign and I would be happy to give you a tour

jackbravo 6y ago

> Coding interviews bias towards people who can code under pressure with someone looking over their shoulder, which is almost nothing like real-world coding. I don't understand this. Why instead play a game, which, as you said, could have disadvantages because there are people that do play and people who've never played. I guess the thinking goes that by easing into a game you can see clear behaviors of people. But you can also try to ease them into coding while you are looking over their shoulder. The interviews I do, and I've seen done at my previous company and the one I'm currently at, it is similar to pair programming, and yes, there is people that is super nervous at the beginning, and that seem to need a lot of hand holding at the beginning, but after coding and having a few small wins start to ease at the test and then they can surprise you with very nice coding or interactions on solutions.