5 ms·

"As part of our hiring process at Mixer Labs, we would often give people a half day coding exercise. " I (and many people who I've hired) don't have time to wa

"As part of our hiring process at Mixer Labs, we would often give people a half day coding exercise. " I (and many people who I've hired) don't have time to waste on your startup. I can tell within two minutes if someone can code or not. I've never understood the need to have someone write code (and then nitpickingly critique the code because it doesn't look like my code) when I can just ask them and know if they are lying or not. And I've hired a LOT of super fantastic programmers.

eladgil 15y ago
What are the key things in your experience that let you know if someone is good or not as a long term fit with your company? Our test was not primarily just code quality. When we brought people in we would look at: 1. Approach. Did they use e.g. an open source pre-existing module or write something themselves? Did they come up with a creative solution? 2. Culture fit. While they were in working with us, how did they interact with the team? Did they enjoy working with us and vice versa? 3. Productivity. How much did they actually get done?
- rubashov 15y ago
  This stuff just seems easy to figure out from a resume and asking questions. If they can talk competently in detail about cool stuff on their resume, and you yourself are smart enough to spot BS, then you know they get stuff done. As for culture fit, again, a socially competent interviewer can size these things up reasonably quickly.
- owenmarshall 15y ago
  There's no way to figure this out from a one on one interview with that person? I mean, a half-day programming test feels excessive. Are you worried about loosing good candidates that just don't want to deal with the headaches? I've heard people try to justify insane hiring processes by saying "we want to filter out the people that don't want to work here. We only want people that really want it." But the best programmers -- the ones you want to hire -- are going to be in high demand by companies that look at their github, have a three minute interview, and pull the trigger.
  - eladgil 15y ago
    We would only do the half day test with a subset of the candidates. A lot of people actually welcomed it - we would explain it was a way for them to also get to know us, see what we were like to work with etc. A lot of people about to make a long term commitment to a company also want to make sure the company is right for them. So this was a good way to get to know one another.
    - pmiller2 15y ago
      Which subset, and why only those people?
- silverbax88 15y ago
  One of the fastest ways is to ask about something they had to "fix" (use the word "fix") at their last job, or some problem in a pet project that they just figured out. This will sound like poetic license, but really good programmers have a light that flares when they start talking about that. They immediately are back in the trenches, telling about how they couldn't figure out some nasty recursion error or some server problem that was untraceable. Usually their hands start getting animated; they loosen up. I couldn't care less about the problem OR the eventual solution. I couldn't care less if he used an open source module to write some new code last week or if he thinks CouchDB is super amazing. What I really want to know is how they think. When you have a programmer who lives in code, he (or she) swim through problems like they are physical objects. They find passion in killing bugs and making code more efficient. I can hire a hundred people who can recite what the Gang of Four is or write a bubble sort. I can't hire guys who care about whether their own code can be improved. But this is just one way. There are lots of ways to tell.
  - steve8918 15y ago
    I 100% completely agree. I think asking programming trivia questions in this day and age is ridiculous because everyone now has memorized every single CareerCup question out there. When I interviewed for previous copmanies, this is actually one of the questions I would use. "What is the hardest bug you've ever worked on?" If you take this question, and keep pressing for more and more details, and expand on that question, you can get a surprising amount of information about someone's experience and understanding. If they can't adequately answer this question, it means they haven't really been deep in code, or don't have a lot of experience, so that sets your expectations relative to the position that you're hiring for. If they claim to have a lot of experience, but can't go into too much depth, it means they're not very excited about getting their hands dirty or they don't understand what really happened, which are both bad signs to me. I don't care if they can write up a function that can spit out the Fibonacci sequence. I can that they are smart enough to figure out relatively complex issues and can communicate this intelligently.
dusklight 15y ago
Have you heard of the Dunning-Kruger effect?
- eladgil 15y ago
  Do you raise this in the context of asking people to self-asses their productivity or something else?
  - axiak 15y ago
    Probably in the context of the grandparent so assured that he knows how to assess programmers' abilities within 5 minutes.
    - silverbax88 15y ago
      Not five minutes. Two minutes. Sometimes less. And you should, too.
      - axiak 15y ago
        I've only been interviewing for a few years so I don't have a large statistical sample, but I can tell you that within a few minutes I can usually only tell if they think like me or not. The rest of the interview determines if they can code regardless of my biases, and can actually get stuff done.
        wnight 15y ago
        > within a few minutes I can usually only tell if they think like me or not. The rest of the interview determines if they can code regardless of my biases If they don't ask for relevant details, etc, they won't be coming up with a solution no matter how they do it.
- medinism 15y ago
  I have not but now that I have I will use in conversation. It will make me look smarter than I am, exacerbating my own Dunning Kruger effect
aboodman 15y ago
If you can do this reliably, you should start a business. Common knowledge is that hiring programmers is insanely difficult. If you have a gift for it, you could make craptonnes of money!
- silverbax88 15y ago
  I did start a business. But I should state that I learned how to interview and hire for years working for other people.
  - sliverstorm 15y ago
    Your parent probably means you should start a business of helping other people hire good programmers.
    - silverbax88 15y ago
      I did. It does quite well. But I am no longer involved in it day-to-day.
- kenjackson 15y ago
  Hiring people who know how to program is actually not hard. I also can screen for this in an hour, no problem. It's everything else that is the problem. Are they lazy? Do disappear for days at a time? Will they not fix bugs in someone else's code? Will they complain NIH at the drop of a hat. Will they care to write all the ugly code we need to ship with, and not just the first 80% of the project? These are the things that are hard to assess. But just knowing if they can code -- that's pretty easy.
- droithomme 15y ago
  I've never had a problem hiring great programmers, and it's not difficult at all really. It's insanely difficult though for common companies when they are not a good or interesting place to work, or they don't want to pay competitively for the people they need, or they have incorrect ideas about what sort of talent they really need. There is no shortage of great developers out there. The shortage is in companies that have management with any clue. These companies obviously are unable to understand or correct this though.
jarek 15y ago
It's kind of a two-way thing. If they're investing enough time into telling me about them, what projects they have and what's exciting about them, what the environment is like, what hardware they use, what they do after work, whether they have any employees using Dvorak †, then sure, if what I'm hearing sounds good, I can devote half a day to them. But on the other hand, I've had a company administer a test and then a coding exercise with contact via email from their secretary, and maybe I'm spoiled but I'm not doing a multi-hour task for you to earn the privilege of talking to a person involved in development. † List exaggerated
- eladgil 15y ago
  Yeah, we would put the coding exercise at the end: http://blog.eladgil.com/2011/06/our-10-step-engineering-hiring-process.html http://blog.eladgil.com/2011/06/our-10-step-engineering-hiri... It was also a burden on our team to spend all this time with a candidate, so this would often be our last step. We would next expect someone to invest in us if we did not also invest time on our end.
d3fun 15y ago
Really!! in 2 mins you would determine if someone can code or note how would you do that.
- rubashov 15y ago
  You can usually establish that someone can't code inside one minute, which happens surprisingly often. You ask: Explain virtual methods to me. If they have C or C++ on their resume you write a three line function that returns a pointer to a stack variable and ask them to explain what's wrong. If you feel like you even have to ask these questions your candidate selection process is badly broken, but the same approach scales up to all levels.
  - steve8918 15y ago
    I disagree. These days people just memorize the CareerCup e-book, and know the answers to almost every major question out there. I've had plenty of people who knew what a virtual function is, and not know how to code. People have gamed the interview system, it doesn't work anymore.
    - deleted 15y ago
      [deleted]
    - kamaal 15y ago
      I find this extremely common, especially among the job hopping crowd. Seems like the only thing they ever do keep reading interview questions all day. Do a horrible job at the current place and then hop to a new company for 30% hike 9 months later. Its sort of impossible to screen such people, because they have solutions to any interview question you can ever ask at the back of their hands.
  - deadmansshoes 15y ago
    "Explain virtual methods to me" - thereby ignoring anyone who can code in Python or Ruby, or anyone from VB.NET and several other languages where Overridable/Overrides is the terminology used.
kamaal 15y ago
Well, recently I interviewed at a place where they called me in 18 rounds of interviews over almost 3.5 full days. The only reason I was putting up with them was because of friend inside who had referred me. The thing is they made me right endless code rounds after rounds and I did code pretty well. I solved close to 90% of the problems without the internet. But most of them were coding problems. During the final rounds, I don't know what got into them. They started going into Core CS areas. And damn then they started the Algo pop quiz. Some how during the last 3-4 rounds they started giving out the perception that they were not happy with some one who did not know the algorithms from the book memorized by heart. And that even good coding skills can't compensate for that. After that they called me back and told, that some other department from the same company would like to have an another round. I just politely denied and, cut the call.
- mnutt 15y ago
  I worked in a company that did that a couple of times, but not to that extent. Usually the job description wasn't well defined and the candidate didn't have any champions within the company. We would keep on bringing the candidate in over and over in the hope that some new perspective would make the choice obvious. (it never did) Oddly enough, when you decline to continue interviewing they often immediately send you a job offer.
  - kamaal 15y ago
    Frankly speaking if you can't decide about a candidate after 3.5 days of interviewing there's probably something really badly broken about your interviewing process. Or just that its so inefficient. I wouldn't have accepted the offer from the company, from the very outlook it looked like bureaucratic jungle where no one is capable of taking a decision/risk until they run through some 20 meetings to decide what to do. And then toss it on to some one else. For that if this is what you make candidate run through, you are sure to get CareerCup interview questions ebook masters but not good programmers.
    - mnutt 15y ago
      Absolutely, any company that does this has a totally broken interviewing process. Though in our case there wasn't too much structure day-to-day, there was total lack of structure.
pavel_lishin 15y ago
> I can tell within two minutes if someone can code or not. How?
Tyrannosaurs 15y ago
I agree with this. There are real diminishing returns on technical interviews. If you've got well thought out, well structured questions that cover the key areas for your organisation, unless you're doing something massively complex or specific, you can work out whether someone has the level of competence you're looking for in a couple of hours.

eladgil 15y ago

What are the key things in your experience that let you know if someone is good or not as a long term fit with your company? Our test was not primarily just code quality. When we brought people in we would look at: 1. Approach. Did they use e.g. an open source pre-existing module or write something themselves? Did they come up with a creative solution? 2. Culture fit. While they were in working with us, how did they interact with the team? Did they enjoy working with us and vice versa? 3. Productivity. How much did they actually get done?

rubashov 15y ago
This stuff just seems easy to figure out from a resume and asking questions. If they can talk competently in detail about cool stuff on their resume, and you yourself are smart enough to spot BS, then you know they get stuff done. As for culture fit, again, a socially competent interviewer can size these things up reasonably quickly.
owenmarshall 15y ago
There's no way to figure this out from a one on one interview with that person? I mean, a half-day programming test feels excessive. Are you worried about loosing good candidates that just don't want to deal with the headaches? I've heard people try to justify insane hiring processes by saying "we want to filter out the people that don't want to work here. We only want people that really want it." But the best programmers -- the ones you want to hire -- are going to be in high demand by companies that look at their github, have a three minute interview, and pull the trigger.
- eladgil 15y ago
  We would only do the half day test with a subset of the candidates. A lot of people actually welcomed it - we would explain it was a way for them to also get to know us, see what we were like to work with etc. A lot of people about to make a long term commitment to a company also want to make sure the company is right for them. So this was a good way to get to know one another.
  - pmiller2 15y ago
    Which subset, and why only those people?
silverbax88 15y ago
One of the fastest ways is to ask about something they had to "fix" (use the word "fix") at their last job, or some problem in a pet project that they just figured out. This will sound like poetic license, but really good programmers have a light that flares when they start talking about that. They immediately are back in the trenches, telling about how they couldn't figure out some nasty recursion error or some server problem that was untraceable. Usually their hands start getting animated; they loosen up. I couldn't care less about the problem OR the eventual solution. I couldn't care less if he used an open source module to write some new code last week or if he thinks CouchDB is super amazing. What I really want to know is how they think. When you have a programmer who lives in code, he (or she) swim through problems like they are physical objects. They find passion in killing bugs and making code more efficient. I can hire a hundred people who can recite what the Gang of Four is or write a bubble sort. I can't hire guys who care about whether their own code can be improved. But this is just one way. There are lots of ways to tell.
- steve8918 15y ago
  I 100% completely agree. I think asking programming trivia questions in this day and age is ridiculous because everyone now has memorized every single CareerCup question out there. When I interviewed for previous copmanies, this is actually one of the questions I would use. "What is the hardest bug you've ever worked on?" If you take this question, and keep pressing for more and more details, and expand on that question, you can get a surprising amount of information about someone's experience and understanding. If they can't adequately answer this question, it means they haven't really been deep in code, or don't have a lot of experience, so that sets your expectations relative to the position that you're hiring for. If they claim to have a lot of experience, but can't go into too much depth, it means they're not very excited about getting their hands dirty or they don't understand what really happened, which are both bad signs to me. I don't care if they can write up a function that can spit out the Fibonacci sequence. I can that they are smart enough to figure out relatively complex issues and can communicate this intelligently.

dusklight 15y ago

Have you heard of the Dunning-Kruger effect?

eladgil 15y ago
Do you raise this in the context of asking people to self-asses their productivity or something else?
- axiak 15y ago
  Probably in the context of the grandparent so assured that he knows how to assess programmers' abilities within 5 minutes.
  - silverbax88 15y ago
    Not five minutes. Two minutes. Sometimes less. And you should, too.
    - axiak 15y ago
      I've only been interviewing for a few years so I don't have a large statistical sample, but I can tell you that within a few minutes I can usually only tell if they think like me or not. The rest of the interview determines if they can code regardless of my biases, and can actually get stuff done.
      - wnight 15y ago
        > within a few minutes I can usually only tell if they think like me or not. The rest of the interview determines if they can code regardless of my biases If they don't ask for relevant details, etc, they won't be coming up with a solution no matter how they do it.
medinism 15y ago
I have not but now that I have I will use in conversation. It will make me look smarter than I am, exacerbating my own Dunning Kruger effect

aboodman 15y ago

If you can do this reliably, you should start a business. Common knowledge is that hiring programmers is insanely difficult. If you have a gift for it, you could make craptonnes of money!

silverbax88 15y ago
I did start a business. But I should state that I learned how to interview and hire for years working for other people.
- sliverstorm 15y ago
  Your parent probably means you should start a business of helping other people hire good programmers.
  - silverbax88 15y ago
    I did. It does quite well. But I am no longer involved in it day-to-day.
kenjackson 15y ago
Hiring people who know how to program is actually not hard. I also can screen for this in an hour, no problem. It's everything else that is the problem. Are they lazy? Do disappear for days at a time? Will they not fix bugs in someone else's code? Will they complain NIH at the drop of a hat. Will they care to write all the ugly code we need to ship with, and not just the first 80% of the project? These are the things that are hard to assess. But just knowing if they can code -- that's pretty easy.
droithomme 15y ago
I've never had a problem hiring great programmers, and it's not difficult at all really. It's insanely difficult though for common companies when they are not a good or interesting place to work, or they don't want to pay competitively for the people they need, or they have incorrect ideas about what sort of talent they really need. There is no shortage of great developers out there. The shortage is in companies that have management with any clue. These companies obviously are unable to understand or correct this though.

jarek 15y ago

It's kind of a two-way thing. If they're investing enough time into telling me about them, what projects they have and what's exciting about them, what the environment is like, what hardware they use, what they do after work, whether they have any employees using Dvorak †, then sure, if what I'm hearing sounds good, I can devote half a day to them. But on the other hand, I've had a company administer a test and then a coding exercise with contact via email from their secretary, and maybe I'm spoiled but I'm not doing a multi-hour task for you to earn the privilege of talking to a person involved in development. † List exaggerated

eladgil 15y ago
Yeah, we would put the coding exercise at the end: http://blog.eladgil.com/2011/06/our-10-step-engineering-hiring-process.html http://blog.eladgil.com/2011/06/our-10-step-engineering-hiri... It was also a burden on our team to spend all this time with a candidate, so this would often be our last step. We would next expect someone to invest in us if we did not also invest time on our end.

d3fun 15y ago

Really!! in 2 mins you would determine if someone can code or note how would you do that.

rubashov 15y ago
You can usually establish that someone can't code inside one minute, which happens surprisingly often. You ask: Explain virtual methods to me. If they have C or C++ on their resume you write a three line function that returns a pointer to a stack variable and ask them to explain what's wrong. If you feel like you even have to ask these questions your candidate selection process is badly broken, but the same approach scales up to all levels.
- steve8918 15y ago
  I disagree. These days people just memorize the CareerCup e-book, and know the answers to almost every major question out there. I've had plenty of people who knew what a virtual function is, and not know how to code. People have gamed the interview system, it doesn't work anymore.
  - deleted 15y ago
    [deleted]
  - kamaal 15y ago
    I find this extremely common, especially among the job hopping crowd. Seems like the only thing they ever do keep reading interview questions all day. Do a horrible job at the current place and then hop to a new company for 30% hike 9 months later. Its sort of impossible to screen such people, because they have solutions to any interview question you can ever ask at the back of their hands.
- deadmansshoes 15y ago
  "Explain virtual methods to me" - thereby ignoring anyone who can code in Python or Ruby, or anyone from VB.NET and several other languages where Overridable/Overrides is the terminology used.

kamaal 15y ago

Well, recently I interviewed at a place where they called me in 18 rounds of interviews over almost 3.5 full days. The only reason I was putting up with them was because of friend inside who had referred me. The thing is they made me right endless code rounds after rounds and I did code pretty well. I solved close to 90% of the problems without the internet. But most of them were coding problems. During the final rounds, I don't know what got into them. They started going into Core CS areas. And damn then they started the Algo pop quiz. Some how during the last 3-4 rounds they started giving out the perception that they were not happy with some one who did not know the algorithms from the book memorized by heart. And that even good coding skills can't compensate for that. After that they called me back and told, that some other department from the same company would like to have an another round. I just politely denied and, cut the call.

mnutt 15y ago
I worked in a company that did that a couple of times, but not to that extent. Usually the job description wasn't well defined and the candidate didn't have any champions within the company. We would keep on bringing the candidate in over and over in the hope that some new perspective would make the choice obvious. (it never did) Oddly enough, when you decline to continue interviewing they often immediately send you a job offer.
- kamaal 15y ago
  Frankly speaking if you can't decide about a candidate after 3.5 days of interviewing there's probably something really badly broken about your interviewing process. Or just that its so inefficient. I wouldn't have accepted the offer from the company, from the very outlook it looked like bureaucratic jungle where no one is capable of taking a decision/risk until they run through some 20 meetings to decide what to do. And then toss it on to some one else. For that if this is what you make candidate run through, you are sure to get CareerCup interview questions ebook masters but not good programmers.
  - mnutt 15y ago
    Absolutely, any company that does this has a totally broken interviewing process. Though in our case there wasn't too much structure day-to-day, there was total lack of structure.

pavel_lishin 15y ago

> I can tell within two minutes if someone can code or not. How?

Tyrannosaurs 15y ago

I agree with this. There are real diminishing returns on technical interviews. If you've got well thought out, well structured questions that cover the key areas for your organisation, unless you're doing something massively complex or specific, you can work out whether someone has the level of competence you're looking for in a couple of hours.