6 ms·

Here's the test I've used in the past: Before the interview, I ask them to write some code to access an HTTP endpoint that contains exchange rate data (USD, EU

Here's the test I've used in the past: Before the interview, I ask them to write some code to access an HTTP endpoint that contains exchange rate data (USD, EUR, GBP, JPY etc.) in XML and to parse and load said data into a relational database. Then to build a very simple HTML form based front-end that lets you input a currency and convert it into another currency. I ask them to send me either a link to a repository (Git, SVN etc.) or a zipball/tarball. If the job specifies a particular language, then I obviously expect it to be in that language. If not, so long as it isn't in something crazy like Brainfuck, they have free range. If the code works and is basically sane, that goes a long way to get them shortlisted. During the interview, I'll pull the code they sent up on a projector and ask them to self-review it. If they can figure out things that need improving in their code, that weighs heavily in their favour. Usually this is things like comments/documentation, tests, improving the structure or reusability. If it's really good, I'll throw a hypothetical idea for refactoring at them and see how they think. The reason this works is that, despite Hacker News/Paul Graham dogma to the contrary, "smartness" isn't the only thing that matters in programmers. It's actually fairly low down the list. When hiring programmers, I want people who are actually able to do the daily practical job of writing code, modest and self-critical enough to spot their own mistakes, and socially capable to actually communicate their decisions and mistakes to the people they work with. I interviewed a guy who was intellectually very smart and understood a lot about CS theory. I asked him why the PHP code he sent me didn't have any comments. "I don't believe in comments because they slow the PHP interpreter down." Sorry, he can be smarter than Einstein but I ain't letting him near production code.

sillysaurus2 13y ago
despite Hacker News/Paul Graham dogma to the contrary, "smartness" isn't the only thing that matters in programmers. It's actually fairly low down the list. [...] I interviewed a guy who was intellectually very smart and understood a lot about CS theory. I asked him why the PHP code he sent me didn't have any comments. "I don't believe in comments because they slow the PHP interpreter down." Sorry, he can be smarter than Einstein but I ain't letting him near production code. That isn't what Paul means by "smart." Worrying about trivial inefficiencies like that is a sign of an ineffective coder. That's the opposite of a great hacker.
- tommorris 13y ago
  So, smart includes some kind of experience-derived common sense that isn't measured well by IQ tests...? Perhaps someone should tell the people who want to use IQ tests to measure the quality of programmers...
  - jonnathanson 13y ago
    Paul Graham uses the word "smart," essentially, to mean someone who "works smart" rather than simply "working hard." In other words, it's someone who won't build a 12-foot fence if a 6-foot fence is sufficient. And it's someone who's capable of realizing as much in the first place. This isn't "common sense," per se. It's more like uncommon sense. Most people follow an effort in -> result out paradigm, believing the two are perfectly correlated. A "smart" person, in PG's view, attempts to min effort and max result. (I've also heard this sort of hacker described as "lazy," in a positive and non-derogatory sense. A "lazy" person finds the most efficient and effective ways to do things, minimizing man-hours and resources.) The relationship between "smart"/"lazy" and IQ hasn't been studied all that well, but it seems plausible that there is some sort of relationship. Perhaps it's a strong one, and perhaps it's not. Either way, I don't believe an IQ test as a single-factor qualification gate will filter effectively for "smart"/"lazy."
    - taurath 13y ago
      I use the word efficient instead of lazy. If I can do x and 1 other thing if I do x efficiently, why not strive to find as many things I can do efficiently as possible?
      - jonnathanson 13y ago
        In this case, the word "lazy" is being used as a sort of tongue-in-cheek reference for "efficient." I don't think it's meant to be interpreted as being literally lazy.
wslh 13y ago
"smartness" isn't the only thing that matters in programmers. Smartness is overrated but experience is critical for speeding up work. I can implement a whole site in Django but without good experience I will be slow (even being good at Python) Nowadays, I think we need to find new ways to evaluate candidates productivity. For example, just following a Q&A method (stackoverflow) I can be faster than other developers.
reginaldjcooper 13y ago
Sounds like a good interview; you're giving them a chance to have a go at something realistic using their typical toolset instead of a whiteboard. I don't understand your point about dogma. I think HN assumes rationality as implied by smartness. Nobody wants to hire a poor coder/communicator to write code, CS degree or no. I bet most readers would find your process to be pretty good and have similar processes of their own.
mathattack 13y ago
What I like about your method is it also allows you to catch folks who fudge the work product.
- eru 13y ago
  And let's people through who fudge the work product, but can explain it. Which is a good skill.
  - mathattack 13y ago
    I think it's only good if they fess up to it. Leveraging other people's work is fantastic, and underdeveloped in our educational system. Taking credit for other people's work, well.... :-)
eddiedunn 13y ago
Hey, that seems like an awesome hiring process. Personally, I think that if you want a brilliant mathematician/problem solver, then that's what you should put in the job listing. Then asking the kind of questions exemplified in the article is likely merited. On the other hand, if you want a good programmer you should mainly look at the quality of the code they produce.
- denzil_correa 13y ago
  I concur. Even in software engineering job positions a lot of times people advertise for X but end up asking Y and judging according to Y.
honzzz 13y ago
Depends on your definition of "smartness". In my book the ability to assess the importance of things and their relevance to given task and ability to prioritize accordingly is very important aspect of being smart. The guy who 'does not like comments because they slow the PHP interpreter down' does not seem to be that smart.
- walshemj 13y ago
  And the mundane task of coding is probably not the most important part of any developers job. Understanding fully what the spec/task requires including covering the use cases the designer forgot to mention.
TazeTSchnitzel 13y ago
> "I don't believe in comments because they slow the PHP interpreter down." For some bizzare reason, a lot of PHP devs obsess over mini benchmarks and tiny speed issues while ignoring massive latency factors (like the DB). I find it amusing, though, as PHP hasn't been a direct interpreter for years. It compiles to an internal opcode array and interprets that. While its opcode compiler is silly (it doesn't build an AST and directly emits opcodes, which causes all sorts of things not to work), it does have a proper lexer and I think comments are completely ignored aside from doc comments[1] which can be obtained in userland with reflection. [1] They look like this: /** DocComment */ /* Normal comment */
- jaredmcateer 13y ago
  > For some bizzare reason, a lot of P̶H̶P̶ devs obsess over mini benchmarks and tiny speed issues while ignoring massive latency factors (like the DB). This sort of behaviour isn't unique to PHP Devs, not even close.
  - at-fates-hands 13y ago
    True. Reminds me of all the roadies (road cyclists) who would come into our bike shop and demand the lightest components to put on their bikes, then rave about how they cut 10 GRAMS off of their bike weight. Totally ignoring the fact they're pushing almost 220lbs themselves.
    - wiredfool 13y ago
      Way back when, I did a small study (4 cyclists, 5 reps). The question: Could the rider tell which was heavier between a water bottle filled with lead shot, and one with glass beads. (2.5kg vs ~ 1kg). 1 rider managed to get more than half right. And he was a frame builder riding his own bike.
      - JanezStupar 13y ago
        Not all weight loss on a bike is bad. Sure most road bikes are pretty much light enough. Thus one should prefer an efficient bike over a light one. But there nothing wrong with having light wheels for keeping the rotational and sprung mass as low as possible on a full suspension mountain bike.
theboss 13y ago
I really like this. I'm finishing graduate school and I can't tell you how disinterested I am in a company when I apply or send an email and they want to ask me math problems or puzzles....And sadly, more ask puzzles than don't You wanted a ______ and you are interviewing for a puzzle-solving mathematician really doesn't sound right. Good stuff.
bdg 13y ago
I love your interview style: actually test for things the dev will actually do. I've applied to other jobs who's owners frequent HN, and they did things like test my ability to write sort algos or data structures... but the job called for a good knowledge of JavaScript, Backbone and general knowledge of django to build a mix between a CMS and an elaborate voting system. They didn't hire me. I'm utterly trash at working on algos under pressure. I felt like a lost child just trying to write a simple sort during the interview. Worse, I made it harder on myself by asking to not write the code in MSWord (or some other similar thing) and instead use jsfiddle. The owner looked at the code and said 'Yeah, that looks about right to me, it should work.' I foolishly hit run. What I learnt was two-fold: 1) this doesn't test my ability to do the thing you want to pay me to do (as proof, I'm writing more complex applications today at a job I enjoy far more than I would have at that company). 2) The employer can't differentiate between a "smart developer" or "average developer" until the script executes.
- lportion 13y ago
  I've had similar experiences. I went for a job a few months ago where I had to write a program on paper to produce report data from raw data collections, it was reasonably complicated. This is the type of thing I breeze through when I'm sat alone at my desk, but for whatever reason I just went into panic mode and couldn't think straight at all. I apologised and told them just that. It was frustrating to say the least. The interview went ahead and we ended up talking about various architectures, design patterns, programming languages, functional programming and even HN, but I'm sure I never got the gig because of the technical test. For the interview at my current gig they walked out of the room for 10 minutes while I had to write sorting algorithms and I was able to do it without a problem. So, for me at least, the element of being watched while I wrote the code seemed to be part of the problem.
alinajaf 13y ago
> I asked him why the PHP code he sent me didn't have any comments. "I don't believe in comments because they slow the PHP interpreter down." Sorry, he can be smarter than Einstein but I ain't letting him near production code. This one wasn't smarter than Einstein. It's been a while since I've been near a live PHP installation, but IIRC running with an opcode cache like APC probably means the comments will be parsed at most once (every server restart?)
- at-fates-hands 13y ago
  This true - but without the opcode cache the slowdown would be minimal since all the comments are ignored. Plus, the comments are there for future developers to use as a reference. Nothing worse than having something break, cracking open a file and have to spend hours having to untie someone else's code. I'd rather have copious amounts of comments, then a minimal speed gain due to not having them at all.
  - base698 13y ago
    Comments like: // Adds two numbers // method takes Object argument and returns list Are useless.
  - wpietri 13y ago
    Amen, brother. As Martin Fowler writes, "Any fool can write code that a computer can understand. Good programmers write code that humans can understand." If Mr. not-Einstein had actually measured some production issue that could be traced back to commenting, he could always have written a filter that stripped them before production. My guess, though, is even he didn't believe this excuse; it was just a glib thing to say in hopes of wiggling out of trouble for being a negligent coder.
    - tommorris 13y ago
      That's the thing: if he'd said "I was cranking this code out quickly, but, yes, it could do with some comments", I'd have been fine with that. The excuse is worse than what he's trying to excuse!
brryant 13y ago
I've also interviewed lots of intellectual "CS theory" types. PhDs, etc. In most cases, they have not worked in real software engineering environments that necessitate comments for simple HTML forms. Just IMHO, I don't think it's a huge knock, as some of these candidates can turn into game changers for your company.
ulyssesgrant 13y ago
No bullshit and to the point, this is my ideal interview
wpietri 13y ago
I'm with you on this. I always do pair programming interviews when I'm hiring. People may be able to bullshit me when I'm asking questions about their resume, but they cannot fool me, the compiler, and the unit tests when we're coding together. Smartness is great, I like smart. But I agree that it's probably the fifth- or sixth-highest-weight factor in my list.
matwood 13y ago
We interview people this way where I currently work. I'm always surprised at the people who send stuff in that flat out doesn't work. There is no time limit, and the task isn't very hard. I actually consider it a type of fizzbuzz.
vxNsr 13y ago
Are you a prof at UIC (or alumni)? I saw a job posting that basically described the first part of this, which made me smile because it actually made sense, instead of abstract nonsense about goats crossing bridges...
- tommorris 13y ago
  No, I'm a freelancer in London. The hiring process was for an organisation I volunteer for who wanted to hire a coder. I didn't want to work there, but I volunteered to help out with interviews and CV screening.
davidkassa 13y ago
This works really well for the candidate too. They walk into an interview with a known quantity and can "own" the conversation by discussing the reasons that they made the decisions that they did. It ends up being a very context-specific portfolio.

sillysaurus2 13y ago

despite Hacker News/Paul Graham dogma to the contrary, "smartness" isn't the only thing that matters in programmers. It's actually fairly low down the list. [...] I interviewed a guy who was intellectually very smart and understood a lot about CS theory. I asked him why the PHP code he sent me didn't have any comments. "I don't believe in comments because they slow the PHP interpreter down." Sorry, he can be smarter than Einstein but I ain't letting him near production code. That isn't what Paul means by "smart." Worrying about trivial inefficiencies like that is a sign of an ineffective coder. That's the opposite of a great hacker.

tommorris 13y ago
So, smart includes some kind of experience-derived common sense that isn't measured well by IQ tests...? Perhaps someone should tell the people who want to use IQ tests to measure the quality of programmers...
- jonnathanson 13y ago
  Paul Graham uses the word "smart," essentially, to mean someone who "works smart" rather than simply "working hard." In other words, it's someone who won't build a 12-foot fence if a 6-foot fence is sufficient. And it's someone who's capable of realizing as much in the first place. This isn't "common sense," per se. It's more like uncommon sense. Most people follow an effort in -> result out paradigm, believing the two are perfectly correlated. A "smart" person, in PG's view, attempts to min effort and max result. (I've also heard this sort of hacker described as "lazy," in a positive and non-derogatory sense. A "lazy" person finds the most efficient and effective ways to do things, minimizing man-hours and resources.) The relationship between "smart"/"lazy" and IQ hasn't been studied all that well, but it seems plausible that there is some sort of relationship. Perhaps it's a strong one, and perhaps it's not. Either way, I don't believe an IQ test as a single-factor qualification gate will filter effectively for "smart"/"lazy."
  - taurath 13y ago
    I use the word efficient instead of lazy. If I can do x and 1 other thing if I do x efficiently, why not strive to find as many things I can do efficiently as possible?
    - jonnathanson 13y ago
      In this case, the word "lazy" is being used as a sort of tongue-in-cheek reference for "efficient." I don't think it's meant to be interpreted as being literally lazy.

wslh 13y ago

"smartness" isn't the only thing that matters in programmers. Smartness is overrated but experience is critical for speeding up work. I can implement a whole site in Django but without good experience I will be slow (even being good at Python) Nowadays, I think we need to find new ways to evaluate candidates productivity. For example, just following a Q&A method (stackoverflow) I can be faster than other developers.

reginaldjcooper 13y ago

Sounds like a good interview; you're giving them a chance to have a go at something realistic using their typical toolset instead of a whiteboard. I don't understand your point about dogma. I think HN assumes rationality as implied by smartness. Nobody wants to hire a poor coder/communicator to write code, CS degree or no. I bet most readers would find your process to be pretty good and have similar processes of their own.

mathattack 13y ago

What I like about your method is it also allows you to catch folks who fudge the work product.

eru 13y ago
And let's people through who fudge the work product, but can explain it. Which is a good skill.
- mathattack 13y ago
  I think it's only good if they fess up to it. Leveraging other people's work is fantastic, and underdeveloped in our educational system. Taking credit for other people's work, well.... :-)

eddiedunn 13y ago

Hey, that seems like an awesome hiring process. Personally, I think that if you want a brilliant mathematician/problem solver, then that's what you should put in the job listing. Then asking the kind of questions exemplified in the article is likely merited. On the other hand, if you want a good programmer you should mainly look at the quality of the code they produce.

denzil_correa 13y ago
I concur. Even in software engineering job positions a lot of times people advertise for X but end up asking Y and judging according to Y.

honzzz 13y ago

Depends on your definition of "smartness". In my book the ability to assess the importance of things and their relevance to given task and ability to prioritize accordingly is very important aspect of being smart. The guy who 'does not like comments because they slow the PHP interpreter down' does not seem to be that smart.

walshemj 13y ago
And the mundane task of coding is probably not the most important part of any developers job. Understanding fully what the spec/task requires including covering the use cases the designer forgot to mention.

TazeTSchnitzel 13y ago

> "I don't believe in comments because they slow the PHP interpreter down." For some bizzare reason, a lot of PHP devs obsess over mini benchmarks and tiny speed issues while ignoring massive latency factors (like the DB). I find it amusing, though, as PHP hasn't been a direct interpreter for years. It compiles to an internal opcode array and interprets that. While its opcode compiler is silly (it doesn't build an AST and directly emits opcodes, which causes all sorts of things not to work), it does have a proper lexer and I think comments are completely ignored aside from doc comments[1] which can be obtained in userland with reflection. [1] They look like this: /** DocComment */ /* Normal comment */

jaredmcateer 13y ago
> For some bizzare reason, a lot of P̶H̶P̶ devs obsess over mini benchmarks and tiny speed issues while ignoring massive latency factors (like the DB). This sort of behaviour isn't unique to PHP Devs, not even close.
- at-fates-hands 13y ago
  True. Reminds me of all the roadies (road cyclists) who would come into our bike shop and demand the lightest components to put on their bikes, then rave about how they cut 10 GRAMS off of their bike weight. Totally ignoring the fact they're pushing almost 220lbs themselves.
  - wiredfool 13y ago
    Way back when, I did a small study (4 cyclists, 5 reps). The question: Could the rider tell which was heavier between a water bottle filled with lead shot, and one with glass beads. (2.5kg vs ~ 1kg). 1 rider managed to get more than half right. And he was a frame builder riding his own bike.
    - JanezStupar 13y ago
      Not all weight loss on a bike is bad. Sure most road bikes are pretty much light enough. Thus one should prefer an efficient bike over a light one. But there nothing wrong with having light wheels for keeping the rotational and sprung mass as low as possible on a full suspension mountain bike.

theboss 13y ago

I really like this. I'm finishing graduate school and I can't tell you how disinterested I am in a company when I apply or send an email and they want to ask me math problems or puzzles....And sadly, more ask puzzles than don't You wanted a ______ and you are interviewing for a puzzle-solving mathematician really doesn't sound right. Good stuff.

bdg 13y ago

I love your interview style: actually test for things the dev will actually do. I've applied to other jobs who's owners frequent HN, and they did things like test my ability to write sort algos or data structures... but the job called for a good knowledge of JavaScript, Backbone and general knowledge of django to build a mix between a CMS and an elaborate voting system. They didn't hire me. I'm utterly trash at working on algos under pressure. I felt like a lost child just trying to write a simple sort during the interview. Worse, I made it harder on myself by asking to not write the code in MSWord (or some other similar thing) and instead use jsfiddle. The owner looked at the code and said 'Yeah, that looks about right to me, it should work.' I foolishly hit run. What I learnt was two-fold: 1) this doesn't test my ability to do the thing you want to pay me to do (as proof, I'm writing more complex applications today at a job I enjoy far more than I would have at that company). 2) The employer can't differentiate between a "smart developer" or "average developer" until the script executes.

lportion 13y ago
I've had similar experiences. I went for a job a few months ago where I had to write a program on paper to produce report data from raw data collections, it was reasonably complicated. This is the type of thing I breeze through when I'm sat alone at my desk, but for whatever reason I just went into panic mode and couldn't think straight at all. I apologised and told them just that. It was frustrating to say the least. The interview went ahead and we ended up talking about various architectures, design patterns, programming languages, functional programming and even HN, but I'm sure I never got the gig because of the technical test. For the interview at my current gig they walked out of the room for 10 minutes while I had to write sorting algorithms and I was able to do it without a problem. So, for me at least, the element of being watched while I wrote the code seemed to be part of the problem.

alinajaf 13y ago

> I asked him why the PHP code he sent me didn't have any comments. "I don't believe in comments because they slow the PHP interpreter down." Sorry, he can be smarter than Einstein but I ain't letting him near production code. This one wasn't smarter than Einstein. It's been a while since I've been near a live PHP installation, but IIRC running with an opcode cache like APC probably means the comments will be parsed at most once (every server restart?)

at-fates-hands 13y ago
This true - but without the opcode cache the slowdown would be minimal since all the comments are ignored. Plus, the comments are there for future developers to use as a reference. Nothing worse than having something break, cracking open a file and have to spend hours having to untie someone else's code. I'd rather have copious amounts of comments, then a minimal speed gain due to not having them at all.
- base698 13y ago
  Comments like: // Adds two numbers // method takes Object argument and returns list Are useless.
- wpietri 13y ago
  Amen, brother. As Martin Fowler writes, "Any fool can write code that a computer can understand. Good programmers write code that humans can understand." If Mr. not-Einstein had actually measured some production issue that could be traced back to commenting, he could always have written a filter that stripped them before production. My guess, though, is even he didn't believe this excuse; it was just a glib thing to say in hopes of wiggling out of trouble for being a negligent coder.
  - tommorris 13y ago
    That's the thing: if he'd said "I was cranking this code out quickly, but, yes, it could do with some comments", I'd have been fine with that. The excuse is worse than what he's trying to excuse!

brryant 13y ago

I've also interviewed lots of intellectual "CS theory" types. PhDs, etc. In most cases, they have not worked in real software engineering environments that necessitate comments for simple HTML forms. Just IMHO, I don't think it's a huge knock, as some of these candidates can turn into game changers for your company.

ulyssesgrant 13y ago

No bullshit and to the point, this is my ideal interview

wpietri 13y ago

I'm with you on this. I always do pair programming interviews when I'm hiring. People may be able to bullshit me when I'm asking questions about their resume, but they cannot fool me, the compiler, and the unit tests when we're coding together. Smartness is great, I like smart. But I agree that it's probably the fifth- or sixth-highest-weight factor in my list.

matwood 13y ago

We interview people this way where I currently work. I'm always surprised at the people who send stuff in that flat out doesn't work. There is no time limit, and the task isn't very hard. I actually consider it a type of fizzbuzz.

vxNsr 13y ago

Are you a prof at UIC (or alumni)? I saw a job posting that basically described the first part of this, which made me smile because it actually made sense, instead of abstract nonsense about goats crossing bridges...

tommorris 13y ago
No, I'm a freelancer in London. The hiring process was for an organisation I volunteer for who wanted to hire a coder. I didn't want to work there, but I volunteered to help out with interviews and CV screening.

davidkassa 13y ago

This works really well for the candidate too. They walk into an interview with a known quantity and can "own" the conversation by discussing the reasons that they made the decisions that they did. It ends up being a very context-specific portfolio.