4 ms·

We get our interviewees to sit in front of an IDE of their choice and write some simple code. And I mean simple! No fancy algorithms or tricks. I ask them to w

by nevster 14y ago

We get our interviewees to sit in front of an IDE of their choice and write some simple code. And I mean simple! No fancy algorithms or tricks. I ask them to write a method that takes a list of Strings and returns a single String which is all those Strings concatenated together with spaces in-between. And oh my word, how people fail. It's truly frightening.

mgkimsal 14y ago
you're looking for something like return myList.join(" ") or return implode(" ", $myList); or something similar? Or maybe even $string = ''; foreach($myList as $item) { $string .= $item." "; } return trim($string); Or something more complex? I can't really imagine it being more complex, but I can imagine people failing. I too am sometimes shocked at how people end up getting programming jobs who don't grok extremely basic concepts. I do understand how some people end up in programming roles - often just because they show some ability to get stuff done - they're the least bad in an org, and no one else wants to do it. But that's different that someone intentionally applying for a job as a "developer" and not understanding the concept of a loop (I've met a couple over the past 15+ years in the world of paid developers).
- nevster 14y ago
  Yes, something like that. We were interviewing for Java devs so basically a method with this signature: public static String concatenate(List<String> strings) And I have a StringUtils class ready to go with an example method and say "imagine there are a bunch of methods in this class and we want production ready code" There's a lot you can learn. - do they decide to write unit tests up front (note that our questionnaire asks about their testing experience and everyone rhapsodizes about their strong testing background) - if not, once they've written the method, how do they respond to the question "How do we know if it works?" (cue people writing main methods within the StringUtils class) - what kind of unit tests do they write? Can they come up with good tests that cover all cases - what names do they give to their methods/parameters/variables. (One guy called his method bangThemTogether) - what bugs do they have - what cases do they miss - if a unit test fails how do they go about fixing it (interestingly only 2 people fired up the debugger) - do they ask questions if they need clarification Minor things - keyboard/IDE skills - do they laboriously type everything or have they taken the time to learn their preferred tool - can they explain what they're doing - do they speak at all... There's a second part to the practical - a bit of a code review/refactoring exercise. Sometimes I skip it because they've taken more than 30 minutes on the first part or their just so bad it's not worth continuing. BTW we've interviewed about 30 people with this technique. They first meet with the dev manager for culture fit and general knowledge of dev processes. Then they meet with a tech lead for a more technical interview - explain your last project etc. Then I do the practical. Many people get through the first two just fine but fail badly on the practical.
  - mgkimsal 14y ago
    Good list of how you interview/test and what your expectations are! I'd have to admit, I don't often even consider writing a test for a method that would just return a join() - I trust the join() method in the underlying language 'just works', although there are other reasons for writing tests (error handling, etc). I do some PHP teaching, and one thing I got some good feedback from was giving students assignments that had asserts in them. Not quite unit-testing specifically (not using a full testing harness), but I'd give them some shell code with assert statements and they'd need to 'make it work'. The sharp ones actually made it work, and gave me feedback that they appreciated that a lot, as it showed them how to think about breaking down things in to testable parts (even trivial stuff). The not-so-sharp ones... easier to spot when they'd give me back code that obviously was never even run in the first place.
  - Androsynth 14y ago
    You are setting yourself up for failure using these methods. - You're being misleading by asking them to write a concat method when you really want unit tests for that method. Just be straightforward. You're being too clever for your own good. - People rarely code in a linear fashion. For example, I often write the meat of my methods first. Then do edge cases / error handling in a second or third pass. I have been dinged for this in interviews before. But imo, it says nothing about me as a programmer. - You want them to be comfortable with tools, but they are behind enemy lines and under enemy fire. Dont expect them to achieve any kind of comfort level (and I would put 'thinking to use debugger' up there as something you would forget/ignore while being uncomfortable). - When you are in extreme concentration mode, do you talk to others? Probably not. If you want them to talk, ask them questions. Again you're expecting them to read your mind. (also, keep them away from a keyboard if you want them to talk. Put them in front of a white board instead) As others have said, talking to people and getting to know them and how they solve problems is much better than giving them mechanical interviews.
    - nevster 14y ago
      It's really not as black and white as you seem to be imagining it. There's lots of prompting and asking questions. If they don't think of unit testing straight up, that's fine. It's just a nice indicator if they do think of it. Things generally flow as you describe - in a non-linear fashion. They'll get the basic method done and then I'll ask them about possible edge cases and error handling. The problem is that even with this prompting, many people can't come up with what those may be. And then, if I suggest what they may be (eg how should we handle nulls), how they go about handling that in their code is when it can sometimes become really messy. It's definitely not a failure. When a good programmer comes in, it's immediately obvious. They have no trouble. I mean, it really is basic stuff - writing a loop with a few if statements - not some fancy egg dropping puzzle or the big O notation stuff that Google gets you to do.
robomartin 14y ago
This is interesting. I'd like to know more. Are you targeting young inexperienced programmers? Are they just out of school? What kind of filtering do you have prior to bringing them in? I have never hired a recent grad for programming. I used to own an electronics manufacturing company near UC Riverside (California) some time back. We had pick-and-place machines and the usual assortment of equipment you'd have for the assembly and testing of circuit boards. We made microprocessor-driven industrial motor controls. During that time I thought it'd be great to hire engineering students. As someone who was a capable electronics hobbyist before I even touched college I assumed that these kids would be hell-bent to work in electronics as I was when I was in college. Not so. I probably went through 15 or 20 of them before I found a couple that were worth the effort. When it came to programming I never hired anyone with anything less than five years of demonstrable experience.
- nevster 14y ago
  We were looking for experienced Senior Java Developers. I'm talking about people who have 10 or more years of experience on their CV, working for well known companies - large banks etc. We give the CVs a fairly good going over before deciding to get someone in. They've also responded to a short questionnaire about their experience with agile methodologies. (And of course, the job agencies have supposedly already done some sort of filtering.) Interview nerves are one thing but even if you're nervous you should still be able to do the equivalent of multiplying 5 * 10.
  - robomartin 14y ago
    I am speechless. Don't know what to say.
    - nevster 14y ago
      Just for some clarification, when I say fail, I really mean fail to do it well. Most of them at least manage to get it working but don't necessarily cover all the problems that can arise and can get it to 100%, with some prompting. It's a real joy when someone comes in who obviously loves programming. Like night and day.
theyak 14y ago
Reading your description below (test cases et. al.), it doesn't seem as simple as you first make it seem. I start with something like, "Write a loop that displays the numbers 1 to 100." About 80% of the people I phone interview fail this test. Almost 100% of newly graduated students fail. I'm not sure if they are overthinking it, if schools are no longer teaching anything practical, or what.
lsiebert 14y ago
What language? Interestingly enough, you can do that with a single perl command, join. edit: nevermind answered as Java

by nevster 14y ago

mgkimsal 14y ago

you're looking for something like return myList.join(" ") or return implode(" ", $myList); or something similar? Or maybe even $string = ''; foreach($myList as $item) { $string .= $item." "; } return trim($string); Or something more complex? I can't really imagine it being more complex, but I can imagine people failing. I too am sometimes shocked at how people end up getting programming jobs who don't grok extremely basic concepts. I do understand how some people end up in programming roles - often just because they show some ability to get stuff done - they're the least bad in an org, and no one else wants to do it. But that's different that someone intentionally applying for a job as a "developer" and not understanding the concept of a loop (I've met a couple over the past 15+ years in the world of paid developers).

nevster 14y ago
Yes, something like that. We were interviewing for Java devs so basically a method with this signature: public static String concatenate(List<String> strings) And I have a StringUtils class ready to go with an example method and say "imagine there are a bunch of methods in this class and we want production ready code" There's a lot you can learn. - do they decide to write unit tests up front (note that our questionnaire asks about their testing experience and everyone rhapsodizes about their strong testing background) - if not, once they've written the method, how do they respond to the question "How do we know if it works?" (cue people writing main methods within the StringUtils class) - what kind of unit tests do they write? Can they come up with good tests that cover all cases - what names do they give to their methods/parameters/variables. (One guy called his method bangThemTogether) - what bugs do they have - what cases do they miss - if a unit test fails how do they go about fixing it (interestingly only 2 people fired up the debugger) - do they ask questions if they need clarification Minor things - keyboard/IDE skills - do they laboriously type everything or have they taken the time to learn their preferred tool - can they explain what they're doing - do they speak at all... There's a second part to the practical - a bit of a code review/refactoring exercise. Sometimes I skip it because they've taken more than 30 minutes on the first part or their just so bad it's not worth continuing. BTW we've interviewed about 30 people with this technique. They first meet with the dev manager for culture fit and general knowledge of dev processes. Then they meet with a tech lead for a more technical interview - explain your last project etc. Then I do the practical. Many people get through the first two just fine but fail badly on the practical.
- mgkimsal 14y ago
  Good list of how you interview/test and what your expectations are! I'd have to admit, I don't often even consider writing a test for a method that would just return a join() - I trust the join() method in the underlying language 'just works', although there are other reasons for writing tests (error handling, etc). I do some PHP teaching, and one thing I got some good feedback from was giving students assignments that had asserts in them. Not quite unit-testing specifically (not using a full testing harness), but I'd give them some shell code with assert statements and they'd need to 'make it work'. The sharp ones actually made it work, and gave me feedback that they appreciated that a lot, as it showed them how to think about breaking down things in to testable parts (even trivial stuff). The not-so-sharp ones... easier to spot when they'd give me back code that obviously was never even run in the first place.
- Androsynth 14y ago
  You are setting yourself up for failure using these methods. - You're being misleading by asking them to write a concat method when you really want unit tests for that method. Just be straightforward. You're being too clever for your own good. - People rarely code in a linear fashion. For example, I often write the meat of my methods first. Then do edge cases / error handling in a second or third pass. I have been dinged for this in interviews before. But imo, it says nothing about me as a programmer. - You want them to be comfortable with tools, but they are behind enemy lines and under enemy fire. Dont expect them to achieve any kind of comfort level (and I would put 'thinking to use debugger' up there as something you would forget/ignore while being uncomfortable). - When you are in extreme concentration mode, do you talk to others? Probably not. If you want them to talk, ask them questions. Again you're expecting them to read your mind. (also, keep them away from a keyboard if you want them to talk. Put them in front of a white board instead) As others have said, talking to people and getting to know them and how they solve problems is much better than giving them mechanical interviews.
  - nevster 14y ago
    It's really not as black and white as you seem to be imagining it. There's lots of prompting and asking questions. If they don't think of unit testing straight up, that's fine. It's just a nice indicator if they do think of it. Things generally flow as you describe - in a non-linear fashion. They'll get the basic method done and then I'll ask them about possible edge cases and error handling. The problem is that even with this prompting, many people can't come up with what those may be. And then, if I suggest what they may be (eg how should we handle nulls), how they go about handling that in their code is when it can sometimes become really messy. It's definitely not a failure. When a good programmer comes in, it's immediately obvious. They have no trouble. I mean, it really is basic stuff - writing a loop with a few if statements - not some fancy egg dropping puzzle or the big O notation stuff that Google gets you to do.

robomartin 14y ago

This is interesting. I'd like to know more. Are you targeting young inexperienced programmers? Are they just out of school? What kind of filtering do you have prior to bringing them in? I have never hired a recent grad for programming. I used to own an electronics manufacturing company near UC Riverside (California) some time back. We had pick-and-place machines and the usual assortment of equipment you'd have for the assembly and testing of circuit boards. We made microprocessor-driven industrial motor controls. During that time I thought it'd be great to hire engineering students. As someone who was a capable electronics hobbyist before I even touched college I assumed that these kids would be hell-bent to work in electronics as I was when I was in college. Not so. I probably went through 15 or 20 of them before I found a couple that were worth the effort. When it came to programming I never hired anyone with anything less than five years of demonstrable experience.

nevster 14y ago
We were looking for experienced Senior Java Developers. I'm talking about people who have 10 or more years of experience on their CV, working for well known companies - large banks etc. We give the CVs a fairly good going over before deciding to get someone in. They've also responded to a short questionnaire about their experience with agile methodologies. (And of course, the job agencies have supposedly already done some sort of filtering.) Interview nerves are one thing but even if you're nervous you should still be able to do the equivalent of multiplying 5 * 10.
- robomartin 14y ago
  I am speechless. Don't know what to say.
  - nevster 14y ago
    Just for some clarification, when I say fail, I really mean fail to do it well. Most of them at least manage to get it working but don't necessarily cover all the problems that can arise and can get it to 100%, with some prompting. It's a real joy when someone comes in who obviously loves programming. Like night and day.

theyak 14y ago

Reading your description below (test cases et. al.), it doesn't seem as simple as you first make it seem. I start with something like, "Write a loop that displays the numbers 1 to 100." About 80% of the people I phone interview fail this test. Almost 100% of newly graduated students fail. I'm not sure if they are overthinking it, if schools are no longer teaching anything practical, or what.

lsiebert 14y ago

What language? Interestingly enough, you can do that with a single perl command, join. edit: nevermind answered as Java