8 ms·

Ask HN: Am I being too hard on the developers I interview?

• tocomment 14y ago
  That would probably make a better homework problem. People get nervous in interviews and can't even do fizzbuzz. In general though it's better to ask people problems related to things they'll actually be working on. E.g., have them analyze a bug you recently found in your own code.
  • jlangenauer 14y ago
    You might be right, though one of the reasons I like this particular problem is that it lends itself to a recursive solution, and some of our production code is heavily recursive.
  • jwblackwell 14y ago
    I have to agree. This probably isn't all that difficult to work out when you're in the zone, home alone but in an interview is completely different. You're in a completely different state and it's hardly reflective of a normal working environment. I think if you are going to give this to people in an interview you should a) make sure you've told them to expect puzzles and b) give them a few questions, some as a warm up with plenty of time to settle in.
  • kls 14y ago
    I never make a candidate take tests. After 20 years in this industry I have come to the conclusion that they are useless. It's applied Curse of knowledge, one assumes because they can do it, all others should be able to. I have seen some of the best developers in the world that cannot do fizzbuzz in an interview situation no matter how comfortable you make them. Personally, I have a really good track record of finding and hiring talent. So much so that, friends in other companies will ask me to find them someone. Anyways, one of the most powerful things I do to find talent is ask them either send me some code ahead of time or better yet bring a laptop with a working project they have done, on it. I then ask one simple question, show me the piece of code in this project that you wrote and are most proud of. Someone that is BS'ing can't BS through that. It shows real world application, and people are generally most proud of things that they had to overcome adversity on, they are happy to talk about how they solved the problem. People try to make interviewing hard, but with that single question you will know everything you need to know about their technical skills. Personality skills is another matter, but I just take them to lunch and hang out with them to figure out if they are a fit personality wise.
    • tocomment 14y ago
      What a great philosophy! Are you hiring now??
      • kls 14y ago
        Take a look at my comment to klaut, the same applies for you, feel free to contact me, but I am not actively looking for people.
    • klaut 14y ago
      That is an excellent way of conducting an interview. Are you hiring? ;)
      • kls 14y ago
        At the moment I have no plans to expand. My current company is just a one man consultancy (service based). At some point though I will start or join another product based start-up, at which point I will be. If you would like, contact me via my info in my profile and when that time comes I will let you know.
• darkxanthos 14y ago
  This doesn't seem too hard to me but maybe that's because I'm mathy. At one of previous places my boss had found that he was able to boil down the interview to a take home problem that the candidate should just be prepared to discuss, not even solve. Literally 1 out of 10 did this. The rest gave up before hand. It's hard to find good people.
• Throwadev 14y ago
  I wouldn't expect them to write the code for it, maybe some pseudo code, but a candidate should be able to talk through the problem and come to a solution for this in 10-15 minutes. It also depends on what kind of work you expect them to do. If you are building iPhone apps or doing plain old web dev, you probably don't need candidates who can solve this in <10 minutes. I'm not surprised by your 90% no-solve stat. Candidates I interview who put Java/C# on their resumes can barely "write a function that returns true if the first letter of a string parameter is capitalized". At least half flat out can't do it.
• stevenameyer 14y ago
  I'm a firm believer in talent dilution being a huge problem, so if this is the standard that you set for full time hires then keep it. Although it may be a good idea to think about wether or not this problem is representative of someones abilities to fill the position. You don't want to fall into the trap of asking an interview question (albeit a fairly easy on in my opinion) that may not tell you how good someone would be at filling the postion your hiring for.
• hcayless 14y ago
  Are you asking them to do this in a particular language, or just to come up with an algorithm? This might work better as the sort of problem you ask an interviewee to talk you through, rather than an "I'll check on you in 10 minutes" kind of exercise. But in general, no, this kind of thing isn't out of bounds, and I'd say if 10% of your candidates are passing it, you're in the right sort of neighborhood.
  • jlangenauer 14y ago
    I tell them to do it in any language they're comfortable with, though obviously, if they're interviewing for a Ruby rule, I'd be surprised if they didn't do it in Ruby. I actually do ask them to write the code - I'd expect most programmers to know the basic syntax of the language the claim to be proficient in, but I've seen Ruby developers who are unable to correctly use Array#each, which would be a red flag.
  • jlangenauer 14y ago
    Also, they don't need to come up with an algorithm: to solve the problem, you simply need to implement exactly what's written in the problem statement - calculating each element based on the previous row, and with a couple of conditional statements (or other ways) to handle the edge cases at the, er, edge of each row. There is a bit of a conceptual leap in working out a recursive solution is possible (as is an iterative one), so I'll sometimes gently drop hints if a candidate seems to be struggling on this point.
• adcoelho 14y ago
  It took me a couple of minutes to remember the pascal triangle due to the way you are presenting the triangle. I'd rather have it structured as such: 1 1 1 1 2 1 ... It makes it easier to understand when you say 'the two numbers to the right and left on the row above'. However, the way you present it may make it easier to find a solution since it hints to storing the numbers in an array that can be used in each iteration and highlights the relations between the positions of the numbers. But i guess much of this shouldn't be a problem to experienced programmers.
  • jlangenauer 14y ago
    That's an artifact of the HN display system. When it's presented to the candidates, it's definitely correctly centered.
• bdfh42 14y ago
  I would say that for an average group of candidates the results were about right. About 10% of "programmers" can write a simple program that solves a simple problem when asked - most can't.
• jyu 14y ago
  Are other senior developer positions in your area offering £50k to £60k? The equivalent $80k to $95k seems to be the going rate for junior/mid developers in the US.
  • jlangenauer 14y ago
    We don't have a specific salary figure for the positions - those numbers are what the candidates are asking for.
  • Peroni 14y ago
    £50k - £60k would be considered senior territory in London. Most senior developers are closer to £70k and there are a smaller percentage earning closer to £100k.
  • lewispollard 14y ago
    As a 'junior/mid' dev in the UK salary range is £25-30k on average.
  • whichdan 14y ago
    Outside of SF and NYC, where are all of these $80k junior positions?
    • aoprisan 14y ago
      Boston, DC area, Atlanta, to name a few
      • whichdan 14y ago
        $80k in Boston is definitely closer to mid/senior starting salaries.
        • dragons 14y ago
          Depends on how you define mid/senior. I started at a Boston company with about 10 years of experience in '08. My "mid-level" salary was ~ $90K. I ended up around $100K after about 3 years working at the same company (my title got prepended with "Senior" at some point, but it didn't start that way). These numbers do not include bonuses, which varied.
• sarabellumac 14y ago
  The question itself is deceptive in my opinion. Pascal's Triangle is as much a graphical presentation as a mathematical one. Let's assume for a second that your candidate does not already know what Pascal's Triangle is. What exactly do you mean by the element above and to the right? In the third row second column you have the number two despite there clearly being no element on the second row to the "right". You and I know that you don't actually use the element "above and to the right" but rather, the "element directly above". You set your candidate up for failure then you are surprised that a large number of them fail? If what you seek is to determine how the candidate solves problems including getting you to clarify your question then fine, but I don't get the impression that is what you are after. You present the question as a simple problem to be solved while you sit and watch. It is simple if you know ahead of time how to represent Pascal's Triangle and what the rules "really are" from a programming perspective. If you don't you are likely screwed. I wager a guess, that the 10% that could solve your problem already knew how. Are you really looking to hire people who happen to know the answer to your slightly tricky question? You might as well ask them how many toilets there are in Boston then come back and complain about the 90% who got it wrong.
• Peroni 14y ago
  Have you set the same challenge with the same time constraints for your current team? If you have and they all pass with flying colours then don't change a thing. If you haven't, then it might be worth testing the challenge with them.
• michaelochurch 14y ago
  No, you're not being too hard on your interview candidates. That's not a hard problem, at least under basic assumptions (not concerned with performance, integer overflow, etc.) I would say you should pass on the 60% who can't solve it, unless they take the initiative and send you a solution after the interview (I've done this, when I could solve the problem but not in time). That might prevent you from passing on an otherwise good candidate who happened to choke once. The 30% in the middle you can attribute to interview stress, if other indications are that they know how to write code. The truth is, though, that a large percentage of "professional programmers" have never built anything from scratch and have been wearing training wheels the whole time. 90 percent of what professional programmers do is maintenance at ~10 LoC per day, never leaving the comfort of the IDE or building a system from the ground, and so there are a lot who've never been in a green field or had to solve a hard programming problem. You don't want to hire such people to build new stuff. You're not being "too hard" on them by filtering them out. In fact, I'm surprised that your fail rate is only 60%.
• dear 14y ago
  10% is disappointing but it also means it's a good filter, particularly when you think recursive algorithms are critical in your projects. A question with 90% passing rate is useless as a filter.
• steferson 14y ago
  Do you actually ask them to do this on a piece of paper? because I went through an interview process not too long ago, and they asked me to take a test that was actually very simple stuff, fibonacci sequence, dealing with strings, etc. And for the life of me I just couldn't do it on paper, if I was still in college I might have done it, but now that I'm a professional developer, I DON'T code on paper anymore, it feels weird and awkward. Maybe do a test, and let a few take the test on the computer, doesn't have to be in an IDE, just let them hack away at notepad, or something. I know most people believe developers should be those guys that can code their way out of any problem with just matchsticks or something, but really, would you ever encounter a situation where you need a developer to actually write code on paper? And before anyone says I'm just a bad programmer, I am usually the go to guy at my workplace, the guy that usually solves hard issues, and the more competent I become at actually useful stuff, and actual day to day programming challenges, the more I feel far from things like your test. Alternatively, you could also use a real world problem, I think there is nothing that can get a guy to prove they are a good developer than making subtle bugs in some code and asking the guy to fix them.
  • jlangenauer 14y ago
    No, it's definitely done at a computer, I know I'd find it more difficult to write code on paper too.
• hkarthik 14y ago
  Many developers who don't come from a CS or Math background will run from such questions for the following reasons: 1) They've never solved problems like this with code before. 2) They fail to grasp how such problem solving skills can judge their "real world" experience. 3) Someone across the street/city/village will pay them more than you are paying and have more softball interview questions because they are more desperate to bring in any programming talent they can find. While reasons 1 and 2 are good reasons to pass on candidates, reason 3 is unfortunately a bigger challenge to solve. It may be unpopular to say this, but I would take a hard look at the problems you need your developers to solve and come up with a more real world scenario to use for interviews and see if you get better results.
• EliRivers 14y ago
  Took a minute to code a solution using the formula for line n of Pascal's triangle, and ten minutes (my usual mix of 30% manic typing, 70% bug-fixin' and finding the typo where I wrote "naemspace" :p ) to get an iterative line-by-line working solution in C++. You say you're hiring? I'm in the UK and I'm more than a little bored where I am.
  • jlangenauer 14y ago
    We do mostly Ruby on Rails as well as C#/Unity 3D. Send me an email at jason@beyondthestory.com .
    • EliRivers 14y ago
      I tick precisely none of those boxes. Ah well, back to debugging today's C++ infinite template error :p
      • Peroni 14y ago
        Don't stick with a job that bores you. It's soul destroying. My site might have a few bits and pieces that may interest you: http://hackerjobs.co.uk/jobs http://hackerjobs.co.uk/jobs
• richeyrw 14y ago
  I don't think you're being too hard, but your instrument may be too blunt. If you basing a large part of your hiring decision off this one test then you may be eliminating people unnecessarily and occasionally including someone who doesn't belong. For example I'm pretty rusty as a programmer, having graduated to management many years ago (and a linux admin before that) and I could pass this test, but you wouldn't want me as a senior programmer. The point I'm getting at, is that like any science experiment you need to control for the variables. Just like an economics experiment might control for wealth, or education, you have to control for people who get nervous during interviews, people who have a blind spot for this particular problem, etc. An easy addition to your methodology would be to have three different problems and let them choose. I would see how that affects their performance. In my company I give potential hires a test for them to complete at home, before even interviewing them. The average time to complete the test is 6 hours, but we've never had anyone outright refuse to attempt it, and it gives us a giant steaming pile of data. So if my experience is any guide (and it might not be) I would think you could make your test more intensive, and potentially end up with more and better candidates.
  • deleted 14y ago
    [deleted]
• pandaexpress 14y ago
  I actually had a bit of trouble with this problem, it took a while to iron out the bugs. I copied down the triangle for reference but accidentally had 1 2 2 1 as the third row. Took me about 15 minutes to do. Doesn't seem like an unreasonable question to ask.
  • pandaexpress 14y ago
    The solution (in Objective-C) - (NSArray)pascal:(int)n { if (n == 1) return @[@(1)]; NSArray array = [self pascal:n-1]; NSMutableArray* mArray = [@[] mutableCopy]; for(int i = 0; i < array.count+1; i++) { int r = i - 1; if (r >= 0 && i < array.count) [mArray addObject:@([array[r] integerValue] + [array[i] integerValue])]; else if (i < array.count) [mArray addObject:@(0 + [array[i] integerValue])]; else [mArray addObject:@(0 + [array[i-1] integerValue])]; } return [mArray copy]; }
• jbrooksuk 14y ago
  Something like this should do the trick, good old coffee. pascal_r = (n) -> lines = [[1]] for i in [1...n] last = lines[i-1] lines[i] = [] for j in [0..i] lines[i][j] = (last[j-1] or 0) + (last[j] or 0) lines[n-1] console.log pascal_r(1) console.log pascal_r(4) Edit: Bloody HN formatting got the better of me again.
• Ologn 14y ago
  I have interviewed people at several companies, but at one company I went through hundreds of resumes, probably over 100 phone screens, and dozens of interviews. I noticed some patterns. One was that every technical question after the third one was usually a waste. Anyone who hit it out of the park on the first three, hit almost all of the subsequent ones out of the park. No one who struggled through the first three would suddenly start doing well on the subsequent questions. Another thing I noticed is people fell into a Gaussian distribution. Some were on the left end of it, but most of those were weeded out by the recruiters, or by resumes or finally by phone screens. Just like a bell curve, most of the people were in the middle. Of the dozens I interviewed in person, only three were on the right end of the bell curve. Which is before you even get to soft skills. Your results speak for themselves. You have found an exercise that helps mark the top 10% from the 11-40% from the 41-100%. You might want to read some blog posts like Jeff Atwood's "Why can't programmers program?"
• mtrimpe 14y ago
  Also consider that you might be below market rate and all the good guys are passing on you. The story reminds me of a blog post from a while ago [1] which concluded with an update that after raising his rates by $5 the first candidate to apply passed with flying colors. [1] http://yakovfain.com/2012/10/11/the-degradation-of-java-deve.. http://yakovfain.com/2012/10/11/the-degradation-of-java-deve....
• samwillis 14y ago
  Because I wanted distracting for 5 min, in python: def pascal(row, prev_row=None): if row == 1: return [1] else: if not prev_row: prev_row = pascal(row-1) new_row = [] for i in xrange(row): if (i == 0) or (i == row-1): new_row.append(1) else: new_row.append(prev_row[i-1]+prev_row[i]) return new_row def none_recursive_pascal(row): current_row = None for i in xrange(1,row+1): current_row = pascal(i, current_row) return current_row for i in xrange(1,10): print pascal(i) for i in xrange(1,10): print none_recursive_pascal(i) EDIT: Added a none recursive version.
  • deleted 14y ago
    [deleted]
• UnoriginalGuy 14y ago
  > Each element in Pascal’s Triangle is found by adding the two numbers to the right and left of it on the row above it; where these don’t exist, they are assumed to be zero. That gibberish. Unless you know what "Pascal’s Triangle" is prior to seeing this you have no shot of even understanding the exercise. You haven't even said how you're meant to iterate through the numbers. If this is meant to be an explanation of a "Pascal’s Triangle" it fails at that. You aren't testing programming, you're testing obscure maths knowledge and the ability to decrypt your question/explanation. I bet a lot of the other posters had to google this just to answer it in their impressively quick times.
  • victorhn 14y ago
    I remember seeing Pascal's triangle in high school algebra as a way to obtain the coefficients of a polynomial in the form (a+b)^n, i don't see how can one forget the concept, this is far from obscure maths knowledge.
  • deservingend 14y ago
    There's nothing obscure about this. This is basic secondary school math.
    • UnoriginalGuy 14y ago
      I never studied it or even heard of it. So from my perspective it is obscure. I've never needed this in my programming career up until now. Have you?
  • logn 14y ago
    I agree. "adding the two numbers to the right and left of it on the row above it" makes no sense to me whatsoever. I could interpret that in about many different ways, and during an interview I'd probably crack. To put this in perspective, I got a 1450 on the SATs. My thought process: I take a number, add the right and left on the same row, then put the result on the upper row. Or, add the left and right and current and put it on the row above. After a while I guess you mean any position is the sum of the numbers to the 1 position upper left and upper right.
• SparrowOS 14y ago
  Coeffecients of (s+1)^n Did you say you're in MATLAB? I am toying with doing a circuit simulator. I did a general purpose numeric integrator for mass-springs. Circuits are about the same. As I think about it, Laplace is not relevant because I want nonlinear circuit elements. I am curious to try solving circuits with Laplace, though. This is all for fun. God says... masses union Togo continently depraved some enough_said parity largely ignoble disgusted I'm_done Courage STRICT ----- If you do eigenvalues/vectors, it requires factoring polynomials. Factoring polynomials is a black art, I'm pretty sure. Scientific numeric computing is not for amateurs. MATLAB used a fortran LINPACK for a long time, just because that shit's hard.
• pbreit 14y ago
  The question completely confused me, probably because I've never seen it before and the triangle is apparently not printing correctly here. I don't really care for interview questions that are unlikely to show up in the real world. This feels like sort of a "gotcha" question. Perhaps you will get the best of the best in your 10%. But I suspect you are missing a lot of candidates who are very good at real-world programming.
• px1999 14y ago
  I'm working from random statistics for London that I found on the internet, so please forgive me if this doesn't apply. You seem to be paying average - average+10k (for all, not just senior) developer salaries, yet you specify senior programmer. At this level, you're only going to be attracting mid-level talent. There could a problem with your recruitment targeting, which could mean you're not getting ideal people in the door to begin with. At the mid-level, you should expect a high failure rate on simple tasks. You're dealing with people who wound up on the job market after getting a bunch of experience. This means that a lot of them were fired, came from untenable/poorly managed companies which went under/downsized, or were working under such unbearable conditions that they quit. Employed devs trade up - unless your pay packet, benefits or feel-good factor (eg if you're an NGO, charity, working on 'real' problems, etc) give your company an edge, you're not dragging people away from jobs where they're encouraged to develop their skills, work under good conditions and understand software engineering processes. Job postings regularly ask for ridiculous and irrelevent things, so a lot of the employed devs who read your job ad and apply will not meet the criteria you specify. I know it's a huge generalisation, but the best devs are usually only on the market a couple of times in their lives, and they're quickly snapped up by companies at a premium and kept happy with various incentives. Tl;dr: you should be prepared for most candidates that you interview to not be appropriate for the job you've posted. Thirdly, there's a problem with your, ahem, problem. It's not a great problem, imo because there's a fair amount of nuance to it. I'd want to shine by giving a good solution - vaguely remembering that a value in Pascal's triangle can be derived from combinations or permutations or something similar I'd spend some time trying to figure out if there's some algorithm "n!/(n+k)! - no, that's not it...", then I'd try to do it in-place calculating only half of the triangle, and then probably wind up using two rows of half length to calculate the numbers there. I'd be trying to impress you by coming up with a solution with more useful properties (memory boundedness, complexity, speed) than "returns the row of pascal's triangle". You're probably not even hiring people for their comp sci experience. Maybe you just wanted to vent, or to advertise here - that's ok, but keep in mind that you're going to have to quickly reject unsuitable candidates and quickly pick up suitable ones, that it's just a numbers game, and that this is why most companies that offer mid-range compensation are "always hiring". The problem isn't too hard as such, it's perhaps a little undirected. Also, some people just interview terribly. Dunno, just some food for thought.
• simonster 14y ago
  pascal(n) = [binomial(n, i) for i=0:n]
  • deservingend 14y ago
    lol, now define your binomial function.
    • simonster 14y ago
      binomial(n, k) = factorial(n)/(factorial(k)*factorial(n-k)) factorial(n) = n <= 1 ? 1 : n*factorial(n-1)
• FilterJoe 14y ago
  data point: I started learning Python/Django 4 months ago (after a 20 year break from programmiung). Took me 40 minutes to solve. I'm not yet fluent with syntax so I had to look up (and still managed to mangle list processing syntax and was slow to interpret error messages related to accidentally swapping my variables for size of triangle and the list. My logic was fine once I got got the syntax down and swaps corrected. So your test would have caused me to be weeded out, I guess. Had you asked me to pseudo code it I could have done it in less than 10 minutes. Note, however, that learning web development requires learning many different pieces in addition to Python, so I simply haven't had the time to thoroughly learn all the syntax of any given piece.
  • FilterJoe 14y ago
    Worried that I'm too slow, I then tracked down the reference to the fizzbuzz test: http://www.codinghorror.com/blog/2007/02/why-cant-programmers-program.html http://www.codinghorror.com/blog/2007/02/why-cant-programmer... That one took me 11 minutes. There was less syntax and more logic tested with fizzbuzz. The only thing that tripped me up for a few minutes was I found out (through debugging) that integers in Python stay integers when divided. So I had to float(i) before dividing by 3 or 5 to get it to work. I don't know if you're trying to weed out people weak on syntax but your test may very well do that. Syntax fluency generally gets better over time, I would think. Not sure basic logic and debugging skill gets better over time.
• dgunn 14y ago
  I'm not sure what your startup does, but I would bet there are better things to ask a candidate than this. You get these problems in school and become quite good at them in the moment but after doing real world work, this mindset will fade. I consider myself a pretty good coder. I've made full production apps that work just fine. But who knows how I would handle this problem if handed to me at an interview I'm already nervous about. Give this problem in the same setting to some of your currently employed programmers. If they have similar results as your candidate then your candidate filter sucks and you should re-evaluate it.
• loungin 14y ago
  It took me about 20 minutes to solve using a recursive solution error-free, and I have not been doing any math intense work lately and am well out of practice. Anyone with decent exposure to math has at least SEEN pascals triangle, so I think it is a fair question. Try as you might to make them comfortable, they are probably still nervous regardless.
• vinnybhaskar 14y ago
  Unless you are hiring a developer for a critical banking software or something similar, you should never base your interviews on Math. Rather, interviews should be based on the problem solving attitude of the candidate. I'd prefer a candidate who can come up with an innovative and cheaper solution to the problem at hand than a person who is better at Math. Though that person would also need to be good at the programming skill that he/she is being hired for.
• iamjochem 14y ago
  given that "Ruby Programmer" >= "PHP Programmer" (not necessarily true, but be honest PHP is considered pretty low on the food chain around here :)), and given that I have no CS or Math background, I whipped up an answer to this in about 5 mins: https://gist.github.com/4611869 Even taking into account the effect of 'nerves' in an interview setting, on the face of things the 10% mentioned seems a bit depressing. I do wonder how relevant or helpful this particular exercise/test is with regard to accurately gauging a candidate's potential efficacy within [a|your] development team. My instinct suggests to me that being able to write an algorithm under scrutiny and/or pressure is actually a fairly low priority skill (in most development shops) when compared to the other many other skills that are required to be a successful senior developer.
• bdevine 14y ago
  Hi, new guy here. My solution took me about 15 minutes to code in Python, but I looked up my algorithm (I am not nearly as math-oriented as I should be; programming came later in life to me). I'm genuinely curious about anyone's opinion of where I would be ranked among candidates. def gen_pascal_row(n): row = [1] for i in range(1,n+1): row.append(row[-1] * n/i) n -= 1 return row ETA: This might be better. https://gist.github.com/4612320 https://gist.github.com/4612320
• squidsoup 14y ago
  What is your business domain? You might find it more beneficial to ask your candidates to solve something relevant to the work they would be doing for your company.
• thisiswei 14y ago
  def cal(n): if n == 1: return [1] else: x = cal(n-1) return [1] + [x[i]+x[i-1] for i in range(1,len(x))] + [1] who wanna hire a junior developer? I will work for free in SF,CA