21 ms·

The Terrible Technical Interview

• fsk 12y ago
  False negatives are a lot worse than most interviewers think. (False negative = you pass on a good candidate, false positive = you hire someone who turned out to be unqualified) If a "good" candidate is a 1-in-100 find, then each false negative means you have to look at another 100 candidates. Also, if you decrease your false negative rate by more than you decrease your false positive rate, you're actually hiring MORE bad candidates (while spending more time interviewing). I.e., every time you pass on a good candidate, that gives you another chance to make a mistake and hire someone bad. Your odds of hiring one specific "bad" candidate may be small, but if most candidates are "bad", that actually makes it more likely to hire someone bad each time you pass on someone good.
  • geebee 12y ago
    Arg! I'm on my phone and I meant to upvote! Apologies, this was a very insightful comment. I had never thought of it this way before but this is an instance of Bayes's rule. If the false negative rate goes too high and the percentage of good programmers is small then yes, the process could actually increase the odds of a bad hire.
    • fsk 12y ago
      Actually, it happens as long as your "stricter hiring practices" increase your false negative percentage by a lot more than they decrease your false positive percentage. Try it out with some numbers. 10100 candidates, 100 are "good". Suppose you have 2% false positives and 1% false negatives. You hire 99 good candidates and 200 bad candidates. Suppose now you have 0.5% false positives and 90% false negatives. (You decreased your false positive rate by 4x but increased your false negative rate by 90x. This is typical for employers who look for every little excuse to reject someone.) You hire 10 good candidates and 50 bad candidates. Your "good hire" percentage went down, and you're churning through a lot more candidates to meet your hiring quota! So, "it is better to pass on a good candidate than hire a bad candidate" is FALSE if you wind up being too picky on passing on good candidates. Assuming you can identify losers and fire them after a year or two (with decent severance to be fair), you're actually better off hiring more leniently. It's also even worse when you realize that the candidate pool is more like: 10200 candidates, 100 are "good", 100 are "toxic", and the toxic people excel at pretending to be "good". Also, the rules for hiring are different for a famous employer and a no-name employer. Google and Facebook are going to have everyone competent applying. If you're a no-name startup, you'll be lucky to have 1 or 2 highly skilled candidates in your hiring pool.
      • fsk 12y ago
        Also, what makes this mistake common is the feedback you get. When you make a false negative, you never find out that you passed on someone amazing. When you make a false positive, it's professional embarrassment for the boss when he's forced to admit he made a mistake and fire them. So the incentive for the boss is to minimize false positives, even at the expense of too many false negatives. The boss is looking out for his personal interests, and not what's best for the business.
        • din-9 12y ago
          Reasons for not wanting to fire people also include: basic humanity.
          • fsk 12y ago
            But it's inhumane to your other employees to keep someone unqualified around.
      • frostmatthew 12y ago
        > Try it out with some numbers. 10100 candidates, 100 are "good". What you're attempting to do works well for hypothetical drug testing[1] or terrorists but not for hiring developers (or anyone else). With the numbers you used you're proposing that less than 1% of all candidates are "good" - nobody would reasonably set the "good" threshold to include only the top 1% of developers. [1] https://en.wikipedia.org/wiki/Bayes%27_theorem#Drug_testing https://en.wikipedia.org/wiki/Bayes%27_theorem#Drug_testing
        • fsk 12y ago
          Even if you set the "good" percentage to 10%, too high a false positive rate will still ruin your results. Based on the people I've worked with over the years, I say that the actual skill distribution is: 5% toxic - These are the people who will ruin your business while deflecting blame to other people. 25% subtractors - These are the people who need more attention and help than the amount of work they get done. In the right environment, they can be useful. (Also, this is mostly independent of experience level. I know some really experienced people who were subtractors.) 60% average - These people are competent but not brilliant. These are solid performers. 9% above average - They can get 2x-5x the work done of someone average. 1% brilliant - These are the mythical 10x-100x programmers. These are the people who can write your MVP by themselves in 1-3 months and it'll be amazing. You first have to decide if you're targeting brilliant, above average, or average. For most businesses, average is good enough. If you incorrectly weed out the rare brilliant person, you might wind up instead with someone average, above average, or (even worse) toxic. Actually, when my employer was interviewing, I was surprised that the candidates were so strong. There was one brilliant guy and one above-average guy (My coworkers didn't like them; they failed the technical screening, which makes me distrust technical screening even more now). They wound up hiring one of the weakest candidates, a subtractor, and having worked with him for a couple of months my analysis of him hasn't changed.
  • sytelus 12y ago
    No. Many companies have hiring rate of 10% [1] which means hiring 1 candidate may cost you 10 person-days assuming full day interviews for each candidate [2]. You can probably double that to cover the cost of phone screens, so let's say 20-person day of work per good hire. Assume you had 4 false negatives just because you are so aggressive and so a good hire ended up costing you 100 person days instead. This cost is still tiny compared to damages that a true negative would cause otherwise, i.e., (1) loss of entire person year or even two because annual reviews need to accumulate evidence for HR to fire (may be less time if you were startup without real "HR") (2) amount of cleanup other people have to do after that new bad hire (3) loss in moral for good people in your team who now perceives your hiring process at the company as "broken" (4) delays + bugs introduced in product because actual work probably didn't got done or badly done despite of you filling up your headcount (5) amount of money lost in salaries, signing bonuses, office space and benefits (typically > $200K) (6) amount of productivity lost because of wasted time by good people in the team trying to "ramp up" your true negative (7) emotional stress you caused to good people wondering them about their job stability and to managers who wasted their time in months of paper work and lot of explaining (8) emotional stress you caused to your true negative being fired who had moved across the country for you, bought a house on mortgage and had 3 school going children (9) Most likely, if you are big company, true negative didn't actually got fired because hiring manager never wanted to admit it. S/he was encouraged to join another team or role or even learned political tricks to get promoted contributing to ongoing bozo explosion[3] (10) I could go on and easily justify probably 3X-10X loss compared the case if true negative was avoided Footnotes: 1 - Many companies specify their hiring rate over total resume they received which is wrong. I'll use 10% as total number of full interviews that needed to be conducted which is average of 5%-15% at most companies. 2 - This is bad math. Assuming random trials, it would be actually 5 person-day on average but intuitive approach doesn't produce entirely bad results here so we will go with that. 3 - http://guykawasaki.com/how_to_prevent_/ http://guykawasaki.com/how_to_prevent_/
    • fsk 12y ago
      Hiring rate of 10%? You mean you hire 10% of all people who submit a resume? Or 10% of all people who come in for an on-site interview? Those are two different things. Why should it take 2 years to fire someone? That sounds like a corporate bureaucracy problem.
• nperez 12y ago
  If your only goal is to measure the ability to code, then doing a whiteboard interview isn't the best way to do that IMO. But what about positions where you will be expected to give presentations, do pair-programming, or mentor junior developers? I think whiteboard interviews can be a measure of your ability to take technical concepts and illustrate/explain them clearly to a team. I used to really hate whiteboards, but as I grew into positions that required more leadership, I realized that I personally needed to develop better presentation skills. If a potential employer were to test me on that, a whiteboard test wouldn't seem unreasonable to me. So I have mixed feelings on this. Not every hire needs to be capable of being a teacher or delivering a solid keynote speech.. some positions would be best filled by someone with those skills.
  • RogerL 12y ago
    If you want to measure giving a public presentation....have them give a public presentation. If you want to measure their ability to pair program ... pair program with them. If you want to measure their ability to mentor ... have them teach you something they know. I want to know how you do X, and measuring X is easy, so please do Y as a proxy is never a good approach. I have never, in my 25+ year career, ever had to whiteboard a problem, where the other person knows the answer (that is laughable - I've given correct answers and been told they are wrong), I don't, and there are hundreds of thousands of dollars at stake. It just has nothing to do with the job or on the job performance.
    • steveax 12y ago
      If you're whiteboarding with a simple pass/fail mentality, I'd say you're doing it wrong. A good whiteboarding exercise lets you see how a candidate explores a problem. Do they ask insightful questions? How do they break a problem down? Do they bluff when faced with something they don't know? These are behavioral attributes that are important - much more important than a simple binary test.
    • imsofuture 12y ago
      You're right, I've never whiteboarded a problem where the other person knows the answer. A thousand times have I sat down in front of a whiteboard with coworkers looking to solve a problem. That's how I interview too.
• ditonal 12y ago
  In my experience, a lot of companies are combining _all_ these things. So you're expected to do a phone interview, a test project/coding test, a whiteboard test, and the management brown-nosing at the end of it where you get to pitch that you've studied a hard technical skill your whole career just because you're so passionate about getting woken up at 3AM on PagerDuty to build _their_ vision, and definitely not because you expect to be paid for your time and skill. The fact that you realize money exists and can be used to pay for things like housing, education, and healthcare means you're not truly passionate about technology. The worst is the NYC tech scene where they have the exact same standards (which they cargo-cult from the west coast), but they decided not to bring over that aspect where engineers are respected and valued. Instead they borrowed the west coast interview and combined it with the east-coast finance style where programmers are considered clerical workers and cost centers. NYC is actually a fun city to interview in because since it combines so many different cultures, you can't even study for an interview because 5 different companies will give you 5 different interviews. Finance companies still love brainteasers and they _love_ mutexes, seriously, if you're interviewing at a finance shop just memorize Java Concurrency in Practice and the producer/consumer wikipedia article, because they are reading questions from there, even though when you show up on your first day you'll have been better off having read Spring in Action and Headfirst Enterprisey Design Patterns or whatever. In general I love how these companies have elite hiring standards but incredibly mediocre interviews. You get asked the same questions over and over again by these companies. Find the biggest sum in a list, find two items in a list that sum to a given number, sort a list of integers/strings but keep integers where integers were and strings where strings were, copy files with ids to all servers with ids, etc. What's the difference between an abstract base class and an interface (by a Python/Node shop), what is a closure, etc etc. I've heard so many of the same questions repeated over and over. They have "high bars" for their candidates but apparently their interviewers can just use the first link off of Google or re-use whatever Facebook was asking in 2010 when they got rejected by them. And then at the end we have a "tech talent shortage". Whatever happened to that part where we claimed a false negative wasn't a big deal? (glad the article calls this attitude out). Tech hiring is totally broken and then they claim there's a shortage. There definitely is a shortage of qualified interviewers, not so much a shortage of qualified candidates. I have a friend who I mentored into the industry, she's very smart but absolutely a junior engineer, and yet she starts a new job and was doing interviews with _no_ training 3 months later. They just threw her into the lion's den and expected her to figure it out. Coding tests are another great one because the exact same people who talk about the importance and value of data throw all of that out the window when it comes to evaluating them. There is no calibration or standards, generally. One person is offended by a hardcoded file path but doesn't care about whether you have tests, and another person is the direct opposite. Many people expect you to write extensive optimizations for the 100Kb input file you were given, another person sees that as absurd premature optimization. Whether you make it through a code screen is entirely tied to whether your coding style happens to jibe with whoever is reviewing you. Again, the West Coast is better because at least compensation packages reflect the hoops you have to jump through and the monkey dances you have to have learned on cue. On the East Coast, anybody paying attention is desperately trying to get into management by the time they're 30 because to do otherwise is to be humiliated and infantalized during the interview process _and_ during your tenure working there. My simple solution to all this nonsense is to make sure your CTO and head tech managers are being made to jump through all the same hoops with all the same standards. Drop this whole "Oh, the CTO is a _manager_ role, he shouldn't have to worry about all that." (again, more of an East Coast attitude). Another incorrect thing tech companies claim that benefits them at the expense of labor that engineers brainlessly parrot: "false positive are expensive because firing is hard". No, it isn't, I've seen many people fired very easily the second they're not up to standards. At worst they're instantly let go because it's at-will employment. At best, their given some absurd Performance Improvement Plan that establishes a paper trail so they can fire them without severance. And if your employer tries to tell you people have come off of those things, I have news for you, people lie, and liars are good at reaching senior management positions. Something else is how you can put in hours and hours of your life, and get _no_ feedback, because telling you how you scored on an interview might expose them to legal liability. Let's be real, the legal liability is when they reject qualified people for things like "culture fit", and if your interview process exposes you to legal liability, maybe it's because it's illegal and unethical. Another thing tech companies should consider is, instead of paying insane money to recruiters to be pushy sales people who are trying to dupe engineers into their low-paying positions, maybe just redirect that money to the engineer instead, it's 2015, the days that sleazy recruiter types are necessary to try to fast-talk an engineer into a position that's not good for him are over because we have the internet and we can read your terrible Glassdoor reviews. I'm convinced one huge reason all this happens is to discourage job-hopping, because in this market, liquidity would probably help salaries move up faster. I know I sound bitter, but again, these are the exact same companies running to the taxpayers to spend hundreds of millions of dollars of middle class Americans money to pay to solve their tech shortage crisis. Yet they absolutely refuse to evaluate their own hiring processes. And a huge chunk of engineers just eat up all this dogma about how hiring is hard and these processes are necessary, and don't think for one second that all these processes are totally designed to please the employer at the expense of engineers being treated well. In a few years, there will be another recession and we're all going to be "rightsized" away, so, demand to get treated well while you still can.
• StevePerkins 12y ago
  I do almost all of the technical phone screens for my company. The process starts with a "where-do-you-see-yourself-in-5-years" personality screen, with an H.R. drone or outside recruiter. It moves on me, and then on to a brief "homework" coding exercise that we ask people to write and submit. If we like their code sample, then they come in for the panel of "whiteboard-exercise" people who conduct the face to face round. At my stage, I seldom bother with certification style questions about the programming language, or with highly abstract thought exercises, because other people will do those things in the face to face stage. Typically, I ask questions that give me an idea about the candidate's depth of experience and awareness. For example, "I see that you've spent X years using Subversion for source control. What are your opinions on trunk-first development vs. branch-first development?" I ask questions that speak to practical experience and design ability, without getting into too much depth. For example, "Pretend that you're using an OO language to build an application for <insert purpose here>. Just off the top of your head, what are some classes that you would expect to see in your class diagram?". Etc. I find that this level of questioning is a much better screening tool than trick questions about programming language quirks or the minutiae of frameworks, or cliche puzzles about how many golf balls fit on an airplane. However, I groan and roll my eyes when I hear people challenge the need for technical interviews at all. Yes, they are necessary. Having performed a thousand interviews by now, I am awestruck by how poor the software development talent pool is. I am aware that the Bay Area is overrepresented in HN's readership, and that crowd tends to take for granted the talent level found in the technical equivalent of Mecca. However, I assure you that the rest of the planet is dominated by sleepy line-of-business developers... who have all the passion beaten out of them in the first 5-10 years, and spend the rest of their career just phoning it in and not growing. I sometimes ask candidates what the letters "MVC" stand for. The successful response rate is around 50/50. I ask candidates to briefly explain the advantages of the Model View Controller pattern, and only 10-20% can field the question. We bring in Java and C# candidates who have been working with their respective language for 10 or 15 years, and they get COMPLETELY EXPOSED during the face to face round when asked a series of basic certification exam style questions. Nothing tricky, just core fundamentals. People who post here are not the norm. The "norm" is atrocious. So yes, unfortunately we all must endure technical interviews... to filter out people who have enough confidence or personality to excel in the other interview segments, yet are utterly useless. Let's put it this way: the more experience I've had with interviewing, the more selective I've been in my own job searching, and the more aggressive I've been in my salary negotiations. If you are really good, and are located outside of San Francisco, then you are worth your weight in gold and should value yourself accordingly. You wield tremendous leverage once you make a strong showing in a technical interview.
• realrocker 12y ago
  I bet I can put together dozens of people who think OOP and MVC are bad programming paradigms, another dozen who can make war on the quantum of documentation. We already know the solution to this problem: We need to hire people who have the skills to hire people and willinglines to do it too. Just being a senior doesn't cut it. Too costly I guess but is it costlier than making bad hires? IMO successful startups are the one's where founders had the skill to identify and hire the right talent. What I don't understand is why big companies are not creating a separate division of engineers with primary task of hiring? Is it because that if such a division existed the people there would eventually lose the skills being cut off from mainstream engineering tasks? I am not sure. But it seems to me that this can be solved by looking at hiring task as an engineering task. What's wrong in seeing hiring engineers as another piece of code which needs to be carefully thought and polished?
• golergka 12y ago
  I'm going to go against the popular opinion here: I don't believe that technical interviews are broken and need fixing. Of course, we know as engineers who deployed code filled with hacks and known bugs to production, everything is broken. The difference is only how broken it is. Let's fix our approach to quality and security, ability to estimate and other things that are much more broken first.
  • frostmatthew 12y ago
    > I don't believe that technical interviews are broken and need fixing I think one of the reasons so many developers feel the process is "broken" is because it's all they know (i.e. most developers didn't have a previous profession). For most jobs it's not practical or possible to get any insight into how a person will do that job prior to hiring them. Whiteboarding isn't intended to perfectly mirror "real life coding" - it's intended to give some insight into one's ability to write software to solve a problem. It's not perfect (and it can certainly be done extremely poorly!), but it shouldn't be dismissed as broken or useless any more than one should suggest actors shouldn't have to audition for roles.
• pandaman 12y ago
  Almost everywhere I went on interviews the questions were about the very basic things. The things you'd knew if you did what you claimed you did on your resume. E.g. if you are a 3d graphics programmer with 10 years of experience you know what cross product is. If you had been shipping multiple 1M+ LOC projects in C++ you know what the word "virtual" mean and other popular syntax as well. The places that asked me things I did not know probably had been looking for somebody with different experience than I had so it's fine with me too, even though it would be better if they evaluated my experience from my resume. So I don't see the author's problem. If the questions on the interview are too hard - you are probably applying at above your level or in the wrong field. I will most definitely not spend my time working on some programming test or do some side project for "show-and-tell" to please you. Heck, he is appealing to other professions allegedly not doing interviews, yet it's even harder to imagine them doing things he suggests. What kind of side project a doctor will have? A civil engineer? A manager? A pilot? A chef? A lawyer?
• jaredmcdonald 12y ago
  > It is time for engineers–especially excellent engineers for whom demand is high–to start to flatly refuse to do whiteboard interviews. This might be a viable strategy for people who have a well-established career/credentials/references (etc), but for junior-level candidates still trying to prove themselves (such as myself) I can't see this working out too well.
  • tonyarkles 12y ago
    That's one of the harder parts of the process, and it's partially because of poor expectations on the part of those doing hiring. To me, it's absolutely ridiculous to expect to hire a junior developer and have them come with a fully-developed set of skills. If they do, that's great. But if you're hiring someone fresh out of school, you've got to be approaching it as hiring someone that you're going to train and mentor. For me, the number one thing I look for in a junior developer is the ability to learn. Here's an example from early in my own career. I was just finishing up my 3rd year of a combined CS/EE program, and looking for a summer job. I got in touch with a biology lab that needed a developer for the summer to build some (very cool) software to support the neurophysiology experiments they were doing. I looked at the job requirements and thought "well, I don't know most of this stuff, but I'm sure I could pick it up." The interview progressed like this: > Do you know Python? I'd heard about it, but have no real experience with it. I downloaded it last week and started playing with it though, and it doesn't seem too different than other languages I've used. > How about VisionEgg (neurophysiology module for Python)? Well, I downloaded it at the same time I downloaded Python. I've managed to get a window to open up, and I'm displaying a square that's got a cool animated habituation pattern on it. (Note: 1 week prior, I had no idea what a habituation pattern was) I do have a bit of OpenGL experience from a class I took, and that's the underlying library that VisionEgg uses. > Well, so far, you're the only applicant who has actually made an effort to look at the specific tools we're using here. I've got one other applicant coming this afternoon, but unless they somehow have more experience with these tools than you do, the job is yours. It turned out to be a great experience, and they hired me back on the following summer. I went from being a total Python noob to contributing patches back into VisionEgg. I think most junior positions should probably progress like that. Give me a keen junior developer, and let me shape and mould them into a not-junior developer. The flip side to this: if you're expecting the person to be productive on day 1 or 2, you'd better be hiring someone with experience. Whether or not they have code they can show you, they should be more than capable of going into serious detail about past projects they've worked on (within the confines of NDAs and such, of course).
• foz 12y ago
  In my organization, we start by looking at what the candidate has actually done. We ask for a github profile, and any side projects they might have worked on. If the CV and body of work look interesting, we go on to a phone screen - general questions, clarifying points on the CV, explaining the job, answering questions. For the on-site interview, we ask candidates to bring their laptop. We advise them to use a typical, comfortable development environment. We've seen candidates struggle with the latest Ubuntu, installed that morning to impress us. So we don't want that. During the hands-on interviews, we ask to see a side project, or any code they are mostly responsible for, and familiar with. We ask them to explain code, maybe change something, refactor a test, etc. What we don't tell candidates is that this part of the interview is also about how they use their tools. It's important to see how good they are with their editor, command-line tools, can they type well, do they get easily distracted, and so on. One of the most effective interviews we do is for project planning. A problem is explained in detail and the candidate is asked to design a solution. Not in code, but to talk it through in detail, drawing or writing docs/stories if needed. This phase helps show us how they break down a project, ask questions, negotiate features, and look for opportunities to reduce complexity. Bonus points for making a pen-and-paper wireframe or throwaway prototype. What we refuse to do is the "puzzle" problems, whiteboard code (which makes no sense), or tricky technical questions. We instead want to find people that use best practices, don't re-invent the wheel, tackle problems pragmatically, and are good with their tools. Over time, this approach seems to work well. However, we also discovered that we have to re-train and test our own interviewers. Without that step, the process can change unexpectedly, become inconsistent, or unfair. Don't just assume your staff is interviewing well - take time to check it out and help them get better.
  • kelukelugames 12y ago
    Wow, this is my worst nightmare. I freeze anytime a co worker is near my desk because I worry they will judge how I type and get around the IDE. Of course that just makes the vicious cycle worse.
    • rasur 12y ago
      LOL, been there, done that, and I work at the same place as foz :)
• msoad 12y ago
  I live in Silicon Valley so I have tons of Software Engineer friends. I also have a friend who's a heart surgery researcher at Stanford. When we talk about our hiring process he just laughs and thinks its impossible to do something remotely close what we do in Software in their field. It seems Software is the only field that people have to do some serious work for the interviews.
  • bshimmin 12y ago
    I guess you find out pretty quickly whether someone's a competent heart surgeon.
  • swatow 12y ago
    Maybe they should. Studies have found that ratings of a surgeon skill based on a video recordings of surgery, are predictive of patient outcomes[0]. The problem with associating prestige and respect with not stooping to being tested, is that testing can be useful and necessary. http://psnet.ahrq.gov/resource.aspx?resourceID=26950 http://psnet.ahrq.gov/resource.aspx?resourceID=26950
• kelukelugames 12y ago
  Maybe I'd have sideprojects if I didn't use all of my spare times studying for interviews.
• TACIXAT 12y ago
  I get to interview candidate team members for the malware research team I'm on. We write a lot of code, but we're not really software engineers. Accordingly, I only ask people very basic things in an interview to see if they can use the tools they list on their resume. Often times it's something that can be handled with a single list comprehension (though no one every does that). It is astounding how few people can do what their resumes say. I'm sure it's stressful if you don't know what you say you do. Then again, if you list 6 programming languages on your resume and can't xor decode a string in the one I let you choose from that list, you deserve to sweat a little.
• tezka 12y ago
  I agree that technical interviews are broken, but not in the way that the author describes. I think the interviews should retain the algorithmic questions, but do away with language gotchas and design patterns etc. More emphasis should be added on the following topics: distributed systems issues, traits of modern hardware (cache efficiency, concurrency issues). There needs to be a part about mathematical competency introduced as well. Knowledge of basic probability, statistics and linear algebra should become part and parcel of the engineering interview process.
• cpprototypes 12y ago
  I think this would be a fair and effective interview system: 1) College graduate: Keep the traditional algorithm programming tech interview. But allow the use of a laptop (no whiteboard) 2) 2 - 7 years engineer: Require a github side project or give a home coding test. Interview will focus on discussing the project implementation. Also ask them to add a simple feature. 3) 7 - 15 years engineer: Ask them to come to the office and show them some bad code from the company source control. Ask them to explain why it's bad and to refactor it to something better. 4) Express route: Instant offer, no interview. Only "interview" is by a VP and it's more about trying to convince the candidate to join the company instead of the other way around. The first 3 scale according to typical life situation and industry experience. The amount of free tend tends to decrease from college graduation to 30's and beyond. For example, a college graduate has lots of free time. Someone in their 20's has less, but still a significant amount. 30's and beyond tend to have little free time due to family responsibilities. But the interview styles also match what they should know by that point. A college graduate doesn't know anything about real industry coding so the typical algorithm coding interview is ok. Someone with 2 - 7 years experience should be good at writing lots of code. But they don't yet have the experience to know that sometimes deleting code is better than adding more. They also don't have enough experience yet to read code well and refactor, their "code smell" sense is not yet developed and they think adding more code is the solution to everything. An industry veteran of 7 - 15 years should be able to read code well, spot all the issues, and be able to refactor into something better. These skills can only be gained after years of experience. So the 3 interview styles scale well according to the free time they are likely to have and testing whether they've really grown as an engineer. The last one, the "express route" is reserved only by referral from the company's best engineers. For example, if the company has this tech ladder: junior engineer, engineer I, engineer II, senior engineer, senior engineer II, principal engineer: Then only senior engineer II or above AND being at the company for at least 3 years would be allowed to make ONE express route referral per year. The reason is that an engineer has to be in the industry for a while to meet other good engineers. And 3 years is enough to learn the company culture. This express route system effectively would give a competitive advantage over other companies. For example, imagine a very good engineer who is already employed. They are busy and don't want to go through any kind of interview process. But if they are given an instant offer, that greatly reduces the barriers for them and they are much more likely to seriously consider switching to the company. The main idea here is that good engineers have a lot of options but not much free time to interview, your good engineers should know a few in other companies, so the express route is a way to get more good engineers who would usually not consider moving because they don't want to spend time interviewing.
• ffran 12y ago
  I usually countered FizzBuzz question with an offer to show of the source code of a small project I wrote while studying. While some didn't care, most interviewers were actually happy about it and let me do it. I tell them about the problems I had and how I approached them, usually leading to a question from me on how the company had similar problems and how they solve them. In my experience this "opens up" the interviewer(s) to talk a bit more honestly about development practices in the company. On test projects. I generally expect that companies (after a pre screening) put similar efforts in me as I put in them. If you let me do a two day project, I'd expect that one person (preferable my future boss or a future colleague) to show me a "company fitting" solution and take the time to discus my work and theirs. If you're not willing to spend that time I'll most likely think you don't respect my time. Which is a factor in choosing a company. I understand you are busy, I hope you understand that I’m too. Edit: not a native speaker, sorry for bad english
• leadgenguy 12y ago
  I am a self taught PHP programmer. Been working in tech as a freelancer since early 2000s. Built real applications. I thought I was pretty good so a few years ago I moved to Silicon Valley to get a startup job. Boy was I wrong. Even though I had created real applications and knew MVC, etc. etc. I was basically told I was worthless because I didn't know algorithms, unit testing, etc. etc. So I moved back home (Tel Aviv) and started my own company (lead gen market). Programmed everything myself. Last year I did over $2 mil (70% margins) and this year I am on track for $3.5-$4 mil in revs. The technical interview was the best thing ever for me because if I had passed I wouldn't be where I am today.
  • bhayden 12y ago
    I really like seeing this. I also write PHP for my projects, and I know it's terrible. I have no delusions about this. I use mysql_query despite knowing it's deprecated and possibly insecure, and if I'm building something that doesn't store any sensitive data and the worst case scenario of someone finding an exploit is that I have to restore from a snapshot, I really just don't care. I focus on building the product quickly and efficiently. Let the programmers of the world thumb their noses at me, at least I'm building something of value for me instead of some company.
    • mahyarm 12y ago
      The thing is when the company gets big and its not just you, your going to wish your original language was something more scalable. For example, you tube engineers probably curse that the thing was made in python, because in a 1000 engineer org, making changes can break things you wouldn't be aware of elsewhere. While something more statically typed like java or go would break on the compile step instead of the run time step. Python was fine when the project was 1 - 20 engineers, but it became a liability later on. That is why people want you to spend a week or two to learn something better and more maintainable, so you wont curse your future self later. But if your making small build utility type things, then it's fine. Contact websites for small firms. Or if it's for your quick test project, etc.
  • MichaelGG 12y ago
• tptacek 12y ago
  Requiring a side project from candidates would have cost us most of our best hires.
  • buchweizen2 12y ago
    I was a bit surprised that the article first talks about how bad it is to pass on good candidates that don't interview well on whiteboards and then suggests to pass on candidates that don't have side projects. Are all candidates without side projects not good?
    • mahyarm 12y ago
      Maybe we should just offer a choice of what kind of interview candidates want.
    • army 12y ago
      My bias is better than your bias ;)
  • facepalm 12y ago
    But I think you advocate doing a project with the candidate, which is also very time consuming. Doesn't that also filter out some good candidates who simply don't have the time? I assume you pay them, but still. After a couple of job interviews in recent times my personal inclination to invest a lot of time has gone down by a huge amount. For my first application I even took the time to contribute to one of their open source projects (as they asked on their job application page). Didn't get a job - how many times am I supposed to invest that much time for a job application? Is determining technical skill even the biggest problem? I think my GitHub account shows I can code, even if it mostly contains small projects - but vastly more complex than FizzBuzz. I'm not even afraid of whiteboard interviews (if I had a dollar for every time I was asked to implement Quicksort or compute Fibonacci numbers recursively, I'd probably have ten dollars by now). Yet I don't get hired. So my conclusion is that there really isn't that much of a talent shortage. Not enough to let companies look beyond my age or my lack of passion, anyway (my answer to the trunk vs branch first development question would be "I don't care much", although I could probably blab about presumed pro and contra arguments). Edit: just looked up Starfighter again. In a way I am excited, as I have recently decided that only online games that have an API really interest me. However, it sounds like it would require a huge time commitment, too. Wouldn't the time be better invested in side projects for GitHub?
• yason 12y ago
  The more I interview the less I tend to give weight to the technical part and the more I focus on evaluating how well I get along with the person, how easily and naturally we can discuss programming topics, and do we happen to converge on building a rapport. People can learn technical stuff and people can grow professionally but the chemistry is harder to fix. And, unlike some programmers might see it, even code is communication and, a bit surprisingly, is subject to chemistry. You can talk to some people via code. Not all people. I consider the technical part merely as a way to filter out people who seem to be missing something blatantly obvious. I'm fully aware that my system will sometimes generate false negatives but there's no such thing as a fully objective interview. I have a few technical things or topics that I usually ask about and I expect every candidate to know at least a few of them but certainly not all. I generally go forward topic by topic until the candidate has at least N things s/he knows about. Of those that the candidate does know about, I ask more and more details until I run out of questions or the candidate runs out of answers. If I'm getting strong signals early that the candidate is likely to be ok, I'll just try to finish up the technical part quickly. Then I proceed to the most important and revealing part which seems to be asking about a project that the candidate is really proud of, work or hobby. In the best case I get a lecture down to details on something the candidate built that s/he's still excited to explain to someone, on the coolest things s/he could build into that project. A good programmer pours so much passion into some, possibly minuscule, part of what s/he's building that you surely should be able to ooze some of that out back. Another important part is asking about hobby projects: if one particular candidate doesn't program on his/her spare time, I'll probably go with any other candidate who does unless s/he's exceptionally strong otherwise. I try to remove pressure from the interview by acknowledging that given some rough preliminary filtering, I could go wrong either way. Maybe I reject a good candidate because I don't know as much myself. Maybe I accept someone who's really nice and who seems to know about things, passing my filters, but turns out s/he just can't produce much code hands down. All that will happen some day. The trial period is there so that both parties can revert an obviously wrong decision. We haven't needed that yet but I greatly prefer to postpone some responsibility for later and relaxing the actual interviews as much as possible. People don't like to be grilled and I don't like to grill people, not only because it's very consuming but because it doesn't seem to be a particularly effective indicator. There will never be a silver bullet to interviewing because there will never be a silver bullet to meeting people the right way, but if I were to suggest one thing it might be to listen more and ask fewer questions. By listening, I don't mean letting a babbling candidate take over the interview. By listening, I mean to figure out who is this person, where s/he's coming from and where s/he seems to be going. Being the fastest and most accurate shooter isn't much of a value unless you know what you want to shoot, what needs to be shot, and why.
• monksy 12y ago
  > This is why personal references and recommendations remain everyone’s favorite hiring technique… > …which in turn is a major reason why the tech industry’s diversity numbers are so disastrous. I'm not buying that argument. I would however agree that recommendations and references create a lack of diverse ideas/approaches. Lack of racial/gender diversity: That would indicate that the people in the minority just aren't reaching out or participating in the community.
  • skybrian 12y ago
    You make it sound so easy. Perhaps they've tried, but it's harder for them to connect? Without actual data, I think I'll go with that; it fits better with what people are saying.
    • monksy 12y ago
      IMO: Its more of people being shy and/or drop the expectation that they'll accepted by the community in the first go. They should come out to the non-discriminatory user groups. (I.e. groups that don't ask for a particular gender i.e. xNUG/xJUG etc) I've found, in others and myself before, is that I've held myself back from going to meetups because I didn't feel immediately welcome the first go, or that I didn't think I'd know people. However, you can't expect for everyone to welcome you with open arms in the first go. It's easier to blame the issue on racism/sexism, than to admit that you may just be shy. (That being said, I attempt to introduce new people in my meetups to the group and others)
      • skybrian 12y ago
        It could also be distance or lack of time or avoiding creepers or having more fun things to do or not knowing about the event in the first place or some hidden factor we didn't think of. The point is that there's a filter and its effects aren't necessarily known to us, so it's hard to deduce what's wrong when some people don't show up. It seems like figuring out problems like this is what marketing is all about.
        • 12y ago
• BurningFrog 12y ago
  This article seems to be written inside a different bubble than mine. Then author lives where all technical interviews are conducted in a similar way, and thinks that is the universal truth everywhere. In my bubble things aren't perfect, but they're quite different.
• mingmecca 12y ago
  I lose about 20 points of IQ in a technical interview. It's nerve racking (I wear black shirts to hide the sweat) and it cost me a job working for a well known VR company. I passed the 'practical', which was a 2 hour coding test to generate a working game application, but the tech interview probably did me in. I've been programming games for almost 20 years, and as far as I know I'm one of the few who has done lead/senior level work on games that were both #1 and #2 simultaneous chart-topppers, but I still couldn't get the job. Their loss I guess, but it still sucks, as I was really enthused about what they were doing and could contribute significantly. I was about 10 years older than their typical engineer however, so maybe ageism played a part. I'll never know. I just wish companies would look at my past body of work, see that I'm friendly and un-abrasive, and bring me aboard.
  • blister 12y ago
    Have you thought about jumping into the world of Serious Games?My company is usually looking for experienced game developers with industry talent. The "Serious Games" side of the house is (to me) twice as rewarding as the other side. We don't live and die on the finicky sales cycle of the public and get to focus on delivering cool products that are already paid for by one customer. If you want to chat more about it, hit me up. me (at) ericharrison.info
    • philsnow 12y ago
      > already paid for by one customer .... tactics sims for militaries ?
• rsuelzer 12y ago
  I remember with horror one technical interview I had. I was asked to complete a fairly basic problem, which I had actually just practiced a few hours before (the fact you can practice for these interviews indicates how questionable they are). As soon as the question was asked, I proceeded to start typing the answer and explaining what I was doing when suddenly my mind went blank, I struggled for nearly ten minutes to regain my confidence but by that point it was too late. The interviewer quickly ended the interview and told me to apply again after I had more programming experience, since apparently he was able to conclude the extent of my knowledge based upon my answer to a single question made during stressful circumstances.
  • army 12y ago
    Um, you can practice for anything that involves communication or demonstrating your skills. Its just arrogant to think that you can go into an interview cold and rely on the interviewer to tease out your positive qualities. If they're a good interviewer they'll do their best to do that but its also on the interviewee to convince the interviewer that they're a good fit for the job. I wouldn't present on a technical topic without preparation, because I'm not going to do a good job of explaining it or communicating the ideas - I'd be wasting the audiences time. Same with interviews - you need to be prepared to explain your work and put your experience in as favorable a light as possible.
• dba7dba 12y ago
  Why not require programmers to submit samples of work (most likely personal, side projects due to legal issues) done over a period of time, like graphic artists do when they apply for positions? I would think that would be a much more correct assessment of the programmer. Interviews should really be limited to check personality, and not much more imo.
  • rifung 12y ago
    This seems like what the article suggested. The issue then just becomes that not everyone has the time to do side projects and some of us are lucky enough that we really enjoy what we do for our job, so even when we have spare time we still work on the company project.
    • dba7dba 12y ago
      While I agree with you, I've learned no company will stay as loyal to you as you are to them. You always need to plan ahead to fend for yourself. And having SOME kind of side project (no matter how small) is always advised, imho, for future employment.
• logn 12y ago
  I think people overstate how much interviews are the problem. Managers/HR are the problem. In most companies the team manager has little to do with finding and initially screening candidates and they (or HR) are too afraid of firing people. I think managers alone should be handling this. I also don't think hiring by committee decision makes much sense. It reduces the one-on-one evaluation time by the hiring manager. If you have 6 people doing 30-minute interviews of a candidate, everyone gets a few good questions in then the decision comes down to a consensus of gut feelings. Plus the manager can deflect blame for bad hires. But changing this require re-organizing people and job duties in a company and they'd rather just look for the next hot interviewing trend. (Edit) But as far a trends go, looking at real work is at least better than contrived exercises.
• jimbobimbo 12y ago
  I've been on both sides of the process and in both cases I prefer the whiteboard. As an interviewer, I simply don't have an hour or two to evaluate someone's project. I want to see a strong resume and ask a few questions to understand whether the resume is real or not. A simple whiteboard question or two is great for that. You can always see whether the person is capable of writing code and it always brings up questions around the choices they made. As an interviewee, I rather not be engaged in a live coding session: the whiteboard is a very efficient tool to get an idea across, without sweating on syntax errors, searching for API documentation and whatnot. Test projects - I have a job, thank you very much.
  • mikekchar 12y ago
    I am going to have to disagree with you. The whiteboard question does not always work. In fact, while I am generally able to do whiteboard questions without a problem, I had an experience 2 years ago where I completely blanked and was absolutely unable to write any code on a whiteboard. If they had handed me a computer (or if I had not been too flabergasted to ask for one), I would have been fine. In the end, the company in question lost out on my skills and I ended up at a different company. Not really so bad for me, as I have never been scarce of job offers, but it made the problem very obvious to me. It is easy to dismiss something that you have never experienced yourself. Some people are good at whiteboard interviews and some are not. In my experience, the ability to be good at whiteboard interviews is mostly independent of how good the person will be at their job. The questions are either too difficult to avoid the stress-blanking issue, or too easy to weed out the barely-capable-but-not-someone-you-would-ideally-like-to-hire candidates. We currently do pair-programming interviews with ping-pong. I write a test, you write the production code, then you write a test and I write the production code. We discuss programming issues as we go. This seems to work well for people who are familiar with TDD and real pair programming (as opposed to: one guy codes and the other guy watches them). It does not work so well for people who have never tried it, though. We actually do most of our work this way, so finding people with this experience is good. Unfortunately, I think we've almost certainly missed good talent just because they don't immediately grasp how this process works. We do some hand-holding at the interview, but then you are always wondering about the person's actual ability. @jimbobimbo, if you don't mind some unsolicited advice, I would recommend that you revisit your stance on maintaining a few non-job projects. A portfolio that shows what you can do on your own is unbelievably valuable (in terms of real dollars) when you are looking for the next job. I have gotten good jobs in the past simply because someone stumbled across my portfolio and decided to hire me. The time required is not all that bad -- say an hour or so 3 times a week for the next year or two would give you a really excellent portfolio. It also allows you to experiment with techniques and tools that would be too risky to introduce into a project without some experience. Building that experience outside of work, also makes you much more valuable even if you don't switch jobs. I think you will find that while it is difficult to find the time, the pay off is excellent. Avoiding doing low priority overtime at work and replacing it by investing in yourself is a good way to start. Having said that, my portfolio is currently a complete mess and I really need to spend some time on it ;-)
• bsder 12y ago
  Look, if you want to talk about code, the easiest and best way to sidestep this issue is to have people bring code (somewhere between 200-1000 lines) to the interview. While they may not have a github presence, they've probably written something on a computer outside of business at some point. I really don't understand why more people don't do this. It doesn't even matter if it is their own code. The fact that they will have to talk intelligently about it means that it will accomplish its task. I used to get this issue interviewing with VLSI chip companies all the time. "Do you know Perl?"--"Yes, I do, but I don't know YOUR particular favorite subset of that write-only language" doesn't go over well in interviews. After getting tired of getting dinged on questions about a language that I had used for almost daily for 5 years and that I abandoned for good reason, I took an old program of mine from Python and ported it to Perl. I bring both. Now, I have something concrete to talk about, can actually compare the points of two languages, and most interviewers realize that "Gee, you're probably better at this than I am." This shifts the conversation from "Technical Jeopardy!" to "Ah, you know how to program, let's look.", "I did this. Here's why.", "Oh, that's interesting. What does that do?", etc.
  • Darmani 12y ago
    I tried this with the first few people I interviewed. None of them had any code from outside work to bring. (This includes a brilliant and hardworking coworker that I practice-interviewed -- he just hadn't coded outside work in a very long time.) We wound up switching to giving code challenges instead.
    • pmiller2 12y ago
      >It doesn't even matter if it is their own code. The fact that they will have to talk intelligently about it means that it will accomplish its task. I think you missed this part. Anybody can go on github and grab a repo and pick a specific section of code to talk about. This still accomplishes the goal of finding out if the candidate can understand code and communicate about it.
    • jwcrux 12y ago
      Why not offer both options? Bring in a piece of code you want to talk about, or have the option to do a coding challenge. Either way, you're asking the candidate to develop code, and then explain their decisions in a meaningful way.
• yeureka 12y ago
  I conducted quite a few interviews at my previous job and my solution for the whiteboard anxiety problem is to give a set of coding exercises/questions on a piece of paper to the candidate and give him/her as much time as necessary to answer these in private in a separate office. When the candidate is ready he/she announces it and we review the solutions together. It is not perfect, but I find it much less taxing than being asked to solve a technical problem in a white board in front of an interviewer.
• suchire 12y ago
  IMO, having interviewed hundreds of candidates, whiteboard coding selects works very well for hiring a very specific type of fresh-out-of-CS-undergrad student, but it breaks down in almost every other scenario. For more experienced candidates, we've swapped out most of the whiteboard questions for several other types of interviews, including in-depth technical discussion of one of their past projects, a discussion of how to architect a non-trivial system, and a more soft-skills interview on how they work with others, break down projects, and so on. This gives us a much rounder picture of a candidate, and allows candidates who are very good at something to shine a bit more.
• moey 12y ago
  I think the best approach would be options for different people to choose. Think you can pass a hardcore whiteboard interview? Go for it. Would you rather spend a day and solve a real problem we have? Lets see what you got. Think you have the attitude, just not the knowledge to match our exceptions yet? Do you have time for an internship? Ok cool. Lets play. One size fits all is a bad idea for interviews. Just as how teachers test their students.
• crdb 12y ago
  I've seen two kinds of motivations: that for career climbing, and that of an engineer who enjoys making things. I personally prefer my engineers to have the second. I tried hiring the normal way (CV, white board) and had one candidate worth speaking to in over 200 applicants. We decided to try something else. We stopped reading any cover letter or CV and wrote that in the job ad. We asked candidates to build a simplified version of what the job was about, taking about 4 hours, then to send us the code which we'd discuss with them over the phone at their convenience. Anybody whose code passed would be hired, and if people couldn't be bothered because they were already famous they could send us their libraries. Nobody took the Starcraft best of 7 option :( Why we did this: - having to write code eliminates those who can't, or who can't be bothered; not having complex unrelated hoops to jump through eliminated those who are good at career climbing; - we replaced the hours candidates would normally spend dealing with HR or travelling to a site for something ("cultural fit") that overselects the polished anyway, with something people we want would enjoy doing (solving a problem, writing code); - no pressure, access to your "external brain" (Google, your own libraries) unlike with a white board; allows for the anxious to pass, and those who outsource a lot of their knowledge to the machine (like me); - you can tell a lot about how people will approach their job from the way they approach the small version of it; documentation, structure, abstraction levels, choice of libraries, and if you're wrong about the candidate's reasons you can always clear it up in the chat; - it's fair because everybody has to tackle the same problem; - no discrimination since it's all ability based: I took to Skype-text-chatting with candidates instead of voice calls mostly because I don't like talking on the phone. It was only when she sent us her passport scan for the contract that we realised one of our candidates was female. Another team member didn't have a CV; it's because he was just finishing high school, as we discovered when we talked to him, but his code was good enough, so he got the offer and skipped university. I've never had to put out another job ad (not for myself anyway) because I have so many qualified candidates left over from that round, and from the team network. Surprisingly, we didn't get spammed; the requirement to post code seems to have been mostly understood and we only had a half dozen "dear Sir/Madam"s. So I disagree with you, but YMMV. We were hiring for a small, distributed team solving relatively well known problems. Requirements are probably different at Google or Facebook.
  • teen 12y ago
• kamaal 12y ago
  The best interview I've had is where I was given a small project but with a tough deadline and I was asked to come back with a working code/demonstration. The remaining interview was a code review session, and we talked about design, choice of tools, libraries and other general stuff. I was stunned by the end of the interview because they got out everything one would want to know about how good a person is at their everyday job. The worst interview I face is when people try to test my algorithm skills, that generally a test of how much I could memorize from careercup.com
• sytelus 12y ago
  The guy who wrote this is not even a real professional software developer[1] but it seems that he has strong opinions on technical interviews anyway. In fact I looked up few people he mentioned in article and none of them seem to be real full time software developers either. Nevertheless whoever has microphone gets to get heard, I guess. I have seen a lot of PMish/weak programmers make cries about technical interviews. Yes, whiteboarding interviews are not perfect and false negatives rate is typically high but the companies who want nothing but the best would care less about false negatives and rather more about false positives. Anyone who has worked long enough in software engineering professionally knows the enormous cost of false positives [2]. These sort of companies also typically care less about which language and frameworks candidate knows at the moment. They rather put huge weight on evidence whether a candidate can analyze a challenging problem, apply computer science and obtain a solution. Why? Because these companies are actually working on problems that does requires deep hard computer science. Most - perhaps 80% - of software development houses are not like that. Programmers in those companies have never bothered to think about collisions in hash tables and they often wonder why anyone would care when libraries take care of everything. They never need to create data structure for trees or graph or even linked lists because simple things have always got job done. They wonder why they are being asked questions on those "arcane" things when nobody really uses computer science stuff that they were taught at school. That's the culture problem. Those 80% of companies tend to copy interview process at other 20% even though it is meaningless for what they do. Interviewers typically chose their questions from algorithms text books or even internet searches. Then they get stuck on it for a decade and proudly mention it as their "favorite question". That's completely wrong. I almost always tend to ask CS questions that I myself needed to solve to get my job done. I typically retire my questions in month because I probably would have found another problem where applying right algorithms and data structures was the key. When I ask these questions to candidates, (1) it's rare chance that it would have been covered in books like Cracking the Coding Interview (2) no risk because candidate decides to do interview brain dump (3) I know candidate is smarter than me if s/he can solve within hour under pressure (4) If they fail miserably I know this guy could have screwed up our project just last month had he been hired. Bottom line: Ask questions from your actual day job. It takes a lot of skill and effort to abstract away other complexity and form a good interview questions and that's why being an interviewer is hard. Whiteboarding works if you do that. It doesn't work if you are trying to copy companies which are doing different kind of work than you do. It most certainly doesn't work if you never actually needed to solve the problem you are asking to get your actual job done. 1. http://rezendi.com/aboutauthor.htm http://rezendi.com/aboutauthor.htm 2. http://www.joelonsoftware.com/items/2007/06/05.html http://www.joelonsoftware.com/items/2007/06/05.html
• danans 12y ago
  Articles demonizing coding interviews always attack strawmen that are the worst kind of interview questions: 1) Trivia questions (i.e. "what is the name of the method to do X in Java") 2) Questions with precisely 1 solution (you either get the one solution or you don't). These are indeed useless technical interview questions. But that doesn't meant there aren't good technical coding interview questions. I tend to favor the kind that present a candidate with some example data and a set of constraints or patterns, and ask them to write code that analyses the data for those patterns or constraints and reports them. This sort of problem is ubiquitous in my domain. Importantly, I always choose problems for which there are multiple possible attack angles, not just one. And I don't give a hoot about syntax errors, or what language they use. This sort of question gives you a good sense about the candidate's analytical capability (breaking down the "word problem"), and their ability to translate their problem-solving thought process into code. Because there are always multiple angles of attack, candidates have some leeway to exercise creativity. In the end, it's not terribly important to me that they get the optimal solution. I do care whether they demonstrate strong analytical capability in the literal sense, meaning they can decompose the problem and the associated programming exercise into their logical parts and implement them. I also look for good communication skills in the questions they ask when reasoning through the problem - this is something that only an in-person technical interview can reveal, AFAICT. There are probably many smart candidates that don't do well on these questions because they're just having an off day, or nervous, or don't perform well under pressure. I sympathize with them, because I've been there and felt all of the above. But if the goal of a technical interview is to assess a candidate's analytical and coding abilities, and their ability to do both simultaneously, there is no shortcut I know of to just giving them a role-relevant problem to work on. EDIT:wording.
• rifung 12y ago
  I completely get that technical interviews aren't great at accurately predicting success on the job. What I don't get is all the whining that seems to happen from interview candidates who seem to think they are hot stuff and deserve to get hired but blame it on a bad interview process. It's not like the interviewers aren't aware of the shortcomings of the technical interview. I have yet to learn of another industry where people routinely blame interviewers for their being rejected. I mean it seems like in other industries people are just hired almost solely based off resumes, and while everyone also realizes it sucks, they don't seem to think the interviewer is an idiot for not hiring them.
• fit2rule 12y ago
  I just went through one of these terrible interviews at a local game company, and I have to say I agree with this article 100%. I can code, have been doing it for 30 years, and I can handle myself in challenging technical situations - and I have the resume to prove it. But put me in front of 3 guys I've never met whose purpose for being in the meeting is to expose every single one of my weaknesses, push me up in front of a whiteboard and ask me to explain the best way to solve a maze puzzle, and I'm going to freeze up and fail the interview. The reason is, this is simply not how I work. I work by sitting at my desk, thinking about the problem presented, and writing code to have the computer do all the work. It may very well be organizationally convenient to have these psychological lynchings occurring, but I highly doubt it gets the absolutely best candidate in the door. I look forward to new solutions to the problems of finding qualified people; in my case, I feel I unjustly failed an interview at a company I could have been quite productive. Such is life in the modern software world, alas ..
• apinstein 12y ago
  I have found the most reliable measure for a candidate is looking at their public interactions/contributions. Seeing what projects they choose to work on in their spare time, how they speak at user groups, how they interact on an "issue" online. Blog posts are also great. For me, ability to communicate is such a huge part of being a successful developer, that being able to see how they communicate is always a very strong signal of intelligence, mastery, and interpersonal style. So this works great for people that partake in such things, but there are clear still many great developers (perhaps the vast majority) that don't have this type of extensive public profile. These are the applicants that I fear false-negatives for the most, and the candidates for which I am not confident of my interview techniques. Could it be that the proper way to technically interview such a candidate is to offer them a "fellowship" to work on an open-source project? Point them at any project (used by the company or a favorite project of their own) that they'd like to contribute to, and offer them a small stipend and enough time to make a legitimate contribution to open source? And use that as the technical portion of the interview?
  • dgemm 12y ago
    I think the challenge here will be in the overhead of finding and evaluating suitable projects, given that you can use each one only once.
• apinstein 12y ago
  I have found the most reliable measure for a candidate is looking at their public interactions/contributions. Seeing what projects they choose to work on in their spare time, how they speak at user groups, how they interact on an "issue" online. Blog posts are also great. For me, ability to communicate is such a huge part of being a successful developer, that being able to see how they communicate is always a very strong signal of intelligence, mastery, and interpersonal style. So this works great for people that partake in such things, but there are clear still many great developers (perhaps the vast majority) that don't have this type of extensive public profile. These are the applicants that I fear false-negatives for the most, and the candidates for which I am not confident of my interview techniques. Could it be that the proper way to technically interview such a candidate is to offer them a "fellowship" to work on an open-source project? Point them at any project (used by the company or a favorite project of their own) that they'd like to contribute to, and offer them a small stipend and enough time to make a legitimate contribution to open source? And use that as the technical portion of the interview?
• nevaben 12y ago
  I've found this technical interview process to work very well. This is done after the initial phone screen. 1. Show them a problem with your product along with the code. For front end developers it could be how form invalidation errors are being presented to users. 2. Ask them to figure out what why the code is doing this and observe them troubleshoot the code. 3. Tell them to fix the problem in the code and observe them apply a fix, test, and debug it. 4. Ask them to architect a better solution to the problem and to explain what makes it better. What would would be drawbacks to their solution. The benefits of this approach is that you can evaluate directly how someone solves problems, not how well they communicate, nor how knowledgeable they are. It also helps me judge how fast they are. Sometimes I ask people to estimate how long it would take to solve this and time them. The last step helps establish how they think through their solutions and how well they can communicate their ideas. Additionally, I get to see how quickly they might get up to speed with our codebase and I can hear other solutions to some of the technical problems we are facing which is incredibly beneficial as a small startup.
• jdmichal 12y ago
  Is it maybe time to start allowing software professionals to have a portfolio? I would guess that companies are the main driving force behind not having portfolios that can be shown, but at the same time they are thereby hurting themselves because they can't properly evaluate employees.
• socrates2016 12y ago
  An issue with technical interviews is the shortness of the measurement. Real-life programming occurs over days, weeks, and months. You think about a problem and come up with progressively better solutions while thinking through and understanding tradeoffs. Its more a marathon than a sprint. Technical interviews are like sprints. So its as if you hire runners based on how well they can sprint but then race them in marathons.
• amarjeet 12y ago
  I follow this process for most of tech/product/analyst type roles. Specifically, in the case of tech, to me, it really depends upon the level of coding exposure a candidate has. If he/she is a fresher then I would prefer some real-time exercise (white-board/phone/skype exercises) - it helps me get the idea of a person's thought process. I don't look for right answers in that exercise. If the candidate has some code exposure, I prefer to dive deep into his previous work and figure out how he executed the projects. In both the cases I am trying to asses a candidate's approach to execute something. I also try to figure out the candidate's ability to pickup new things and start executing them quickly. This is measure of a candidate's potential. I assign similar weightage to 'Potential' as to 'Past Work History'.