21 ms·

The two questions I ask every interviewer

307 points by deafcalculus 9y ago

_chris_ 9y ago
>If you want to make a change to your interview process, give it to some of your current employees first. If anyone fails it, ask yourself if that person should be fired. The answer is probably no. That's a fascinating idea that's never crossed my mind. E.g., how many current employees can pass whiteboard coding exams?
- FeatureRush 9y ago
  Apparently there are already companies that take it a step further. They test sample of their employees with some fancy IQ test that measures them across several dimensions. From those results profile of desirable new hire is build and all candidates get measured by the same test. On reddit in threads about interviews I've already seen couple people discussing those strange tests where in many questions there are no obvious correct/wrong answers, like for example all presented shapes can in their own way fit in presented pattern.
  - eklavya 9y ago
    If there is no obvious right/wrong answer, I wonder if they are limiting their pool taking people who all think alike (kinda like in-breeding) or are they using it diversify the ideas and knowledge etc. (like diversifying gene pool).
    - FeatureRush 9y ago
      Different answers probably give points to different traits the test measures, the ambiguity is just irritating for test takers coming from more classical tests where you know (or at least suspect) what is expected from you. This whole test setup is not cheap so one can at least hope they considered the issue of diversity? The other thing is did they actually tested which teams perform better in specific problem domain that interests them?
  - ataturk 9y ago
    Ugh, that sounds so Orwellian. Corporate Eugenics.
- pavlov 9y ago
  Many companies instruct candidates should study for interviews with specific books or courses (e.g. “Cracking the Code Interview”). That indicates the idea is not that someone who is a competent professional should pass without studying. The passively hostile whiteboard coding exams are more like a hazing ritual than an actual job performance test.
  - watwut 9y ago
    I prefer that over being ambushed by unexpected topics. Here, you measure ability to learn, at minimum. Unfortunately, you measure also how much free time candidate has. Which is still better then having candidate to guess what is popular now or which if competing preferences your company has.
- tomseldon 9y ago
  Testing the change is one thing, but if the original people were hired and asked something very similar, you have an existing group selected based on that characteristic and so will likely do well. e.g. if you have always done a whiteboard interview, then you've selected for people that do well in that type of scenario. Changing the question but keeping the format will likely just prove that those people are still suited to that type of task, but not that an otherwise good hire will be filtered out. I'm not against whiteboard interviews or similar, but just pointing out that selection bias is an important factor here if you're judging effectiveness by comparing it to results for existing employees.
- ssijak 9y ago
  Interviewing is a random event for many companies. You as a candidate could fail miserably on one day and get great feedback the other day. It depends on so many factors in larger companies. For example in one of my previous employers, they would send a random engineer (juniors included) to interview a candidate and give a mark from 1-10. There were engineers who would not pass their own interview and questions, and on the other hand, there were others who would just see that you can code anything and you are cool to work with and would let you in. And anything in between. There were times where juniors wanted to be smart and asked some puzzles they got from the net, and when later I asked them to solve them they failed. So many interviews depend on so many random events and circumstances and. The biggest takeaway I got from all this experience is that as a candidate I should not be intimidated by anything and should just be cool and myself, because there is so much randomness that trying to find reason in it will just make things worse. And ALWAYS negotiate as much as you can because your biggest raise is when you get hired.
  - ataturk 9y ago
    What you are describing is a phenomenon referred to as "that's just life." It's just random shit--one day you interview the team there could be going through holy hell trying to solve a problem under a deadline. Another time, they're in a lull and screwing around on Slack and in a good mood.
- BeetleB 9y ago
  >That's a fascinating idea that's never crossed my mind. E.g., how many current employees can pass whiteboard coding exams? I once confronted a colleague who had refused a candidate because he couldn't answer a technical question. When I heard the question, I asked him "How many people in the current team can answer this?" (Hint: Only one). His response: "Yeah, but we told him to read the first 6 chapters of textbook X!" So we denied him a job because he didn't do his homework.
pjc50 9y ago
Note that if your company is small, it can be very hard to do any kind of meaningful statistical process control on hiring. The "track careers of people you rejected" idea is an interesting one, but depends very much on how public people make their careers. I suppose you can rely on LinkedIn for 90% of people. (What is HN's opinion of the Stack Overflow Developer Story?)
- richmarr 9y ago
  We run an early stage applicant tracking system and see ourselves broadly as a rejection engine. Something like 95% of the activity on our system relates to candidates who end up being rejected. Since our assessment data is structured our current iteration of the product uses that to create value for rejected candidates, informing them where they were strong or weak in comparison to the rest of the group, and showing them some aggregate scoring. It goes down very well with hiring managers, who typically don't have the time to respond in detail or at all to rejected candidates, and (for the most part) goes down very well with candidates.
- ec109685 9y ago
  Very good point. A nice blog post illustrating the problem with drawing conclusions on small sample sizes: https://jvns.ca/blog/2014/07/11/fun-with-stats-how-big-of-a-sample-size-do-i-need/ https://jvns.ca/blog/2014/07/11/fun-with-stats-how-big-of-a-... You need a lot more than a startup’s number of employees and their eventual job reviews to make strong conclusions on the quality of the interview process, and need to rely more on industry best practices instead.
- speby 9y ago
  >The "track careers of people you rejected" idea is an interesting one, but depends very much on how public people make their careers. I really like this idea. This reminds me a bit of the "anti-portfolio" that some VCs track (aka a bittersweet regret of the unicorns they passed on years ago). I think the same could be said of great talent that an organization passed on that then goes on to build great products, invent, and/or otherwise lead great teams despite that. However, employment is a tad different than investment so, to me, the only caveat to this is that it simply isn't the case that every super talented person will bring about great results and value creation at every single organization. Much of what a talented person's output depends on is the very room full of other people in which that person works with. And while "fit" is such a cliche thing these days, this is what matters so much in terms of whether or not an otherwise talented person will be successful or not within one organization versus another.
mlashcorp 9y ago
I always interview for "culture fit". In my experience, that's almost always the bottleneck, and the technical part is rarely a problem. Learning a new skill is not an earth-shattering problem if you have a good attitude towards work, and good interpersonal skills are a must in teams
- richmarr 9y ago
  That prompts two questions for me if you don't mind sharing. (a) How do you define and assess "culture fit", and why? (b) How do you know the things you're optimising for are actually good proxies for productivity? Appreciate these aren't questions with simple answers (or maybe even answers at all), but hiring/assessment process is my current field and I'm always curious where peoples thinking is at.
  - Spooky23 9y ago
    That’s a fancy way of saying “we hire people we like”. It’s an approach that isn’t perfect but is good enough for many roles. We pick spouses based on that approach and it works out pretty well for most people.
    - mlashcorp 9y ago
      You are correct, "culture fit" is not a good descriptor of what I meant. We hire people we believe we will be able to work with. It's not a popularity contest, we evaluate problem solving, analytical thinking and communication skills, such as the ability to explain complex topics in easy terms. I'll give you an example, usually we are interviewing people straight out of the university, but this can be applied to any candidate. I'll start by asking the candidate to talk about his master's project. As he talks about it, I'll guide the conversation towards these topics : -What was the problem -How was it defined? -How did you tackle it? -Explain your solution -support your decisions -Was there a cost/benefit analysis? -How was the problem decomposed? This will give me an initial picture of the candidate's ability to address a problem, and is far more important to me than wether he knows "the number after F in hexadecimal" or some other nonsense.
dvt 9y ago
> I've heard "Hmm, I don't actually know what it is that we're looking for" from a few recruiters Why do you even care what recruiters say? They're there to make as many hires as possible and get their commission. Don't get me wrong, this is actually a decent question, but you might as well ask a car salesman what he likes doing in his free time. It doesn't matter -- he's trying to sell you a car. > How do you evaluate how well you're meeting your goal? -- The majority of the companies that I ask this to essentially answer "we don't" to this question. I'm going to go ahead and call BS on this one. Literally every company (outside of early-stage startups) has some kind of performance review. Most of these are completely artificial and completely suck, but they're definitely there.
- tehwalrus 9y ago
  Performance reviews and their flaws are discussed in detail later in the article. In particular, they detect false positives but not false negatives.
- ndh2 9y ago
  The point was the performance reviews aren't used to evaluate or iterate on the hiring process.
nmalaguti 9y ago
Each time I read a post like this I say to myself “they’re not wrong, but something is also missing.” If there was a way to evaluate a candidate based on how well they would do at their actual job, in an hour or less, that couldn’t be games or cheated, we’d all be using it. That isn’t to say we shouldn’t work on improving the hiring process, but how much time, both from the perspective of the candidate and the company, is too much time to invest in a potential hire? Would we get better signal if you came to work for us for a week and paired with everyone on the team? Sure! But how many candidates can take a week off from their current job? The author talked to 11 companies. Could they invest 11+ weeks in their job search full time in order to find their next role? What about the loss in productivity? Can you have more than one candidate in the office in a given week? What if your ramp up time for full time employees is actually longer than a week? Are you biasing for people who ramp up fast instead of people who will be ultimately more productive and impactful? I often feel that engineers write these posts because they know that they themselves are good at their job, but feel it is silly to have to develop an alternative set of skills in order to signal that they are a good hire. Or they get rejected and blame the process or those alternative skills. I’ve felt the same way and wanted the system to be better tailored for my skill set, but balanced against the time investment for some alternatives, I see why we have the process we do.
- rlpb 9y ago
  > I often feel that engineers write these posts because they know that they themselves are good at their job... The Dunning-Kruger effect suggests that almost certainly, some non-zero proportion engineers who have written this type of posts are not. https://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect https://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect
  - custos 9y ago
    While what you're saying is true, I think the dunning kruger effect has an inverse bell curve based on level of confidence vs knowledge. High confidence low knowledge. Moderate confidance moderate knowledge High confidence high knowledge. Of course that only applies to the things the expert knows like the back of his hand. There are things I am 100% confident I know, because I've done it thousands of times. Then there are some advanced things I still question if I truly understand because I've only successfully done it a few times. Of course when I was younger, the things I'm 100% confident now I went through the phase of "oh I can do this super easy" to "holy crap this is harder than I thought" to "I've done this 1000's of times, I got this"
dandare 9y ago
His point, that hiring process is not a scientific process, is probably correct. But the same is true about performance review process (in my opinion). At best, the performance review is unscientific and biased, at worst it is a popularity contest, while usually, it is a formality. > I've never had an interview that tried to evaluate my ability to work in a team or prioritize tasks - both of which are probably more important to doing well in a software engineering job than being able to find anagrams in O(n) time. And I have never seen a test that could reliably evaluate your "ability to work in a team or prioritize tasks". The hiring process is usually unscientific because making it reliably scientific is very expensive.
- ataturk 9y ago
  Performance reviews are designed to be gamed by managers. I have never seen one that a) wasn't gamed and b) had any scientific basis whatsoever.
- rafiki6 9y ago
  Yes, but what about the commonly trotted out line of "hiring mistakes are expensive so we'd prefer to not hire than make a mistake". Perfect example of having cake and eating it...You have to eat the cost somewhere. Either it's in the hiring process, the firing of a bad employee, or spinning your wheels waiting for the right candidate.
kreetx 9y ago
A side note, but the latter two answers (2. being a good fit and 3. doing a good job) sound very much the same to me.
- 0xffff2 9y ago
  I think "fit" is referring to ability to get along with the rest of the team, regardless of ability to do the job. They aren't quite opposites, but they are mostly unrelated.
  - kreetx 9y ago
    It can mean that too, yes, but if the recruiter word-by-word says 2. then it could also mean 3. But then the rest the rest of the argument quite some weight since 3. is not rare anymore, as it's included in 2., too.
kevinmobrien 9y ago
Interviewing is subjective. It’s about sitting down in a specific context and extrapolating, based on incomplete information, about a future set of conditions that will likely change when time catches up to them. If we can start with the idea that interviews are a very blunt instrument, the process becomes at least somewhat compressible: it selects for people who recognize the parameters and perform well within that context. At best, that pattern-matches to success. At worst, it filters out talented folks who don’t interview well, but could code well.
lucideer 9y ago
> less than 10% of the companies that I've interviewed with have said that their interview process was designed to evaluate how effectively someone could do the job that they're being hired for. I don't think an interview process can do this tbh. A good interview process can evaluate if they can do the job that they're being hired for, in most cases, but I would say only an actual trial/probationary period can evaluate how effectively they can do it. And even then, how effective they are may be environmental - completely external to their own internal competence. After concluding that a candidate can do the job, the secondary purpose of the interview process is filtering for people that will positively affect the work environment of their colleagues (and/or filter out those who may negatively affect others). As an interviewer, I would make some effort not to lean too heavily on my own biases and try and also consider how they'd get on with my colleagues that I know well, but there is unfortunately always going to be some implicit bias here.
derekmhewitt 9y ago
Just as a spitball idea... What about a structured "cohort" hiring process where you give potential employees short term (literally one or two days, where you have a group exercise like a "hackathon" where you make a sample product for demo, then moving up to one or two week rounds) contracts that allow you to evaluate and accept (pass on to future rounds) or reject a large candidate pool and thereby filter your potential future employees. Given the current costs of recruiters, HR personnel, interviews and headhunters this might actually end up being cheaper than current "traditional" hiring methods and could generate some very effective development teams.
- scrumper 9y ago
  That means you can only hire people who don't currently have a job.
dilemma 9y ago
Using proxies for competence is a slippery slope towards misdirection and irrelevance - and as a result, incorrect hiring decisions. In another thread today, someone mentioned they reject every CV that has two typos or more. It's a proxy, and it is irrelevant for the job. Once you start going down the path of proxies, you add more and more far removed ones. To keep hiring performance high, resist adding the first proxy. Actually ask questions relevant to past work experience that maps to the actual job. Maybe do work samples. To circle back to the two questions in the blog post, no interviewer can answer them because they use proxies too far removed.
- richmarr 9y ago
  Agreed. Work samples are the most predictive form of assessment, you just need to be able to isolate them from other factors such as confidence bias... i.e. no whiteboard interviews (unless hiring for someone whose job it will be to solve problems on whiteboards in front of strangers in situations that effect their financial security and sense of worth)
- logfromblammo 9y ago
  Think of the resume as a deliverable. This potential employee sent out the deliverable without reviewing it for spelling and grammatical correctness. They almost certainly did not ask someone else to proofread it and suggest corrections. Would you want to put this person in a position where they may have to deliver binaries, code, or other documents to a customer? Would they ignore or skip steps in your delivery process, too?
arkadiytehgraet 9y ago
As an interviewee, one of the questions (that I shamelessly stole from somewhere on HN) I like to ask recruiters is this: "Do you like working here?" The answers and the way the answers are given usually tell you quite a lot about the company and the specific place you are going to work.
- comstock 9y ago
  It’s a fair question, but if you asked me that in front of my boss or coworkers I would likely immediately take a disliking to you because you put me in an awkward situation where there are few easy ways to answer honestly.
  - arkadiytehgraet 9y ago
    This is why this question is so good: you either like working there, and then you have no problem answering this question honestly (and without any awkwardness), or you don't like, then surely you have to come up with some elaborate yet unclear answer to this question, which is quite clear at that point, and then I do not really care whether you take a disliking to me or not, since it does not really matter any more.
    - comstock 9y ago
      Maybe in general that’s true. In my case even if I feel it’s a nice place to work, I wouldn’t like being on of the spot in front of my boss/coworkers. In a one-to-one interview it would be fine. Which was perhaps more the context of your comment.
      - burger_moon 9y ago
        Not being able to speak freely in front of your peers would be a hard pass from me. I'm not going to work somewhere I can't feel comfortable speaking honestly.
        9y ago
deleted 9y ago
[deleted]
comstock 9y ago
> If you want to make a change to your interview process, give it to some of your current employees I don’t think I would respond well to this as an employee.
- dx034 9y ago
  Why not? I think it's a valid test for a company. If the majority of your employees wouldn't pass the hiring test, why use it for new employees (unless you're looking for someone more senior)? Employees can probably best give feedback on how to adapt it so that it mirrors the actual work.
  - comstock 9y ago
    Many people seem comfortable allowing their companies to treat them as low value commodities and will do anything asked of them. In general I don’t exactly think like this. I do the work within the scope of what I’ve been hired to do. I’m lucky, at least right now, that I have this choice because my skills are somewhat rare. Want work outside the scope of what I’ve been hired to do? Pay for it, or find someone else. I don’t want to be asked to interview again, it’s not a pleasant experience for me, and critically it’s being presented as “something you have to do”. If you want to do this, offer additional compensation for doing something outside of the scope of my work.
    - dx034 9y ago
      I think a good work environment (and that includes competent coworkers) is in everyone's interest. What if you can take the test but the employer doesn't know the result? I'd trust my employer enough not to use interview questions as my performance evaluation. If you don't, the level of trust appears to be very low.
    - BeetleB 9y ago
      >Many people seem comfortable allowing their companies to treat them as low value commodities and will do anything asked of them. If that's how you feel about it, would you not agree that asking such questions in an interview is treating the candidate as "low value"?
notacoward 9y ago
He almost gets it right. Hiring for some abstract "best" isn't so great. Hiring for "culture fit" is practically an invitation to discriminate. OTOH, people rarely do only one thing at a company, so it's important to remember that you're hiring for a job rather than a specific task. You need to evaluate fundamental skills, including collaboration or leadership skills (and styles), not just knowledge of specific subject matter.
- throw999888away 9y ago
  I have mixed feelings on culture fit becoming a bad word. I don't disagree with your point but I think there is a danger with ignoring its value. For example --leaving aside your views on SV startup sweatshops-- if you're in a start up, you will at some point work late hours and you will at some point go through stressful times. Sometimes the things that get you through those times is that feeling of camaraderie and actually getting on with your colleagues. Screening on culture-fit is a good way to guard for that, and at least for me it is unwise to not associate value with that. Now, it could be that your culture is inclusivity. But then isn't hiring for people who fit that mould still hiring for culture fit? Isn't that still creating a monoculture? Isn't discriminating against a bigot brogrammer (who may well be great at coding) still discrimination?
  - notacoward 9y ago
    The problem is that "culture fit" is too broad and too vague. Often it includes things that are completely unrelated to job performance. If you want to make sure people are able/willing to work long hours, ask about that (and be aware that you're probably discriminating to some degree against people with families or health issues). If you want to know how someone will handle stress, ask about that. Not only will you be measuring actual job-related skills or qualities, but you'll also be telling the candidate something they should know about the work environment. On the other hand, hobbies and musical tastes and political views should not be part of a hiring process, and that's the kind of thing "culture fit" often seems to include.
    - 9y ago
johnpython 9y ago
If there is a gross lack of diversity within your company and you routinely use "culture fit" as an excuse to turn down PoC candidates, what conclusion can be drawn about your organization?
austincheney 9y ago
My goals as an interviewer: 1. Do you have a solid understanding of the foundations for your skill/discipline? 2. If you were given $100,000 VC capital and told to build a revolutionary new thing how would you build it? If a candidate cannot adequately address the first objective then I will find another candidate who can. This is pure and simple elimination with simple non-biased questions on things the candidate should know to do their job without abstractions, tooling, and other trendy bullshit. The second objective is where I can evaluate my bias against the candidate's, but only after the candidate has passed the technical evaluation in a non-biased manner.
- qaq 9y ago
  No offense but if candidate has a credible answer to "2. If you were given $100,000 VC capital and told to build a revolutionary new thing how would you build it?" wtf would she/he work for you :)?
  - Can_Not 9y ago
    It's because he wants to focus on building the thing, not focus on acquiring VC funding.
    - marcosdumay 9y ago
      You are really asking to be bullshited (even if people had a good answer, they won't have an elevator pitch to tell you in an interview). Unless you are looking for people that can make an argument, you are selecting for the wrong trait.
      - austincheney 9y ago
        > You are really asking to be bullshited The question is rhetorical, so what does this matter? It is part of a job interview. It is not a VC pitch. If this confuses a candidate and they answer a question not asked I would not hire them. Interpreting and following instructions are important skills.
listentojohan 9y ago
One of the good interview experiences I had, included going through a project I had previously done, framing the problem, how I'd have worked through it (with class/function level whiteboarding), and what could be improved. It was a project from a previous job, but too high level to be replicated. But could see how going through work from a previous job could be problematic (no code was shown).
jasonmaydie 9y ago
"Best people" and "good fit" means they are looking for someone with both technical and soft skills. They don't want a skilled worker who is difficult to manage or communicate with, and they don't want a nice guy who is not skilled. It makes sense to me.
j_s 9y ago
I would like to see YC take over the initial chunk of the hiring process for their companies, and publically document the heck out of how to do it best. Posts like this one demonstrate that free access to the front page of HN for job listings isn't enough: Don't work for CrateJoy | https://news.ycombinator.com/item?id=15486301 https://news.ycombinator.com/item?id=15486301
cubano 9y ago
There are so many cognitive biases in play when it comes to hiring its basically a total crapshoot no matter what processes and shit is in place. People hire people they like, period.
- rafiki6 9y ago
  Disagree, we can basically eliminate the personal part of it. Blind interviewing. I don't need to know name/gender/school etc. Technical skills test as a filter given to current employees to track if it actually tests for necessary skills. Second part of interviewing can be assessed by people who don't see the candidate and we can obscure their voices.
  - mratzloff 9y ago
    The cognitive biases aren't just around appearance or voice...
    - rafiki6 9y ago
      Not sure what else there can be if you create a normalized test that has been designed to predict success by being applied to your current employee population. Take away personal interviewing and voila. I can't judge this person on my inherent bias towards a race, gender, etc. if I can't see them or determine where they are from. The test should be the only bar required to grant a person an opportunity, nothing else. No need for resumes. https://hbr.org/2016/04/how-to-take-the-bias-out-of-interviews https://hbr.org/2016/04/how-to-take-the-bias-out-of-intervie...
scraft 9y ago
The best interview I had (for a lead games programmer position) was a written test, I can't remember the exact details but perhaps 24 pages and 45 minutes to complete it. Before starting I flicked through the entire test to get a feel for what was being asked of me, and then for a little while debated just getting up and leaving the room. It felt a bit crazy, a test on all sorts of disciplines of games programming that I wasn't an expert in, and I didn't intend to become an expert in (and the job I was applying for didn't require me to be an expert in). But I calmed down, tackled the test in the best way I could, answered the questions in an order where my most confident answers (and quick to complete) answers were done first, then moving onto ones I had a pretty good idea of, then ones I knew less well, etc. 40 minutes in, the CEO of the company came in, it is a ~300 man company, so it was interested in itself that he came in. Anyway, he wanted to go through the test, I said I hadn't finished yet, and had made a note of the start time, and still had 5 minutes remaining. He said it doesn't matter, lets have a look. And he flicked through the test and didn't look at my answers and instead found a section of the test which I had completely ignored (it was to do with AI and path finding, both topics I have done almost nothing on during my entire education and career). Of course, this was the question he wanted me to answer, I explained the reasons I hadn't answered it, and said if they wanted me to do things like this it really wouldn't be a suitable job for me. But he persisted, stop worrying about all of these things, just answer the question now. Again I had that slight feeling of wanting to just leave, but again I overcame in. I started talking him through the things I did know about the question, pointing out areas which could cause problems, then started listing what I could do to limit the question to avoid some of these issues (it had a picture of a top down level, the question was inside a box, I said lets initially forget about the box for example). Then I started just talking about an initial algorithm which could route AI around the level - I said it would obviously be really bad (basically AI just walking into things, then working out where to go next, and then a bit later saying oh I could keep track of where I have been in case I end up at the same point, etc.) we talked more about it, he asked some questions, and bit by bit I came up with a solution. He said that's interesting, because during this process you have described parts of various algorithms which someone who has studied AI would know about, but you are going from a brute force perspective, not having any way points in the level, no extra level knowledge. I said I didn't realize I was allowed to do that, if I could do that, I could potentially come up with some nodes in the level, and rather than bumping into objects, go between nodes, work out the distance between nodes, build up a graph so you know the best way to get between points. Again he was happy, I was describing, or partially describing an actual solution. His next question was but how would I work out where to put the way point nodes, and again I just starting looking at the image, thinking about good positions for the way points, looking at the normals from each wall face, and started to see that putting way points as far away as possible from all normals had some advantages, and bit by bit came up with some sort of solution about how to do this. By the end of it, there was a solution which involved preprocessing the level data offline, when the game runs using this data to move around the level, being able to handle paths becoming blocked (or new paths being opened), strategies for running this on a separate thread or CPU, asynchronous to the main game. It was demonstrating knowledge of the full pipeline for making a game from artist making levels to game running at 60Hz on a PC/console, and also taking into account human resource, i.e. we could do things this way, which would give us a slight edge, but it would make the designers life a lot harder, so I'd probably not do that, take the performance hit, which is small and can probably be won back by the level designerse having extra time to optimize anyway. I was offered the job, ultimately I had a better offer elsewhere so I didn't take it, but for me the test made a lot of sense. My day to day job (Tech Director of a 12 man games development studio) is constantly having to solve problems which initially do not appear to have an answer. The ability to break down an issue, not panic, look from various different perspectives, build up a solution, have a good feeling for what parts of the solution are weak and need further research etc. to me makes a huge difference between an 'ok' programmer/developer and an exceptional one. I haven't used the same test approach, but have certainly learned a lot from it when hiring people, never trying to trick interviewees, quite the opposite, trying to give them as many options as possible, and just urging them show me how they would be able to deal with day to day issues and come out on top. I certainly worry that whilst I got through, some people really would have just walked out, and in some cases you could argue those people wouldn't work well when faced with tough problems and lots of pressure, but on the other hand, but outside of the testing environment they'd be fine. Anyway, I personally wouldn't want to create quite as much stress/pressure if I were to use these technique in future.
eagsalazar2 9y ago
No interviewing system is perfect or even good. Having said that interviewing, in my experience, isn't actually the biggest problem companies have with hiring. The biggest failures are when you have that realization that someone isn't working out where most companies don't have the stomach to deal with the problem decisively through candid feedback/conversation or possibly letting that person go. This natural shyness around confrontation hurts everyone involved including both the employee in question and all their team members.
dorfsmay 9y ago
The "good fit" sent shivers through my spine! I'm a bit of an odd ball in terms of background, and in my early 20's I interviewed for really interesting jobs at really interesting companies (Intel, oracle, Digital, and some more). Every single time the tech staff I would have worked with finished the interviews by saying "You just need one last interview with HR, it will be nothing, looking forward to work with you!". And every single time the answer from HR was that " I would not fit in". After 4 or 5 companies turning me down I thought fuck this, I don't want to fit in, I want to produce good quality work and make customers happy, and started to look exclusively for contracts. Thirty years later, I'm still contracting, every single of my customers has been very happy with my work. Funnily enough, nobody tried to hire me as an employee in those 30 years! And they were right, I'd probably wouldn't fit in. I don't accept status quo, I don't buy in cargo cult, I'm not interested in coming in to wait for the hours to go by. Remember, "fitting in" is not necessarily a good thing. People hire me because I don't.
- marnett 9y ago
  Do you think the amount of contracts available has changed over your career?
  - dorfsmay 9y ago
    It obviously fluctuates with the economy. For example, in 2009, I spent 8 months with not a single chargeable hour. What has changed the most is the nature of the work and being able to work remotely. 30 years ago most people who "ended up" being syadmin were people who coulnd't write software. There was a lot of demand but very few people who were willing to commit to what looked like a "career for failed programer". Nobody would go to university with the hope to become sysadmins. Then sysadmin became a sought after job. People did mix of IT and CS in uni in the hope to become sysadmins. Then it became a commodity. Then it sort of disappeared. These days there is a lot of demand for "DevOps" (which means different things to different people). If you're interested in both the ops part, and the delivering software in the most automated way possible part, there is work out there. Until 5 or 6 years ago, nobody, or at least very few people, were hiring remote workers. It still isn't the majority, but you can find remote contracts if you look really hard for them. To give you an idea, 30 years ago I was the sole sysadmin (for UNIX machines) in a software lab, and was in charge of 5 AIX servers. These days I tend to be part of a small team and working on CI/CD pipelines, controlling resources and security on public and private clouds, help with the architecture of some solutions (CDNs, type of stores (redis vs S3 vs...) etc... You do need to adapt. When I receive calls from recruiters asking if I know anybody who could do the same type of work, I contact my old sysadmin friends who often haven't worked for a few months, some of them who are often younger than me, they say they are not interested in that DevOps and those cloud things, they want to stay sysadmin, and some of them like to blame ageism for their lack of finding work.
chrisbennet 9y ago
"In 1985, Freada Klein (then head of organizational development for Lotus) did an experiment... With Kapor’s permission, Klein pulled together the résumés of the first forty Lotus employees... Klein explained that most of these early employees had skills the growing company needed, but many had done “risky and wacko things” such as being community organizers, being clinical psychologists, living at an ashram, or like Kapor, teaching transcendental meditation. Then Klein did something sneaky. She submitted all forty resumes to the Lotus human resources department. Not one of the forty applicants, including Kapor, was invited for a job interview. The founders had built a world that rejected people like them." http://vault.theleadershiphub.com/blogs/if-you-werent-boss-would-your-company-hire-you http://vault.theleadershiphub.com/blogs/if-you-werent-boss-w...
- blfr 9y ago
  It's unclear whether this was and is a mistake. You need different kind of people in the beginning and when the company is executing a tested business model. We can usefully divide intellectual tasks into two sets: filling and framing. Fillers add more useful detail or content within some framework, while framers explore possible new frameworks. While both tasks are essential, framing has higher variance; most frame attempts fail, but a few produce great value. http://www.overcomingbias.com/2006/12/fillers_neglect.html http://www.overcomingbias.com/2006/12/fillers_neglect.html This definitely causes issues later on when the organization or movement becomes incapable of replacing the original founders. The founders would need to both hire a lot of fillers and also mentor others like them to take over in the future. But this causes enormous resentment. Fillers will feel (absolutely correctly, this is exactly what happens) passed over for promotions that they by every other metric thoroughly deserve.
- jakobegger 9y ago
  Why should a company hire people that are similar to the founders? At an early stage, companies need very different people than later on. At first, they need generalists; once you reach a certain size, you need specialists. The founders should be self-driven, they should have a vision, they should know about all parts of the business. Later employees should be excellent in their speciality, should have execution skills, should be able to follow someone else's vision.
unoti 9y ago
In the original article the author said: > I've never had an interview that tried to evaluate my ability to work in a team or prioritize tasks... I found this interesting, because there are some good ways to look for this that I always ask when interviewing. I wonder if the author had been checked on these things and just didn’t notice. One thing I always do in interviews is after I see the approach the candidate is taking, show them another approach and see if they can understand it and run with it. Many candidates struggle with wrapping their brain around any other approach than their own. This is about collaboration and teamwork. It’s also very common to have behavioral questions that talk through what some challenges were and how you solved them, and to ask probing questions about how the candidate dealt with differing opinions in the best way to proceed. This also is looking at teamwork skills. Regarding prioritizing tasks, it’s routine for me to ask a candidate to break a problem down into components. Then evaluate which are the most difficult, and which they’d tackle first and why. I’m looking to see if they try to reduce risks early in their process. This is something more senior people do better. I wonder if the author has indeed been interviewed about teamwork skills and just didn’t realize it.
conductr 9y ago
I think this is a bit of a slippery slope. In that, if most employers really evaluated what skills they needed over cultural fit and their desire for _The Best_ then, I postulate many of these companies would also realize they have absolutely no reason to be paying super high US/EU salaries, especially the salaries for tech talent in the tech hubs. They would realize that they probably just need a kick ass project management framework and access to a pool of low paid but capable programmers (eg. a remote team in India/China/etc). Most startup tech isn't solving any huge tech problems, it's just building to spec/vision and iterating based on feedback and maybe if you're lucky you get to do this under load/at scale.) I know things have been outsourced with countless examples of bad results but I think it's really just a project management problem - not a skills gap (again for most startup tech we see today, but probably a significant portion of enterprise jobs too). I have more of a bootstrap mindset for most things. So I almost never understand the $1M seed funded startups that get an office and hire a few programmers in SV and instantly only have 6 months of runway just to build a Tinder clone or something with little tech complexity. In my mind, they could easily outsource development and spend 90% of their money on sales. Edit: I say "outsource" a lot above, I actually mean remote teammates that you manage
mratzloff 9y ago
I've accumulated so much material about interviewing I've thought about writing a book. No one would read it, of course, because everyone thinks they're an expert at it already. In my experience, 9 out of 10 interviewers are terrible. They come up with assumptions before talking to a candidate, then come up with even more assumptions during the interview. These assumptions are almost always unchecked, meaning that they never ask a single question to verify their guesswork. Perhaps the most critical failure, which this article touches on, is that people can't adequately describe why their hiring successes were successes and why their hiring failures were failures. If they did, they'd understand just how many of their interview process (such as it is) is a crapshoot. They have no conscious understanding of what it takes to be successful at their company, because they've never thought about it in any serious way. They ask for code samples, but never look at them. What's really important is that you know all of their current technologies and use them in exactly the way that they do currently. So to determine that they sometimes give out poorly-conceived coding assignments that are either testing something completely irrelevant or are so large in scope that you are basically rebuilding half their product. They ghost candidates, even candidates that have gone through multiple rounds of interviews, forgetting that people who interact with their company go out into the world and form opinions and things to say about them. This is especially true of candidates who are really engaged with your company's mission. (Besides, it's just rude.) The best predictor of future behavior is past behavior, except when being interviewed, where anything you accomplished at a previous company is viewed with the detached impatience of someone eager for you to stop talking so long so they can get to their questions. Listening to what people actually say, instead of just your impression of what they say, means listening to the motives, emotions, and body language surrounding what they're saying and asking questions that get at the heart of those. It's the single most effective interviewing method, but almost no one does it. Because I hire others, when I am being interviewed, a popular question is "How do I hire?" Everyone is looking for a solution with the appearance of a rigorous methodology. If I wrote that book, I'd have to sell a snake oil solution that guarantees great hires, because no one wants to hear the reality: that it requires experience, empathy, and active listening, and there's no one-size-fits-all solution for each person. People don't like nuance; they want a simple, step-by-step guide, not some amorphous process that might take a lot of self-work.
ksk 9y ago
These posts always read to me as being a bit presumptuous. The fact that you're looking for a job at that particular company means they're doing _something_ right, with regards to hiring, or shipping quality products, or having a good work culture, etc, etc. If I was the interviewer, I would never hire a person with this sort of "know it all" attitude. If you think you can be a better interviewer, then you have to show you are, rather than coming up with a logical argument of "doing it this way makes sense". Basically, unless you yourself are in that position, or have intimate knowledge of it, you don't know why someone made those decisions.
marcoperaza 9y ago
I disagree with the reasoning here. Just because the interviewer says 1 or 2, doesn't mean that they're not actually or also looking for 3. We humans tend to be imprecise in understanding and stating our own motivations.
mrlyc 9y ago
When interviewing potential employers, I ask them 1. Do you conduct regular peer reviews of your documentation and code? 2. Do you have a bug tracking system?
sailorman 9y ago
It obviously fluctuates with the economy.

307 points by deafcalculus 9y ago

>If you want to make a change to your interview process, give it to some of your current employees first. If anyone fails it, ask yourself if that person should be fired. The answer is probably no. That's a fascinating idea that's never crossed my mind. E.g., how many current employees can pass whiteboard coding exams?

FeatureRush 9y ago
Apparently there are already companies that take it a step further. They test sample of their employees with some fancy IQ test that measures them across several dimensions. From those results profile of desirable new hire is build and all candidates get measured by the same test. On reddit in threads about interviews I've already seen couple people discussing those strange tests where in many questions there are no obvious correct/wrong answers, like for example all presented shapes can in their own way fit in presented pattern.
- eklavya 9y ago
  If there is no obvious right/wrong answer, I wonder if they are limiting their pool taking people who all think alike (kinda like in-breeding) or are they using it diversify the ideas and knowledge etc. (like diversifying gene pool).
  - FeatureRush 9y ago
    Different answers probably give points to different traits the test measures, the ambiguity is just irritating for test takers coming from more classical tests where you know (or at least suspect) what is expected from you. This whole test setup is not cheap so one can at least hope they considered the issue of diversity? The other thing is did they actually tested which teams perform better in specific problem domain that interests them?
- ataturk 9y ago
  Ugh, that sounds so Orwellian. Corporate Eugenics.
pavlov 9y ago
Many companies instruct candidates should study for interviews with specific books or courses (e.g. “Cracking the Code Interview”). That indicates the idea is not that someone who is a competent professional should pass without studying. The passively hostile whiteboard coding exams are more like a hazing ritual than an actual job performance test.
- watwut 9y ago
  I prefer that over being ambushed by unexpected topics. Here, you measure ability to learn, at minimum. Unfortunately, you measure also how much free time candidate has. Which is still better then having candidate to guess what is popular now or which if competing preferences your company has.
tomseldon 9y ago
Testing the change is one thing, but if the original people were hired and asked something very similar, you have an existing group selected based on that characteristic and so will likely do well. e.g. if you have always done a whiteboard interview, then you've selected for people that do well in that type of scenario. Changing the question but keeping the format will likely just prove that those people are still suited to that type of task, but not that an otherwise good hire will be filtered out. I'm not against whiteboard interviews or similar, but just pointing out that selection bias is an important factor here if you're judging effectiveness by comparing it to results for existing employees.
ssijak 9y ago
Interviewing is a random event for many companies. You as a candidate could fail miserably on one day and get great feedback the other day. It depends on so many factors in larger companies. For example in one of my previous employers, they would send a random engineer (juniors included) to interview a candidate and give a mark from 1-10. There were engineers who would not pass their own interview and questions, and on the other hand, there were others who would just see that you can code anything and you are cool to work with and would let you in. And anything in between. There were times where juniors wanted to be smart and asked some puzzles they got from the net, and when later I asked them to solve them they failed. So many interviews depend on so many random events and circumstances and. The biggest takeaway I got from all this experience is that as a candidate I should not be intimidated by anything and should just be cool and myself, because there is so much randomness that trying to find reason in it will just make things worse. And ALWAYS negotiate as much as you can because your biggest raise is when you get hired.
- ataturk 9y ago
  What you are describing is a phenomenon referred to as "that's just life." It's just random shit--one day you interview the team there could be going through holy hell trying to solve a problem under a deadline. Another time, they're in a lull and screwing around on Slack and in a good mood.
BeetleB 9y ago
>That's a fascinating idea that's never crossed my mind. E.g., how many current employees can pass whiteboard coding exams? I once confronted a colleague who had refused a candidate because he couldn't answer a technical question. When I heard the question, I asked him "How many people in the current team can answer this?" (Hint: Only one). His response: "Yeah, but we told him to read the first 6 chapters of textbook X!" So we denied him a job because he didn't do his homework.

pjc50 9y ago

Note that if your company is small, it can be very hard to do any kind of meaningful statistical process control on hiring. The "track careers of people you rejected" idea is an interesting one, but depends very much on how public people make their careers. I suppose you can rely on LinkedIn for 90% of people. (What is HN's opinion of the Stack Overflow Developer Story?)

richmarr 9y ago
We run an early stage applicant tracking system and see ourselves broadly as a rejection engine. Something like 95% of the activity on our system relates to candidates who end up being rejected. Since our assessment data is structured our current iteration of the product uses that to create value for rejected candidates, informing them where they were strong or weak in comparison to the rest of the group, and showing them some aggregate scoring. It goes down very well with hiring managers, who typically don't have the time to respond in detail or at all to rejected candidates, and (for the most part) goes down very well with candidates.
ec109685 9y ago
Very good point. A nice blog post illustrating the problem with drawing conclusions on small sample sizes: https://jvns.ca/blog/2014/07/11/fun-with-stats-how-big-of-a-sample-size-do-i-need/ https://jvns.ca/blog/2014/07/11/fun-with-stats-how-big-of-a-... You need a lot more than a startup’s number of employees and their eventual job reviews to make strong conclusions on the quality of the interview process, and need to rely more on industry best practices instead.
speby 9y ago
>The "track careers of people you rejected" idea is an interesting one, but depends very much on how public people make their careers. I really like this idea. This reminds me a bit of the "anti-portfolio" that some VCs track (aka a bittersweet regret of the unicorns they passed on years ago). I think the same could be said of great talent that an organization passed on that then goes on to build great products, invent, and/or otherwise lead great teams despite that. However, employment is a tad different than investment so, to me, the only caveat to this is that it simply isn't the case that every super talented person will bring about great results and value creation at every single organization. Much of what a talented person's output depends on is the very room full of other people in which that person works with. And while "fit" is such a cliche thing these days, this is what matters so much in terms of whether or not an otherwise talented person will be successful or not within one organization versus another.

mlashcorp 9y ago

I always interview for "culture fit". In my experience, that's almost always the bottleneck, and the technical part is rarely a problem. Learning a new skill is not an earth-shattering problem if you have a good attitude towards work, and good interpersonal skills are a must in teams

richmarr 9y ago
That prompts two questions for me if you don't mind sharing. (a) How do you define and assess "culture fit", and why? (b) How do you know the things you're optimising for are actually good proxies for productivity? Appreciate these aren't questions with simple answers (or maybe even answers at all), but hiring/assessment process is my current field and I'm always curious where peoples thinking is at.
- Spooky23 9y ago
  That’s a fancy way of saying “we hire people we like”. It’s an approach that isn’t perfect but is good enough for many roles. We pick spouses based on that approach and it works out pretty well for most people.
  - mlashcorp 9y ago
    You are correct, "culture fit" is not a good descriptor of what I meant. We hire people we believe we will be able to work with. It's not a popularity contest, we evaluate problem solving, analytical thinking and communication skills, such as the ability to explain complex topics in easy terms. I'll give you an example, usually we are interviewing people straight out of the university, but this can be applied to any candidate. I'll start by asking the candidate to talk about his master's project. As he talks about it, I'll guide the conversation towards these topics : -What was the problem -How was it defined? -How did you tackle it? -Explain your solution -support your decisions -Was there a cost/benefit analysis? -How was the problem decomposed? This will give me an initial picture of the candidate's ability to address a problem, and is far more important to me than wether he knows "the number after F in hexadecimal" or some other nonsense.

dvt 9y ago

> I've heard "Hmm, I don't actually know what it is that we're looking for" from a few recruiters Why do you even care what recruiters say? They're there to make as many hires as possible and get their commission. Don't get me wrong, this is actually a decent question, but you might as well ask a car salesman what he likes doing in his free time. It doesn't matter -- he's trying to sell you a car. > How do you evaluate how well you're meeting your goal? -- The majority of the companies that I ask this to essentially answer "we don't" to this question. I'm going to go ahead and call BS on this one. Literally every company (outside of early-stage startups) has some kind of performance review. Most of these are completely artificial and completely suck, but they're definitely there.

tehwalrus 9y ago
Performance reviews and their flaws are discussed in detail later in the article. In particular, they detect false positives but not false negatives.
ndh2 9y ago
The point was the performance reviews aren't used to evaluate or iterate on the hiring process.

nmalaguti 9y ago

Each time I read a post like this I say to myself “they’re not wrong, but something is also missing.” If there was a way to evaluate a candidate based on how well they would do at their actual job, in an hour or less, that couldn’t be games or cheated, we’d all be using it. That isn’t to say we shouldn’t work on improving the hiring process, but how much time, both from the perspective of the candidate and the company, is too much time to invest in a potential hire? Would we get better signal if you came to work for us for a week and paired with everyone on the team? Sure! But how many candidates can take a week off from their current job? The author talked to 11 companies. Could they invest 11+ weeks in their job search full time in order to find their next role? What about the loss in productivity? Can you have more than one candidate in the office in a given week? What if your ramp up time for full time employees is actually longer than a week? Are you biasing for people who ramp up fast instead of people who will be ultimately more productive and impactful? I often feel that engineers write these posts because they know that they themselves are good at their job, but feel it is silly to have to develop an alternative set of skills in order to signal that they are a good hire. Or they get rejected and blame the process or those alternative skills. I’ve felt the same way and wanted the system to be better tailored for my skill set, but balanced against the time investment for some alternatives, I see why we have the process we do.

rlpb 9y ago
> I often feel that engineers write these posts because they know that they themselves are good at their job... The Dunning-Kruger effect suggests that almost certainly, some non-zero proportion engineers who have written this type of posts are not. https://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect https://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect
- custos 9y ago
  While what you're saying is true, I think the dunning kruger effect has an inverse bell curve based on level of confidence vs knowledge. High confidence low knowledge. Moderate confidance moderate knowledge High confidence high knowledge. Of course that only applies to the things the expert knows like the back of his hand. There are things I am 100% confident I know, because I've done it thousands of times. Then there are some advanced things I still question if I truly understand because I've only successfully done it a few times. Of course when I was younger, the things I'm 100% confident now I went through the phase of "oh I can do this super easy" to "holy crap this is harder than I thought" to "I've done this 1000's of times, I got this"

dandare 9y ago

His point, that hiring process is not a scientific process, is probably correct. But the same is true about performance review process (in my opinion). At best, the performance review is unscientific and biased, at worst it is a popularity contest, while usually, it is a formality. > I've never had an interview that tried to evaluate my ability to work in a team or prioritize tasks - both of which are probably more important to doing well in a software engineering job than being able to find anagrams in O(n) time. And I have never seen a test that could reliably evaluate your "ability to work in a team or prioritize tasks". The hiring process is usually unscientific because making it reliably scientific is very expensive.

ataturk 9y ago
Performance reviews are designed to be gamed by managers. I have never seen one that a) wasn't gamed and b) had any scientific basis whatsoever.
rafiki6 9y ago
Yes, but what about the commonly trotted out line of "hiring mistakes are expensive so we'd prefer to not hire than make a mistake". Perfect example of having cake and eating it...You have to eat the cost somewhere. Either it's in the hiring process, the firing of a bad employee, or spinning your wheels waiting for the right candidate.

kreetx 9y ago

A side note, but the latter two answers (2. being a good fit and 3. doing a good job) sound very much the same to me.

0xffff2 9y ago
I think "fit" is referring to ability to get along with the rest of the team, regardless of ability to do the job. They aren't quite opposites, but they are mostly unrelated.
- kreetx 9y ago
  It can mean that too, yes, but if the recruiter word-by-word says 2. then it could also mean 3. But then the rest the rest of the argument quite some weight since 3. is not rare anymore, as it's included in 2., too.

kevinmobrien 9y ago

Interviewing is subjective. It’s about sitting down in a specific context and extrapolating, based on incomplete information, about a future set of conditions that will likely change when time catches up to them. If we can start with the idea that interviews are a very blunt instrument, the process becomes at least somewhat compressible: it selects for people who recognize the parameters and perform well within that context. At best, that pattern-matches to success. At worst, it filters out talented folks who don’t interview well, but could code well.

lucideer 9y ago

> less than 10% of the companies that I've interviewed with have said that their interview process was designed to evaluate how effectively someone could do the job that they're being hired for. I don't think an interview process can do this tbh. A good interview process can evaluate if they can do the job that they're being hired for, in most cases, but I would say only an actual trial/probationary period can evaluate how effectively they can do it. And even then, how effective they are may be environmental - completely external to their own internal competence. After concluding that a candidate can do the job, the secondary purpose of the interview process is filtering for people that will positively affect the work environment of their colleagues (and/or filter out those who may negatively affect others). As an interviewer, I would make some effort not to lean too heavily on my own biases and try and also consider how they'd get on with my colleagues that I know well, but there is unfortunately always going to be some implicit bias here.

derekmhewitt 9y ago

Just as a spitball idea... What about a structured "cohort" hiring process where you give potential employees short term (literally one or two days, where you have a group exercise like a "hackathon" where you make a sample product for demo, then moving up to one or two week rounds) contracts that allow you to evaluate and accept (pass on to future rounds) or reject a large candidate pool and thereby filter your potential future employees. Given the current costs of recruiters, HR personnel, interviews and headhunters this might actually end up being cheaper than current "traditional" hiring methods and could generate some very effective development teams.

scrumper 9y ago
That means you can only hire people who don't currently have a job.

dilemma 9y ago

Using proxies for competence is a slippery slope towards misdirection and irrelevance - and as a result, incorrect hiring decisions. In another thread today, someone mentioned they reject every CV that has two typos or more. It's a proxy, and it is irrelevant for the job. Once you start going down the path of proxies, you add more and more far removed ones. To keep hiring performance high, resist adding the first proxy. Actually ask questions relevant to past work experience that maps to the actual job. Maybe do work samples. To circle back to the two questions in the blog post, no interviewer can answer them because they use proxies too far removed.

richmarr 9y ago
Agreed. Work samples are the most predictive form of assessment, you just need to be able to isolate them from other factors such as confidence bias... i.e. no whiteboard interviews (unless hiring for someone whose job it will be to solve problems on whiteboards in front of strangers in situations that effect their financial security and sense of worth)
logfromblammo 9y ago
Think of the resume as a deliverable. This potential employee sent out the deliverable without reviewing it for spelling and grammatical correctness. They almost certainly did not ask someone else to proofread it and suggest corrections. Would you want to put this person in a position where they may have to deliver binaries, code, or other documents to a customer? Would they ignore or skip steps in your delivery process, too?

arkadiytehgraet 9y ago

As an interviewee, one of the questions (that I shamelessly stole from somewhere on HN) I like to ask recruiters is this: "Do you like working here?" The answers and the way the answers are given usually tell you quite a lot about the company and the specific place you are going to work.

comstock 9y ago
It’s a fair question, but if you asked me that in front of my boss or coworkers I would likely immediately take a disliking to you because you put me in an awkward situation where there are few easy ways to answer honestly.
- arkadiytehgraet 9y ago
  This is why this question is so good: you either like working there, and then you have no problem answering this question honestly (and without any awkwardness), or you don't like, then surely you have to come up with some elaborate yet unclear answer to this question, which is quite clear at that point, and then I do not really care whether you take a disliking to me or not, since it does not really matter any more.
  - comstock 9y ago
    Maybe in general that’s true. In my case even if I feel it’s a nice place to work, I wouldn’t like being on of the spot in front of my boss/coworkers. In a one-to-one interview it would be fine. Which was perhaps more the context of your comment.
    - burger_moon 9y ago
      Not being able to speak freely in front of your peers would be a hard pass from me. I'm not going to work somewhere I can't feel comfortable speaking honestly.
      - 9y ago

deleted 9y ago

[deleted]

comstock 9y ago

> If you want to make a change to your interview process, give it to some of your current employees I don’t think I would respond well to this as an employee.

dx034 9y ago
Why not? I think it's a valid test for a company. If the majority of your employees wouldn't pass the hiring test, why use it for new employees (unless you're looking for someone more senior)? Employees can probably best give feedback on how to adapt it so that it mirrors the actual work.
- comstock 9y ago
  Many people seem comfortable allowing their companies to treat them as low value commodities and will do anything asked of them. In general I don’t exactly think like this. I do the work within the scope of what I’ve been hired to do. I’m lucky, at least right now, that I have this choice because my skills are somewhat rare. Want work outside the scope of what I’ve been hired to do? Pay for it, or find someone else. I don’t want to be asked to interview again, it’s not a pleasant experience for me, and critically it’s being presented as “something you have to do”. If you want to do this, offer additional compensation for doing something outside of the scope of my work.
  - dx034 9y ago
    I think a good work environment (and that includes competent coworkers) is in everyone's interest. What if you can take the test but the employer doesn't know the result? I'd trust my employer enough not to use interview questions as my performance evaluation. If you don't, the level of trust appears to be very low.
  - BeetleB 9y ago
    >Many people seem comfortable allowing their companies to treat them as low value commodities and will do anything asked of them. If that's how you feel about it, would you not agree that asking such questions in an interview is treating the candidate as "low value"?

notacoward 9y ago

He almost gets it right. Hiring for some abstract "best" isn't so great. Hiring for "culture fit" is practically an invitation to discriminate. OTOH, people rarely do only one thing at a company, so it's important to remember that you're hiring for a job rather than a specific task. You need to evaluate fundamental skills, including collaboration or leadership skills (and styles), not just knowledge of specific subject matter.

throw999888away 9y ago
I have mixed feelings on culture fit becoming a bad word. I don't disagree with your point but I think there is a danger with ignoring its value. For example --leaving aside your views on SV startup sweatshops-- if you're in a start up, you will at some point work late hours and you will at some point go through stressful times. Sometimes the things that get you through those times is that feeling of camaraderie and actually getting on with your colleagues. Screening on culture-fit is a good way to guard for that, and at least for me it is unwise to not associate value with that. Now, it could be that your culture is inclusivity. But then isn't hiring for people who fit that mould still hiring for culture fit? Isn't that still creating a monoculture? Isn't discriminating against a bigot brogrammer (who may well be great at coding) still discrimination?
- notacoward 9y ago
  The problem is that "culture fit" is too broad and too vague. Often it includes things that are completely unrelated to job performance. If you want to make sure people are able/willing to work long hours, ask about that (and be aware that you're probably discriminating to some degree against people with families or health issues). If you want to know how someone will handle stress, ask about that. Not only will you be measuring actual job-related skills or qualities, but you'll also be telling the candidate something they should know about the work environment. On the other hand, hobbies and musical tastes and political views should not be part of a hiring process, and that's the kind of thing "culture fit" often seems to include.
  - 9y ago

johnpython 9y ago

If there is a gross lack of diversity within your company and you routinely use "culture fit" as an excuse to turn down PoC candidates, what conclusion can be drawn about your organization?

austincheney 9y ago

My goals as an interviewer: 1. Do you have a solid understanding of the foundations for your skill/discipline? 2. If you were given $100,000 VC capital and told to build a revolutionary new thing how would you build it? If a candidate cannot adequately address the first objective then I will find another candidate who can. This is pure and simple elimination with simple non-biased questions on things the candidate should know to do their job without abstractions, tooling, and other trendy bullshit. The second objective is where I can evaluate my bias against the candidate's, but only after the candidate has passed the technical evaluation in a non-biased manner.

qaq 9y ago
No offense but if candidate has a credible answer to "2. If you were given $100,000 VC capital and told to build a revolutionary new thing how would you build it?" wtf would she/he work for you :)?
- Can_Not 9y ago
  It's because he wants to focus on building the thing, not focus on acquiring VC funding.
  - marcosdumay 9y ago
    You are really asking to be bullshited (even if people had a good answer, they won't have an elevator pitch to tell you in an interview). Unless you are looking for people that can make an argument, you are selecting for the wrong trait.
    - austincheney 9y ago
      > You are really asking to be bullshited The question is rhetorical, so what does this matter? It is part of a job interview. It is not a VC pitch. If this confuses a candidate and they answer a question not asked I would not hire them. Interpreting and following instructions are important skills.

listentojohan 9y ago

One of the good interview experiences I had, included going through a project I had previously done, framing the problem, how I'd have worked through it (with class/function level whiteboarding), and what could be improved. It was a project from a previous job, but too high level to be replicated. But could see how going through work from a previous job could be problematic (no code was shown).

jasonmaydie 9y ago

"Best people" and "good fit" means they are looking for someone with both technical and soft skills. They don't want a skilled worker who is difficult to manage or communicate with, and they don't want a nice guy who is not skilled. It makes sense to me.

j_s 9y ago

I would like to see YC take over the initial chunk of the hiring process for their companies, and publically document the heck out of how to do it best. Posts like this one demonstrate that free access to the front page of HN for job listings isn't enough: Don't work for CrateJoy | https://news.ycombinator.com/item?id=15486301 https://news.ycombinator.com/item?id=15486301

cubano 9y ago

There are so many cognitive biases in play when it comes to hiring its basically a total crapshoot no matter what processes and shit is in place. People hire people they like, period.

rafiki6 9y ago
Disagree, we can basically eliminate the personal part of it. Blind interviewing. I don't need to know name/gender/school etc. Technical skills test as a filter given to current employees to track if it actually tests for necessary skills. Second part of interviewing can be assessed by people who don't see the candidate and we can obscure their voices.
- mratzloff 9y ago
  The cognitive biases aren't just around appearance or voice...
  - rafiki6 9y ago
    Not sure what else there can be if you create a normalized test that has been designed to predict success by being applied to your current employee population. Take away personal interviewing and voila. I can't judge this person on my inherent bias towards a race, gender, etc. if I can't see them or determine where they are from. The test should be the only bar required to grant a person an opportunity, nothing else. No need for resumes. https://hbr.org/2016/04/how-to-take-the-bias-out-of-interviews https://hbr.org/2016/04/how-to-take-the-bias-out-of-intervie...

scraft 9y ago

The best interview I had (for a lead games programmer position) was a written test, I can't remember the exact details but perhaps 24 pages and 45 minutes to complete it. Before starting I flicked through the entire test to get a feel for what was being asked of me, and then for a little while debated just getting up and leaving the room. It felt a bit crazy, a test on all sorts of disciplines of games programming that I wasn't an expert in, and I didn't intend to become an expert in (and the job I was applying for didn't require me to be an expert in). But I calmed down, tackled the test in the best way I could, answered the questions in an order where my most confident answers (and quick to complete) answers were done first, then moving onto ones I had a pretty good idea of, then ones I knew less well, etc. 40 minutes in, the CEO of the company came in, it is a ~300 man company, so it was interested in itself that he came in. Anyway, he wanted to go through the test, I said I hadn't finished yet, and had made a note of the start time, and still had 5 minutes remaining. He said it doesn't matter, lets have a look. And he flicked through the test and didn't look at my answers and instead found a section of the test which I had completely ignored (it was to do with AI and path finding, both topics I have done almost nothing on during my entire education and career). Of course, this was the question he wanted me to answer, I explained the reasons I hadn't answered it, and said if they wanted me to do things like this it really wouldn't be a suitable job for me. But he persisted, stop worrying about all of these things, just answer the question now. Again I had that slight feeling of wanting to just leave, but again I overcame in. I started talking him through the things I did know about the question, pointing out areas which could cause problems, then started listing what I could do to limit the question to avoid some of these issues (it had a picture of a top down level, the question was inside a box, I said lets initially forget about the box for example). Then I started just talking about an initial algorithm which could route AI around the level - I said it would obviously be really bad (basically AI just walking into things, then working out where to go next, and then a bit later saying oh I could keep track of where I have been in case I end up at the same point, etc.) we talked more about it, he asked some questions, and bit by bit I came up with a solution. He said that's interesting, because during this process you have described parts of various algorithms which someone who has studied AI would know about, but you are going from a brute force perspective, not having any way points in the level, no extra level knowledge. I said I didn't realize I was allowed to do that, if I could do that, I could potentially come up with some nodes in the level, and rather than bumping into objects, go between nodes, work out the distance between nodes, build up a graph so you know the best way to get between points. Again he was happy, I was describing, or partially describing an actual solution. His next question was but how would I work out where to put the way point nodes, and again I just starting looking at the image, thinking about good positions for the way points, looking at the normals from each wall face, and started to see that putting way points as far away as possible from all normals had some advantages, and bit by bit came up with some sort of solution about how to do this. By the end of it, there was a solution which involved preprocessing the level data offline, when the game runs using this data to move around the level, being able to handle paths becoming blocked (or new paths being opened), strategies for running this on a separate thread or CPU, asynchronous to the main game. It was demonstrating knowledge of the full pipeline for making a game from artist making levels to game running at 60Hz on a PC/console, and also taking into account human resource, i.e. we could do things this way, which would give us a slight edge, but it would make the designers life a lot harder, so I'd probably not do that, take the performance hit, which is small and can probably be won back by the level designerse having extra time to optimize anyway. I was offered the job, ultimately I had a better offer elsewhere so I didn't take it, but for me the test made a lot of sense. My day to day job (Tech Director of a 12 man games development studio) is constantly having to solve problems which initially do not appear to have an answer. The ability to break down an issue, not panic, look from various different perspectives, build up a solution, have a good feeling for what parts of the solution are weak and need further research etc. to me makes a huge difference between an 'ok' programmer/developer and an exceptional one. I haven't used the same test approach, but have certainly learned a lot from it when hiring people, never trying to trick interviewees, quite the opposite, trying to give them as many options as possible, and just urging them show me how they would be able to deal with day to day issues and come out on top. I certainly worry that whilst I got through, some people really would have just walked out, and in some cases you could argue those people wouldn't work well when faced with tough problems and lots of pressure, but on the other hand, but outside of the testing environment they'd be fine. Anyway, I personally wouldn't want to create quite as much stress/pressure if I were to use these technique in future.

eagsalazar2 9y ago

No interviewing system is perfect or even good. Having said that interviewing, in my experience, isn't actually the biggest problem companies have with hiring. The biggest failures are when you have that realization that someone isn't working out where most companies don't have the stomach to deal with the problem decisively through candid feedback/conversation or possibly letting that person go. This natural shyness around confrontation hurts everyone involved including both the employee in question and all their team members.

dorfsmay 9y ago

The "good fit" sent shivers through my spine! I'm a bit of an odd ball in terms of background, and in my early 20's I interviewed for really interesting jobs at really interesting companies (Intel, oracle, Digital, and some more). Every single time the tech staff I would have worked with finished the interviews by saying "You just need one last interview with HR, it will be nothing, looking forward to work with you!". And every single time the answer from HR was that " I would not fit in". After 4 or 5 companies turning me down I thought fuck this, I don't want to fit in, I want to produce good quality work and make customers happy, and started to look exclusively for contracts. Thirty years later, I'm still contracting, every single of my customers has been very happy with my work. Funnily enough, nobody tried to hire me as an employee in those 30 years! And they were right, I'd probably wouldn't fit in. I don't accept status quo, I don't buy in cargo cult, I'm not interested in coming in to wait for the hours to go by. Remember, "fitting in" is not necessarily a good thing. People hire me because I don't.

marnett 9y ago
Do you think the amount of contracts available has changed over your career?
- dorfsmay 9y ago
  It obviously fluctuates with the economy. For example, in 2009, I spent 8 months with not a single chargeable hour. What has changed the most is the nature of the work and being able to work remotely. 30 years ago most people who "ended up" being syadmin were people who coulnd't write software. There was a lot of demand but very few people who were willing to commit to what looked like a "career for failed programer". Nobody would go to university with the hope to become sysadmins. Then sysadmin became a sought after job. People did mix of IT and CS in uni in the hope to become sysadmins. Then it became a commodity. Then it sort of disappeared. These days there is a lot of demand for "DevOps" (which means different things to different people). If you're interested in both the ops part, and the delivering software in the most automated way possible part, there is work out there. Until 5 or 6 years ago, nobody, or at least very few people, were hiring remote workers. It still isn't the majority, but you can find remote contracts if you look really hard for them. To give you an idea, 30 years ago I was the sole sysadmin (for UNIX machines) in a software lab, and was in charge of 5 AIX servers. These days I tend to be part of a small team and working on CI/CD pipelines, controlling resources and security on public and private clouds, help with the architecture of some solutions (CDNs, type of stores (redis vs S3 vs...) etc... You do need to adapt. When I receive calls from recruiters asking if I know anybody who could do the same type of work, I contact my old sysadmin friends who often haven't worked for a few months, some of them who are often younger than me, they say they are not interested in that DevOps and those cloud things, they want to stay sysadmin, and some of them like to blame ageism for their lack of finding work.

chrisbennet 9y ago

"In 1985, Freada Klein (then head of organizational development for Lotus) did an experiment... With Kapor’s permission, Klein pulled together the résumés of the first forty Lotus employees... Klein explained that most of these early employees had skills the growing company needed, but many had done “risky and wacko things” such as being community organizers, being clinical psychologists, living at an ashram, or like Kapor, teaching transcendental meditation. Then Klein did something sneaky. She submitted all forty resumes to the Lotus human resources department. Not one of the forty applicants, including Kapor, was invited for a job interview. The founders had built a world that rejected people like them." http://vault.theleadershiphub.com/blogs/if-you-werent-boss-would-your-company-hire-you http://vault.theleadershiphub.com/blogs/if-you-werent-boss-w...

blfr 9y ago
It's unclear whether this was and is a mistake. You need different kind of people in the beginning and when the company is executing a tested business model. We can usefully divide intellectual tasks into two sets: filling and framing. Fillers add more useful detail or content within some framework, while framers explore possible new frameworks. While both tasks are essential, framing has higher variance; most frame attempts fail, but a few produce great value. http://www.overcomingbias.com/2006/12/fillers_neglect.html http://www.overcomingbias.com/2006/12/fillers_neglect.html This definitely causes issues later on when the organization or movement becomes incapable of replacing the original founders. The founders would need to both hire a lot of fillers and also mentor others like them to take over in the future. But this causes enormous resentment. Fillers will feel (absolutely correctly, this is exactly what happens) passed over for promotions that they by every other metric thoroughly deserve.
jakobegger 9y ago
Why should a company hire people that are similar to the founders? At an early stage, companies need very different people than later on. At first, they need generalists; once you reach a certain size, you need specialists. The founders should be self-driven, they should have a vision, they should know about all parts of the business. Later employees should be excellent in their speciality, should have execution skills, should be able to follow someone else's vision.

unoti 9y ago

In the original article the author said: > I've never had an interview that tried to evaluate my ability to work in a team or prioritize tasks... I found this interesting, because there are some good ways to look for this that I always ask when interviewing. I wonder if the author had been checked on these things and just didn’t notice. One thing I always do in interviews is after I see the approach the candidate is taking, show them another approach and see if they can understand it and run with it. Many candidates struggle with wrapping their brain around any other approach than their own. This is about collaboration and teamwork. It’s also very common to have behavioral questions that talk through what some challenges were and how you solved them, and to ask probing questions about how the candidate dealt with differing opinions in the best way to proceed. This also is looking at teamwork skills. Regarding prioritizing tasks, it’s routine for me to ask a candidate to break a problem down into components. Then evaluate which are the most difficult, and which they’d tackle first and why. I’m looking to see if they try to reduce risks early in their process. This is something more senior people do better. I wonder if the author has indeed been interviewed about teamwork skills and just didn’t realize it.

conductr 9y ago

I think this is a bit of a slippery slope. In that, if most employers really evaluated what skills they needed over cultural fit and their desire for _The Best_ then, I postulate many of these companies would also realize they have absolutely no reason to be paying super high US/EU salaries, especially the salaries for tech talent in the tech hubs. They would realize that they probably just need a kick ass project management framework and access to a pool of low paid but capable programmers (eg. a remote team in India/China/etc). Most startup tech isn't solving any huge tech problems, it's just building to spec/vision and iterating based on feedback and maybe if you're lucky you get to do this under load/at scale.) I know things have been outsourced with countless examples of bad results but I think it's really just a project management problem - not a skills gap (again for most startup tech we see today, but probably a significant portion of enterprise jobs too). I have more of a bootstrap mindset for most things. So I almost never understand the $1M seed funded startups that get an office and hire a few programmers in SV and instantly only have 6 months of runway just to build a Tinder clone or something with little tech complexity. In my mind, they could easily outsource development and spend 90% of their money on sales. Edit: I say "outsource" a lot above, I actually mean remote teammates that you manage

mratzloff 9y ago

I've accumulated so much material about interviewing I've thought about writing a book. No one would read it, of course, because everyone thinks they're an expert at it already. In my experience, 9 out of 10 interviewers are terrible. They come up with assumptions before talking to a candidate, then come up with even more assumptions during the interview. These assumptions are almost always unchecked, meaning that they never ask a single question to verify their guesswork. Perhaps the most critical failure, which this article touches on, is that people can't adequately describe why their hiring successes were successes and why their hiring failures were failures. If they did, they'd understand just how many of their interview process (such as it is) is a crapshoot. They have no conscious understanding of what it takes to be successful at their company, because they've never thought about it in any serious way. They ask for code samples, but never look at them. What's really important is that you know all of their current technologies and use them in exactly the way that they do currently. So to determine that they sometimes give out poorly-conceived coding assignments that are either testing something completely irrelevant or are so large in scope that you are basically rebuilding half their product. They ghost candidates, even candidates that have gone through multiple rounds of interviews, forgetting that people who interact with their company go out into the world and form opinions and things to say about them. This is especially true of candidates who are really engaged with your company's mission. (Besides, it's just rude.) The best predictor of future behavior is past behavior, except when being interviewed, where anything you accomplished at a previous company is viewed with the detached impatience of someone eager for you to stop talking so long so they can get to their questions. Listening to what people actually say, instead of just your impression of what they say, means listening to the motives, emotions, and body language surrounding what they're saying and asking questions that get at the heart of those. It's the single most effective interviewing method, but almost no one does it. Because I hire others, when I am being interviewed, a popular question is "How do I hire?" Everyone is looking for a solution with the appearance of a rigorous methodology. If I wrote that book, I'd have to sell a snake oil solution that guarantees great hires, because no one wants to hear the reality: that it requires experience, empathy, and active listening, and there's no one-size-fits-all solution for each person. People don't like nuance; they want a simple, step-by-step guide, not some amorphous process that might take a lot of self-work.

ksk 9y ago

These posts always read to me as being a bit presumptuous. The fact that you're looking for a job at that particular company means they're doing _something_ right, with regards to hiring, or shipping quality products, or having a good work culture, etc, etc. If I was the interviewer, I would never hire a person with this sort of "know it all" attitude. If you think you can be a better interviewer, then you have to show you are, rather than coming up with a logical argument of "doing it this way makes sense". Basically, unless you yourself are in that position, or have intimate knowledge of it, you don't know why someone made those decisions.

marcoperaza 9y ago

I disagree with the reasoning here. Just because the interviewer says 1 or 2, doesn't mean that they're not actually or also looking for 3. We humans tend to be imprecise in understanding and stating our own motivations.

mrlyc 9y ago

When interviewing potential employers, I ask them 1. Do you conduct regular peer reviews of your documentation and code? 2. Do you have a bug tracking system?

sailorman 9y ago

It obviously fluctuates with the economy.