15 ms·

What happens when you stop relying on resumes

448 points by appamatto 11y ago

appamatto 11y ago
I think this is very interesting counterpoint to TripleByte's data which implied that talking about passion projects was lower signal than their coding quizzes. The benchmark was performance in a long form coding interview for TripleByte, whereas Aline's is the final offer, so not exactly apples to apples.
- leeny 11y ago
  Aline here. Fwiw, I don't disagree with TripleByte's findings... anecdotally, after having interviewed somewhere between 500 and 1000 people in my career, I think they're right -- I, too, have observed a disconnect between how polished people sound when they talk about their work and what happens when they actually have to write code. What we're actually comparing here, though, isn't coding vs. describing projects. It's describing projects vs. resumes. And I expect that, there, resumes provide a lower signal.
  - Kluny 11y ago
    Hey Aline, thanks for the article. I've subscribed to your RSS because your posts are so consistently high quality and well researched, but I actually saw this one on HN before I saw it in my inbox. Cheers.
  - msandford 11y ago
    > I think they're right -- I, too, have observed a disconnect between how polished people sound when they talk about their work and what happens when they actually have to write code. I've done a lot of work that people would probably find very interesting and useful. But I tend to choke on whiteboard code interviews because they're so high stakes. Any time spent thinking about the problem looks bad, so you have to talk a lot. But I can't really think and talk at the same time. So I end up talking rather than thinking and I do poorly. Now obviously I'm going to push to move the status quo towards something that doesn't put me at a competitive disadvantage. So we both know that I'm biased. But the idea that people can talk about what they've done and answer any questions that you have (about what they did and programming in general) but still screw up on the actual "coding" part might mean that the part where you make them write code is more noise than signal. The problem is that you never find out because if someone bombs the coding part you simply chuckle and say "well that person is clearly a liar, or something!" and they don't go any further in the hiring process. So they never get hired, and because they're never hired, you can't evaluate their work performance. Which might be excellent when they're not being actively scrutinized by multiple people all at the same time in a high stakes situation. Unfortunately to try and get some objective data on this you'd have to hire several people who talk about their projects well but don't do well on the coding part. An understandably impossible task unless your client is a Google or Microsoft and they know it's just a big experiment regarding hiring. But until someone does that and reports back (and they won't because it'll be a competitive advantage) it's tough for me to swallow the "talks good but can't code so NOPE" that I tend to see bandied about. Putting someone in a pressure cooker and then measuring their performance will only tell you how they perform in a pressure cooker. Which is usually quite distinct from what they're going to do day-to-day.
    - leeny 11y ago
      I think what you describe is a real problem and unfortunately one that getting data around is really tough, for the reasons you describe. For what it's worth, when I observed a disconnect between how well people spoke about their projects and how well they coded, it was generally a situation where someone had perfected a pretty polished self-pitch rather than a situation where I drilled down deeply into what they had done, asked them what they'd have done differently if we varied up certain constraints, etc. And when they fucked up on coding, it was on warmup problems that was something you'd reasonably expect anyone with some experience to be able to do (e.g. explain why you might want to use a hash table over a linked list for certain scenarios, reverse a string in place). That said, one of the reasons I'm really psyched about interviewing.io (the thing I'm working on now) is that we're getting a lot of comparative interview data, i.e. where the same person gets interviewed a bunch of different ways. Excited to see if we can draw some good conclusions about what works and what doesn't.
      - msandford 11y ago
        > we're getting a lot of comparative interview data, i.e. where the same person gets interviewed a bunch of different ways. Excited to see if we can draw some good conclusions about what works and what doesn't. So I think that only works if you hire everyone, whether they interview well or not. Or else the process is biasing the results and it's not representative anymore. If you really wanted to get better information you'd have to go interview people who are already employees at a particular company and have outsiders (people who don't already know them) conduct the interviews. Then when you're done you can compare the simulated hire/no hire results and the interviewers recorded confidence numbers in their evaluation against the performance evaluations of the interviewed employees. So long as the outsiders conduct many different types of interviews (especially besides what the company normally does) you might get a clearer view into what kind of interviewing works well and what doesn't. I know some people that applied to and got hired by Google. Google seems painfully aware of how uncorrelated their interviewing process is with their hiring results. The hoops that these guys jumped through I never would. So even if I was talented enough to work at Google (I won't speculate here) they'll never actually be able to hire me unless they actively recruit me and don't make me run the gauntlet. The whole problem is a really tough nut to crack. I suspect that all the pipelines are going to be biased one way or another. If I were in charge of hiring, I'd want to try and use several of them so as to not miss out on good candidates who are undervalued for whatever reason. There's a lot of talent out there, despite everyone thinking that there's a talent shortage. The error actually lies in trying to have a one-size-fits-all solution to a problem that's definitely not uniform. Companies are failing to adapt to the human-ness of their "human resources" and it's costing them.
fecak 11y ago
Another great article from Aline. Past employers, schools, and GPAs can obviously generate false positives. I like this idea overall and would be interested in seeing the results of others. 400 applications to 1 hire isn't a great result, and I was somewhat surprised that only about 5% of applicants were even interviewed. A few months ago I launched a side project, doing (of all things) resume review and revision services. When my clients want a review of a resume that I know won't get results, and I ask "Give me more to work with", the types of things I hear are eerily similar to the "awesome stuff" quotes in this post. I try to incorporate those things into the resume when possible. Is it the resume itself that is the problem, or is it that candidates are just less inclined to include additional details (that may seem irrelevant) that could differentiate them from others? Some resumes will list accomplishments that make it rather clear of their qualifications, but everyone doesn't have that luxury. When a candidate doesn't have a long list of work accomplishments, do they think to include this type of content that might get our attention?
- vonmoltke 11y ago
  > 400 applications to 1 hire isn't a great result, and I was somewhat surprised that only about 5% of applicants were even interviewed. Though Aline didn't give hard numbers on this, I would not be surprised if a majority of the applicants completely ignored Keepsafe's instructions. She mentioned that a bunch just dumped generic cover letters or links in the box instead of actually answering the question.
- brettlangdon 11y ago
  I can speak from my own experiences reviewing engineering candidates. Sometimes when you see someone's resume and they have great work accomplishments, like "last 4 years at [successful, engineering focused company], building their [product everyone knows about] platform", it makes deciding to interview that person fairly easy. However, there is that issue when someone is either just starting out from college, who doesn't have a lot of work experience or who hasn't had the best of work experiences (usually not their fault). For these people the main thing that I look for and try to evaluate is their ability to learn on their own. The main thing to consider when reviewing a candidate like that is, if they aren't as up to speed on the technology we need them to, how long do I think it will take to on-board them and get them self-sufficient on our platform. If someone does not have the best resume, but their Github profile is full of projects, even half finished projects, of them trying out different languages or frameworks or maybe making (or trying to make) contributions to open source, it usually makes it easier to say "lets give them a chance," especially if they have been playing around with the languages/frameworks we use. Hope that helps answer your question.
deleted 11y ago
[deleted]
blfr 11y ago
Would a Northrop Grumman engineer with a GitHub full of cool projects really be overlooked in a recruitment process? Not once, that can always happen, but regularly?
- soham 11y ago
  Yes, regularly. Valley has a very short attention span. NG isn't in the list of hip sexy companies. Same goes with profiles from Cisco, VMWare, Intel, Synposys, IBM, and pretty much all the big companies that were pinnacles of business and your career at one point, but they are not considered hip anymore. Looking at Github profiles, however much people talk about it, also doesn't regularly happen. It's a chore to type it out in the browser, if you're looking at a paper resume or if it's not hyper-linked. And again, if you worked at NG, what possibly interesting things you could have done? It's not Pinterest or Uber.
  - ILIKEPONIES 11y ago
    This brings up a really good point. Hyperlinking to Github, or other work, in your resume is important. You should make it as easy as possible for the reviewer to say 'yes' in a short amount of time.
    - leeny 11y ago
      What sucks is that even if it's hyperlinked, people probably won't click it. I don't have clickthrough data for resumes (if someone does, that'd be awesome), but anecdotally, clicks won't happen unless you really call out to the reader that it' worth their time. When you're reading resumes you go into awful zombie mode (I do this too). If you're writing one, pretend your audience is a braindead zombie that needs to be spoonfed everything. If you have cool projects, list them and describe them concisely in a way that makes clear that they're interesting and a big deal. And link to each project if possible to increase odds of click.
      - brettlangdon 11y ago
        Obviously everyone is different in how they review resumes, but I am Software Engineer and whenever I see a resume with a github, I'll usually visit that first before even reading the rest of their resume. Since I work in and around Github and open source, usually someone's Github will tell me what I really want to know about them (not always, but really good candidates have a Github profile that stands out).
soham 11y ago
Thanks Aline, for yet another well researched article. I have to say though, that in my experience, these experiments in sourcing work quite well when your hiring is small. The moment you hit some sort of scale, it becomes very very difficult, if not impossible to run and rely on such experiments. E.g. in the first growth phase at Box, we were tasked with hiring 25 engineers a quarter. At that scale, the company deals with too many resumes and too many stakeholders in the hiring process. And at that point, you also have a group of people explicitly looking at resumes, less involvement from actual hiring managers, deadlines to meet, land to grab etc. Not saying one thing is better than the other, just that hiring at scale is an entirely different game. The other thing, which is implied in the article, but may get lost if the reader isn't careful: regardless of how a candidate is sourced, the interview bar still remains the same. i.e. AJ also must have had to clear same or similar technical interviews like other engineers that got hired there.
- leeny 11y ago
  Hey Soham. We have to stop meeting like this :) Yeah, scaling this stuff is hard. I do think there's a big danger in scaling it by offloading filtering powers to non-technical people because then you have to rely on proxies. Proxies aren't inherently bad, of course, but the ones we have now (school and past employment) are pretty bad, and if ultimately, we're getting things wrong more than we're getting them right, it outweighs the temptation to cut costs and time. For a company like Box with a super strong engineering brand, it's OK to have a pretty high false negative rate, of course. You can reject a lot of good people and still have a revolving door of others who want to work there. However, smaller companies often take their cues from big ones and adopt the same processes without realizing that they may not work the same way. And yes, thank you so much for calling that out. AJ had to meet the same bar as everyone else. Fortunately, he killed it.
vonmoltke 11y ago
> While AJ’s government work experience gave him a good amount of cred in the public sector, he found that making the move to industry, and startups especially, was near impossible. It wasn’t that he was blowing interviews. He just couldn’t get through the filter in the first place. ... > It was AJ, a candidate that Zouhair Belkoura, KeepSafe’s cofounder and CEO, readily admits he would have overlooked, had he come in through traditional channels. This was the story of my job search three years ago. It still kinda is.
- pzxc 11y ago
  I'm going through this right now as well. I ran my own business for 7 years, making content websites monetized through google adsense, building games, and freelancing, among other things. The primary source of revenue was adsense, and it paid the bills enough for me not to need a "real job" for years. 18 months ago, the adsense revenue started to dry up, and in an effort to make some additional income, I started looking for a normal job again. I ended up reluctantly taking a job working for the State of California, mostly because the schedule was flexible and it was a 5 minute commute. In the last 18 months I've gotten two promotions, including one four months after I started which is unheard of in state service (and which my bosses had to fight HR to get). I spearheaded the acquisition and implemented of a version control system, which no one hear had ever used (they were just FTPing files all the time, risking overwriting of others' work etc), and got buy-in from all the developers who now say they couldn't imagine working without it. I also now wear a variety of hats besides programming including sysadmin work, dba work, architecture design, etc. My bosses rely on me more and more everyday just for my opinions and advice, let alone the work I do. Yet now, when I send out my resume, it's almost always crickets. A year and a half ago, when the first item on my work experience list was "failed entrepreneurship", the response rate to my resume was about 80% (not even talking about interviews, just getting a response at all). Now it's more like 20%, all because the first item in my work experience section is my current job working in public service. I admit, many of my coworkers probably deserve the reputation that public sector work has. A significant number of them are clock-watchers that the bosses don't even try to assign anything important to, because they know they don't give a shit and can't easily be fired (union), they are just filling a chair waiting for their public pension to accumulate over 20-30 years. Nevertheless, I also list on my resume all of the above, including the pioneering (for us) work I've accomplished here implementing version control etc. I never intended to stay here more than a couple of years, but I also never intended for this job to have such a negative impact on my job prospects. I think I am probably going to revise my resume to leave this job off completely, and just say I've been working for myself for 9 years instead of 7. I predict sadly that this will return the response rate back to what it was before I had this job.
ammon 11y ago
Interesting post. Talking and writing about yourself well, even when not matched with programming well, probably helps get job offers (and in many cases helps be a good employee). It's difficult to tease these two thing apart. I imagine looking at speaking and writing can still miss great people (as can every filter), but I can believe that it's much better than looking at resumes.
Harj 11y ago
We have the same belief in the limited usefulness of resumes at Triplebyte. We found talking about projects with candidates both more enjoyable and interesting than looking at words on a resume. Especially when people are enthusiastic about what they built. The difficulty we had was not seeing a strong correlation between talking about projects and doing well at programming during an interview.
- levlandau 11y ago
  I think the other thing to look into is what "talking about projects" means. I'm willing to bet that there's some version of talking about projects that correlates well with programming ability. I've had to screen ~ 500 engineers and we basically use the same method aline describes as a first filter and then talk about the project. Imo it's other things like reliability, speed in real world conditions etc that are a bit harder to verify but those are hard to verify in a programming interview as well.
  - Harj 11y ago
    That's definitely true. Do you have a particular set of questions you ask during project interviews? If so, which ones tend to be the most revealing?
    - tjradcliffe 11y ago
      Not the GP, but I focus on tradeoffs and roads not taken. Get them to talk about a project they've worked on, and then ask about alternative design decisions around some interesting feature and see what they say. Example: I developed a little state-machine framework for managing complexity in a large, legacy code-base. It allowed me to refactor a lot of ad hoc distributed logic into the transition table and clean up a lot of weird corner cases that made the code fragile and difficult to change. Questions might include: "Why did you write your own rather than use an existing state machine framework like the one in boost?" (for C++ frameworks there's pretty much always one in boost, so even if you don't know anything about the area you can throw this in for fun and see what they say). Also: "Why a state machine rather than some other approach to refactoring?" And so on. This process gets at taste and good judgement, it gives you a sense of how tolerant of alternatives they are, and so on. Additional edit: one of the things I look for in answers is people who say, "Yeah, that particular decision might have been a mistake... I always wondered what would have happened if instead I had..." Good developers are able to admit that not everything they do is perfect, and are willing to give alternative views a bit of credence.
codeonfire 11y ago
When people abandoned resumes like this it's because there is some corruption in their hiring process and lesser skilled management is attempting to hire down for political control. Their only goal is to go through the motions and stay employed. Some people don't want experts. They want to have the illusion of a functioning business unit. People with no formal training that have projects that sound impressive to the layperson are not going to make waves or quit when they find out that management really doesn't know what they're doing. Just as one can build a model airplane in their garage, those same people will never be able to build an airliner. It's not a good idea to hire hobbyists if you're in the airliner business. When you have layperson managers judging what is a strong candidate and what is important technology, you are going to get hobbyists skilled in popular tech and your company is going to get worthless hobbyist tech for your organization. For example, the article refers to an 'open source Android animation library.' To many people that sounds like a massive, great achievement. On the tech scale of importance it is a 0 or 1 out of ten. No business can be formed around an animation library, it's not a difficult or uncommon thing, and there are thousands of alternatives.
- woah 11y ago
  What are you talking about? The odds are that the company hiring started as a "worthless hobby"
- codeonfire 11y ago
  In a way, the entire premise of what a software engineer is has been destroyed. Imagine if mechanical engineers were hired based on their ability to design a simple bracket. Imagine if electrical engineers were hired because they amazed management by building a flashing LED circuit. After many years, the entire criteria for being an engineer is dumbed down to "are you able to make a flashing led or simple bracket or something equally impressive?" That's what the people chosen to manage are familiar with. That is what has happened to software engineering.
  - Nacraile 11y ago
    I'm not sure I buy your implicit assertion that other engineering disciplines are somehow better at hiring than the software industry is. As far as I can tell, mechanical and electrical engineers are hired based on relevance of work experience, some discussion of that experience, and maybe a few specific technical details which should be known by anybody with the experience they claim to have. All of which is isomorphic to "Worked at Google, could explain his project in depth, and coded tree inversion on a whiteboard". The only difference I'm aware of is that a Mech or EE would probably be subjected to bullshit situational interviews by HR. This kind of feels like the general "software isn't rigorous like the real engineering disciplines" angst, which as far as I can tell is also bullshit. In reality, the "rigorous" old-school disciplines still manage to make colossal messes of complex, unprecedented projects, in the same way that software companies often make colossal messes of complex, unprecedented software projects. Sure, civil engineers can build normal roads and bridges and buildings reliably, but then again, software engineers have no problem throwing up CRUD apps and wordpress blogs. There are plenty of bridge collapses, exploding batteries, stalled tunnel-borers, and so on to match all of software's spectacular failures. It turns out that complicated things are really hard to build, regardless of your field.
s_q_b 11y ago
I agree we need an alternative to resume-based hiring, and the hiring process in general. For example, I don't do well in whiteboard interviews, which is odd because I normally don't have a public speaking issue. It feels like there's some muscle memory attached to coding that isn't well replicated with poor handwriting in a room full of people. Whiteboard lines of code are simply not the manner in which developers work once hired. That is the reason for the disconnect between speaking well about projects (easy to verbally explain and sketch) and the programming portion (bizarre.) Right the industry is doing the equivalent of interviewing lawyers by asking them to write a legal brief on a white board. We're testing the wrong thing: a proxy for the work, when we we could easily test the work itself. I much prefer work sample tests rather than whiteboard Q/A as it better replicates the actual job. Give me a few hours with problems I would actually face on the job, my dev environment, internet access, and a set of problems that truly reflect the work, and I find it much more natural. Is it too much to ask that an interview measure skills the job actually requires, in an environment that emulates the work?
nickpsecurity 11y ago
Great article. A good move. I think one of the reasons the method is successful is that it asks applicants to keep it real. The traditional channels want people to come off a certain way. People also know about their filtering rate. So, the incentive for them is to tell companies what they want to hear and in a way that conveys unreliable information. Seems your example changed the incentives, got useful information in return, and that led to a positive result. Unsurprising in hindsight. I'm going to send your article to a few people to see if I can get any to try that approach.
a-dub 11y ago
Resumes are fine, it's really in how you treat them. When I read resumes I generally ignore where people have worked and gone to school and instead look for what they have done. If there's either a good match between the general type of stuff they've done in the past and what the role is, or if there's stuff on there that is interesting enough such that I'd enjoy hearing about it, I give a thumbs up. When I interview, I tend to spend most of the time asking in depth questions about the projects I find most interesting on the resume. What was easy? What was hard? X sounds like it would be a problem, how did you solve it? What was fun? What was headbangonthewall miserable? Generally this gives a sense as to whether or not there's any bullshitting going on, and gives a sense for whether or not the candidate has a good head for thinking about hard problems. Finally, I'll ask a few questions to probe for "difficult-to-work-with" red flags and finish with a few fairly easy "technical challenges" that offer opportunity for the candidate to either walk about having solved the problem, or walk away having solved the problem and demonstrated understanding of the solution from top to bottom.
- __z 11y ago
  What kind of "difficult-to-work-with" questions do you ask? I'd like to filter out these people but I'm not sure what sorts of questions to ask to suss that out.
  - a-dub 11y ago
    Basically I goad them into complaining about past bad work experiences and then pay close attention for subtle clues that may indicate that they are systemically disrespectful or unwilling to compromise on things that don't seem worth fighting for. Were they bothered by people, situations or outcomes? Tell me about the most frustrating time when you needed a thing, or consensus on a thing, and you had to go through way too much to get it. Tell me about the most frustrating time when you needed a thing, or consensus on a thing, and no matter how hard you pushed, you never got it. And, of course: The fastest way to be shown the door around here is to be an agitator of your colleagues on the grounds of race, sex, religion or any other attribute that has little to do with work. Nobody here is in the business of policing behavior and nobody wants to be. This requires perhaps more discipline than other places as the hammer falls harder and quicker here if things go awry, so it demands either heightened discretion or a heightened sense of self awareness and awareness of those around you. Do you think you would be able to work under such conditions?
dropit_sphere 11y ago
Is it just me, or does this still seem like a crazy hard problem? There were still 415 not-resumes to wade through for one offer.
tikhonj 11y ago
First of all, I definitely agree that resumes are woefully overused and overrated. They have structural problems and incentivize people to use "negative selection" criteria where people are eliminated based on not having a specific feature rather than selected for excelling on something¹. This article neatly demonstrates that resumes are not necessary and that not using them can unlock new sorts of candidates. However, I don't think there's a conclusion to be made about the actual method used here. I suspect that it worked because it was different, not because it carried a fundamentally strong signal. If everyone did this, project descriptions would be gamed even more than resumes—it would select for people who prepared for the selection process² more than anything else. This reminds me of various captcha strategies I've seen used by small forums to great effect—solving some math, typing a word into a text box, choosing a popular character's picture… etc. They all work, perfectly. But only because spammers don't care about the small fry: it's not worth their time to modify their bots for your little site. If any given captcha becomes used widely—or your forum grows big enough—they will bypass it trivially. Now, an essay like this isn't quite as bad as a captcha, but the idea is the same: it works because it's new and different. If everybody used it, it would probably be a step back. Ultimately, I think the real moral is that more companies should do their own thing, even if that thing is not great in the abstract. Being different carries a value of its own, and it breeds biodiversity that's healthy for the system as a whole. (Of course, many of the things companies try are really bad for various reasons, but that's a different story…) ¹ In particular, most people have a bunch of "red flags" they look for with, at best, cursory rationale—everything from passing on people who didn't go to the right school to those who have breaks in their work history, based on "common sense" or "experience" rather than anything meaningful. Most of these criteria seem counter-productive. ² I also think this is really true for college admissions and especially the admissions essay. A project blurb for hiring is more or less the same idea in a new context.
- function_seven 11y ago
  > This reminds me of various captcha strategies I've seen used by small forums to great effect—solving some math, typing a word into a text box, choosing a popular character's picture… etc. They all work, perfectly. But only because spammers don't care about the small fry: it's not worth their time to modify their bots for your little site. If any given captcha becomes used widely—or your forum grows big enough—they will bypass it trivially. Which reminds me of Jeff Atwood's original "captcha"[1] on the Coding Horror blog. It was a static image of the word "orange" every time. It was the most trivially defeatable captcha ever, but he didn't care because it worked. It would have been senseless for him to invest time and effort into implementing a complex captcha engine, when the existing solution was filtering spam bots just fine. [1] http://blog.codinghorror.com/captcha-is-dead-long-live-captcha/ http://blog.codinghorror.com/captcha-is-dead-long-live-captc...
roymurdock 11y ago
So, essentially the company that was hiring (KeepSafe), allowed applicants to submit highly flexible cover letters, and then they actually read the cover letters. It would be great if the majority of companies used both the resume and the cover letter effectively. It feels like most companies that require a cover letter only do so to screen out the laziest 10% who can't be bothered to write up a generic 1 page essay filled with ass-kissing and vague jargon. The cover letter is just a relic from the olden days when the application was slower and more formal. There were less applicants for each position so HR probably had more time to read/screen. This study presents an interesting alternative: Let people submit some text along with their resume on any topic of any length, and see how their personality comes through in the writing. Probably wouldn't work extremely well at a large company, but it seems like it served KeepSafe quite well.
- autokad 11y ago
  like the above commented, people would game the text, hire coaches, etc. I don't think there is anything fundamentally wrong with the cover letter / resume process. i just don think companies use resources properly at hand. I am biased, i think most HR are useless and most tech people aren't great at picking candidates either. they seem to have their own biases that they cant get over (for example: putting too much weight on technical skills and none on soft skills). honestly, if i were a company i'd send select employees to school. either take classes or better yet teach a class. you'd get a semester long interview process and the employee would get something out of it.
mpenn 11y ago
The writer claims to "rely heavily on data," but the punchline of the article is purely anecdotal that 1 person at 1 company got hired, was good and would have been overlooked. I am sure there were also many candidates with good resumes who were now overlooked. This article starts out with an air of science and ends with a completely unproven conclusion. While I do agree in my gut that resumes are not an amazing filter, she has completely failed to present evidence that her alternative interview process is better. And in fact, while KeepSafe still has the no resumes option open, they are now accepting resumes again -- I do not great confidence that the alternative system was anything more than a PR move by the company.
- roneesh 11y ago
  Shortly after she says she's come to "rely heavily on data", she says "This post, however, is going to be a bit of a departure. Rather than making broad, sweeping conclusions based on a lot of data points, I’m going to narrow in on one story that happened". I think she did a great job of doing exactly what she set out to do, and since this is just one anecdote, any qualitative or numerical data she presents won't be worth much, all the more reason to omit it and just share the story.
  - mpenn 11y ago
    My bad on that -- I definitely misread the beginning. After rereading, she presents the beginning and end fairly based on an anecdote and not data. I'd still love to know why the company didn't switch to this full throttle.
wambotron 11y ago
I worked at two places where people had resumegasms over "brand name" schools. Neither of them hired anyone who was ever any better than completely average (and yes, I include myself). I don't really care where anyone went to school. It doesn't mean anything. Really, going to school at all doesn't mean much. I need to see what you've done outside of that to make any meaningful evaluation. It doesn't matter if it's a huge project. You can give me a couple 10-line things that do something useful and I'll still get to see how you name things, format code, use built-in libraries, etc. Then we can chit chat about project management and how much you love or hate it.
caublestone 11y ago
We use an alternative screening process where we prompt prospective interviewees with 2 extreme customer service complaints (this product sucks, it didn't arrive on time, i hear it's poison etc.), ask them to provide an answer directed to the customer and create a plan to prevent the issue from happening in the future. It's fascinating to see how people approach the problem of making people happy now and preventing dissatisfaction down the road. If they pass (1/10 do), we use a 30 minute phone call to identify interests and motivations which ends up being the strongest indicator of value add. This might work for B2C companies only but I'd love to see any company identify people that are truly passionate about making people happy. Edit: Clarifying that this is for non-engineering roles.
- kansface 11y ago
  I don't suspect strong correlation between the desire to make customers happy and productivity writing software - maybe you can tell me if my intuition is correct.
  - caublestone 11y ago
    Ah, I meant to say we do this for non-engineering roles. For engineering we provide coding challenges as a pre-screen.
sjtgraham 11y ago
I screened and interviewed a lot of developers in my last position and it stood out to me that résumé quality seemed to be inversely correlated with the candidate's actual ability. I distinctly remember the absolute best developers that I hired also had the most atrociously bad résumés. The candidates with résumés that literally almost knocked me off my chair failed miserably at the most basic programming task.
- dexterdog 11y ago
  That's largely because so many diamonds get jobs thrown at them and wind up in the "I haven't updated my resume. No worries, just send what you have." They get the interview, knock it out of the park and don't bother to update it.
- vonmoltke 11y ago
  > I distinctly remember the absolute best developers that I hired also had the most atrociously bad résumés. The candidates with résumés that literally almost knocked me off my chair failed miserably at the most basic programming task. Comments like this make me wonder if one reason for the ridiculously high failure rate in programming interviews is that the resume screening stage is actually acting as a negative filter, that is disproportionately removing the candidates you want and passing the ones you don't.
- rokhayakebe 11y ago
  Frankly I cannot see an expert in anything other than marketing sitting down and crafting their resume. "Appearance starts where performance ends." Let me illustrate: Growth Engineer One line Resume: "I was employee 7 at Snapchat when we had 6 engineers and 400,000 users. During my tenure as the only growth engineer, our userbase grew to 10 million users over the next 6 months." SEO resume: "I joined XYZ when it was ranking at page 10 for major industry keywords. 9 months later. Google the following keyword BDHDUYD, which accounts for 40% of your market. If you find the company in the top 3 results, we should schedule an interview." Software engineer: I do not know, but I am sure you can insert a short paragraph here.
  - 11y ago
AndyNemmity 11y ago
I just went through a job search, and never created a resume. I only submitted my linkedin. If anyone required a resume, I immediately responded that we weren't a good culture fit. Worked out really well. Not sure it's to be duplicated, but for me it went fantastically.
- puredemo 11y ago
  What if I've never had a linkedin because of all their repeated data security issues? Your comment sort of sounds like a shill post, fyi.
- suyash 11y ago
  LinkedIn is basically an overblown resume, so that doesn't really make a different, try without LinkedIn and resume and see how many people bothering to contact you back.
- noarchy 11y ago
  It often happens that recruiters will contact me on LinkedIn, on which my resume is essentially laid out in some detail, and then still demand a resume. I assume they have to enter an old-style resume into some sort of internal database, as per company policy.
deleted 11y ago
[deleted]
sytelus 11y ago
You can do much more easier, automated filtering: Ask candidate to submit link to any of the followings: 1. Github a/c 2. StackOverflow a/c 3. Their blog 4. Anything they made online If candidate fails to submit link for any of above then just don't interview them. I would guesstimate this simple check filters out 70% of the junk resume and probably 20% of the good resumes. It can scale like crazy and expanded even more (for example, use APIs to get their profile information and rank resumes).
- istvan__ 11y ago
  Also I would ask them to create it as a pull request on Github push it to a branch in a private repo. Another 10% of candidates gone. (There might be overlap with the guys who don't have Github presence and anything online but you get my point.)
- sliverstorm 11y ago
  If candidate fails to submit link for any of above then just don't interview them. Since when has every person worth hiring had their own blog? Edit: sorry, your use of "any" was ambiguous, I read it as "if missing any of the above".
- throwaway12309 11y ago
  * 20+ years of development experience 15 of those professionally * published author of various books * worked in systems that most startups would shit their pants with the requirements * some of the big name companies have tried to poach me based on the products I've worked * used professionally: C, C++, F#, C#, Ruby(Rails), Swift, Obj-C, Kotlin, Haskell, Scala I would never be called based on that criteria as I have no inclination to spend my free time doing stupid shit online for hipster new developers that think GitHub is the end-all. ps: to be fair, I probably wouldn't want to work in a company that has this mentality, so maybe that really does work
  - Cyph0n 11y ago
    So you're saying that the people who spend their free time developing projects that you most probably use in some form or another are doing “stupid shit”. I wouldn't hire you either.
sopooneo 11y ago
All these discussion of how hiring is broken. Doesn't this imply an enormous market opportunity for the recruiting agencies or companies that can properly capitalize on either undervalued candidates or better knowledge?
- shanemhansen 11y ago
  Recruiting companies aren't in the business of finding good candidates, they are in the business of finding candidates their clients like. Companies have the same problem the military does. They hire a certain stereotype because they've always hired a certain stereotype and the people in charge match that stereotype. If there's any opportunity for disruption, it would be at tiny companies where there are people that 1) just need talent and 2) know that they process that made them successful is broken.
- jerf 11y ago
  Alternatively, it implies that there is no solution. You can obviously not make stupid mistakes like taking two months to call a candidate back, but beyond the basic competence there may be an irreducible element of rolling dice. It may be the case that evaluating humans is fundamentally difficult. I'm not advocating for that. I could argue either side of it without any cognitive dissonance. Just pointing out the additional possibility.
paulgayham 11y ago
For what position? How can you judge candidates without a position to judge how well they fit? I can just see this guy going out to Web devs, System devs, DBAs etc. and them all disagreeing because they're looking for different things (and value things differently).
pbreit 11y ago
What surprises me is how many people dislike cover letters. When I go through applicants, a decent cover letter demonstrating some enthusiasm about the company and a unique point or two is appealing.
- deleted 11y ago
  [deleted]
- deleted 11y ago
  [deleted]
- deleted 11y ago
  [deleted]
- m3talridl3y 11y ago
  It surprises me how many employers think that any of their engineers give a crap about the actual business. The better ones have some form of passion for creating good code, the actual business is an implementation detail.
  - pbreit 11y ago
    I guess I've only worked for businesses that I gave a crap about.
  - vonmoltke 11y ago
    The domain, product, or use case is what draws me in. But then, I'm of the "classic" engineering mindset; a maker. Code is a means to an end, not an end in itself.
ageofwant 11y ago
This is very bad news. I have always highly valued resumes as a very effective candidate filter. It works as follows: I take the pile of resumes, shuffle them thoroughly an divide roughly in half. The pile to the left goes in the bin. I repeat this process until I have the luckiest candidate's resume in my hand. This is the type of guy I want to associate with: one on whom fortune smiles, repeatedly.
bsder 11y ago
> His GitHub, full of projects spanning everything from a Python SHA-1 implementation to a tongue-in-cheek “What should I call my bro?” bromanteau generator, hinted at a different story, but most people never got there. While AJ’s government work experience gave him a good amount of cred in the public sector, he found that making the move to industry, and startups especially, was near impossible. Huh? The companies this woman hires for don't look at github? Not looking at public code that someone has published is more broken than relying on resumes. If someone has published code and it doesn't suck, I'll probably bring them in for an on-site, period. I may even tell them that "We're going to talk about "file foo.c in your code where you implemented feature Z. So be prepared." And, I suspect with startups it was more a case of "How many years were you in government? That would makes us so unhappy that we would leave. Why didn't you?" That's a different way of asking "Is this really the place for you?" As a hiring manager in a startup, when I knew I only had 9 months of runway without more funding, I'd feel REALLY bad about taking someone with a family away from their very stable job. As someone who has recruited employee single digit, I often have made a point to meet the family when recruiting someone--even if I have to fly to them. I need both the prospective employee and their partner to understand that the big probability is that the company won't be around in 24 months, there won't be any payoff, and a new employment search is likely to be the result. Yeah, there is a small probability that we'll survive and an even smaller probability that we'll get some money. It's a really delicate balance for me, at least, to properly sell the company (Startup! Options! Novel!) and reality (Bankrupt! Flameout! Layoffs!). I'd say I'm batting about 50%. For every employee I scare off, I absolutely convince one to join. Funnily enough, every single one who didn't run away said the same thing: "My wife told me I had to work with you." They were stunned that someone so important (Hah! Management in a startup is a good way to understand how unimportant you are really quickly ...) would take the time to make sure the family was informed properly about the risks and rewards.
andrewstuart 11y ago
Everyone should just give up on trying every sort of new angle to "identify great developers". It's purely subjective, there is no meaningful universal definition. It comes down to whether or not the people doing the recruiting all have the same subjective opinion.
dzhiurgis 11y ago
So you've turned down 399 people to employ 1. Not sure if that is any different when employing via resume, but that sends shivers to my spine. 399:1 ratio says to me that there is oversupply of engineers.
- dsymonds 11y ago
  It means there's an oversupply of people who want engineering jobs. Most people applying for software engineering positions aren't able to program even the most basic of things. That doesn't mean there's too many qualified engineers.
dools 11y ago
I do something very similar in my hiring processes on odesk. I ask each candidates opinion on something (articles usually). This allows me to eliminate 90% of applicants. Then I review the 10 applicants who's answers weren't complete gibberish and give the best 2 or 3 a programming task. Works pretty well, I've only ired one guy turned out to be inadequate for the role (out of about 20 hires over the past 4 years for php dev work).
7402 11y ago
I've always found resumes to be quite useful in figuring out who to bring in for an interview - BUT, most of the people I've been involved in hiring have had between 5 and 15 years of experience, usually at at least 2 or more real companies. I can well believe that if you're only interested in people right out of school, then it's harder to figure out candidates from their resumes. I am a little puzzled, though, about why others seem to find resumes so opaque. It seems like resume-reading is a lost art. A resume is usually a document that someone has spent a lot of effort on to make themselves look good. If you learn to read them, that can tell you a lot about the author. (Note: searching for buzzwords is not "reading.") A resume should not be regarded as simply a collection of facts - of course you'll be misled if you do that; a resume should be regarded as a document of self-expression. After a while, you can see useful patterns in what people put in resumes - a least for more-experienced applicants. Almost every resume suggests a bunch of next questions, which can be asked in a phone screen or interview to get a pretty good idea of what a person is about. It's worth recalling that absolutely all software engineers at all software companies in the world from the first ones around 1955 up to 2002 were hired without benefit of LinkedIn, StackOverflow, Github. Almost all of these engineers submitted resumes, which were reviewed prior to offering interviews. Yes, there were hiring mistakes in the old days, but I don't see a huge number of people taking about how the hiring process now is so much easier, smoother and more foolproof than it used to be.
tempestn 11y ago
> Resumes don’t have an explicit section for building rockets or Minecraft servers, and even if you stick it somewhere in “personal projects”, that’s not where the reader’s eye will go. That's true, but a good cover letter can go a long way toward helping with this. Most cover letters are generic, bland, and obviously copy-pasted from a template. (Or more often from a previous application, sometimes with info about the previous company left in!) A cover letter that talks about something exciting you've done recently, and ideally how it might be related to the job, or even just how it demonstrates skills you'll use in the job (and describes exactly how), is awesome in comparison. A letter like that would absolutely get you an interview with me, almost regardless of experience. One of our current co-op students actually had almost no programming experience on paper; he had actually switched out of a theatre degree iirc. But his cover letter was awesome (the theatre degree probably not being coincidental). Got him the interview, which got him the job, and I haven't regretted it. Just a co-op of course, but the point stands. The cover letter is probably the most important part of your application. Take the time to write a good one.
lordnacho 11y ago
The "new hiring process" experiment comes up here quite often, and I do appreciate it. However, how can we conclude anything from a procedure that only examines the hired population and none of the unhired?
- fsk 11y ago
  The only way you can truly be sure of your hiring process is to pick a random sample of people who fail, and hire them anyway. I see a couple of issues with this: - Could it get you in legal trouble? - Are you capable of evaluating who are the best employees AFTER they are hired? - Only a large corporation would have the resources to gamble on hiring a random sample of people who failed their interview process. Here's one half-hearted way to do it. Pick a random sample of resumes you reject, and give them phone interviews anyway. Pick a random sample of people who fail your phone interview, and give them an on-site interview anyway.
Mimu 11y ago
I fail to see how someone building games at 14 or own one of the most successful Minecraft server could not pass the resume filter. Not saying the process described in the article is bad, even though I believe anything can be gamed, but I don't really see a big difference. Main change is the way recruiters looked at what they got, resume or essay wouldn't have change a lot I think. Maybe off topic but if companies want the best people, maybe THEY should write the essay explaining why people should join instead of sitting in their high tower waiting for minions to come.
vultour 11y ago
Something else piqued my interest in this article. This lady claims that the company is fighting for candidates with Google, although the only thing they do (if i read it right) is provide an encrypted version of Dropbox. How does this require world class engineers? I've coded a file syncing app quite fast as a personal project once, and I don't think I could call myself even a regular developer. I do not believe such application would be even remotely as complex as anything Google does.
chris_wot 11y ago
My submission would be: I was a help desk pleb at a well known inkjet/scanner/camera company 15 years ago and this company "extended" their clipper database to record third party cartridges, but recorded them in .ini format. That's right, one file per record, in key=value pairs. I was bored and accidentally mentioned to the guy whose job it was to copy and paste the data from each of all 90,000 ini files into an Excel spreadsheet that Perl could do it, and I'd even use references to hashes to do it. He had no idea about that last bit, but I did it for him on the proviso that he didn't tell anyone, and reduced 10 weeks of work to 30 seconds. They unfortunately made me employee of the quarter but neglected to tell me so I missed my awards ceremony.
moises_silva 11y ago
GapJumpers seems like a good alternative to resume screening: https://www.gapjumpers.me/ https://www.gapjumpers.me/ Even if not using GapJumpers itself, you can follow the concept by requesting solving a problem or submitting a piece of original technical content along with the resume.
ilaksh 11y ago
I think allowing people to submit a link to a demo or portfolio makes a lot of sense, especially since everyone has computers and the internet now. Welcome to the amazing new world of hyperlinked multimedia, hiring people.

448 points by appamatto 11y ago

I think this is very interesting counterpoint to TripleByte's data which implied that talking about passion projects was lower signal than their coding quizzes. The benchmark was performance in a long form coding interview for TripleByte, whereas Aline's is the final offer, so not exactly apples to apples.

leeny 11y ago
Aline here. Fwiw, I don't disagree with TripleByte's findings... anecdotally, after having interviewed somewhere between 500 and 1000 people in my career, I think they're right -- I, too, have observed a disconnect between how polished people sound when they talk about their work and what happens when they actually have to write code. What we're actually comparing here, though, isn't coding vs. describing projects. It's describing projects vs. resumes. And I expect that, there, resumes provide a lower signal.
- Kluny 11y ago
  Hey Aline, thanks for the article. I've subscribed to your RSS because your posts are so consistently high quality and well researched, but I actually saw this one on HN before I saw it in my inbox. Cheers.
- msandford 11y ago
  > I think they're right -- I, too, have observed a disconnect between how polished people sound when they talk about their work and what happens when they actually have to write code. I've done a lot of work that people would probably find very interesting and useful. But I tend to choke on whiteboard code interviews because they're so high stakes. Any time spent thinking about the problem looks bad, so you have to talk a lot. But I can't really think and talk at the same time. So I end up talking rather than thinking and I do poorly. Now obviously I'm going to push to move the status quo towards something that doesn't put me at a competitive disadvantage. So we both know that I'm biased. But the idea that people can talk about what they've done and answer any questions that you have (about what they did and programming in general) but still screw up on the actual "coding" part might mean that the part where you make them write code is more noise than signal. The problem is that you never find out because if someone bombs the coding part you simply chuckle and say "well that person is clearly a liar, or something!" and they don't go any further in the hiring process. So they never get hired, and because they're never hired, you can't evaluate their work performance. Which might be excellent when they're not being actively scrutinized by multiple people all at the same time in a high stakes situation. Unfortunately to try and get some objective data on this you'd have to hire several people who talk about their projects well but don't do well on the coding part. An understandably impossible task unless your client is a Google or Microsoft and they know it's just a big experiment regarding hiring. But until someone does that and reports back (and they won't because it'll be a competitive advantage) it's tough for me to swallow the "talks good but can't code so NOPE" that I tend to see bandied about. Putting someone in a pressure cooker and then measuring their performance will only tell you how they perform in a pressure cooker. Which is usually quite distinct from what they're going to do day-to-day.
  - leeny 11y ago
    I think what you describe is a real problem and unfortunately one that getting data around is really tough, for the reasons you describe. For what it's worth, when I observed a disconnect between how well people spoke about their projects and how well they coded, it was generally a situation where someone had perfected a pretty polished self-pitch rather than a situation where I drilled down deeply into what they had done, asked them what they'd have done differently if we varied up certain constraints, etc. And when they fucked up on coding, it was on warmup problems that was something you'd reasonably expect anyone with some experience to be able to do (e.g. explain why you might want to use a hash table over a linked list for certain scenarios, reverse a string in place). That said, one of the reasons I'm really psyched about interviewing.io (the thing I'm working on now) is that we're getting a lot of comparative interview data, i.e. where the same person gets interviewed a bunch of different ways. Excited to see if we can draw some good conclusions about what works and what doesn't.
    - msandford 11y ago
      > we're getting a lot of comparative interview data, i.e. where the same person gets interviewed a bunch of different ways. Excited to see if we can draw some good conclusions about what works and what doesn't. So I think that only works if you hire everyone, whether they interview well or not. Or else the process is biasing the results and it's not representative anymore. If you really wanted to get better information you'd have to go interview people who are already employees at a particular company and have outsiders (people who don't already know them) conduct the interviews. Then when you're done you can compare the simulated hire/no hire results and the interviewers recorded confidence numbers in their evaluation against the performance evaluations of the interviewed employees. So long as the outsiders conduct many different types of interviews (especially besides what the company normally does) you might get a clearer view into what kind of interviewing works well and what doesn't. I know some people that applied to and got hired by Google. Google seems painfully aware of how uncorrelated their interviewing process is with their hiring results. The hoops that these guys jumped through I never would. So even if I was talented enough to work at Google (I won't speculate here) they'll never actually be able to hire me unless they actively recruit me and don't make me run the gauntlet. The whole problem is a really tough nut to crack. I suspect that all the pipelines are going to be biased one way or another. If I were in charge of hiring, I'd want to try and use several of them so as to not miss out on good candidates who are undervalued for whatever reason. There's a lot of talent out there, despite everyone thinking that there's a talent shortage. The error actually lies in trying to have a one-size-fits-all solution to a problem that's definitely not uniform. Companies are failing to adapt to the human-ness of their "human resources" and it's costing them.

fecak 11y ago

Another great article from Aline. Past employers, schools, and GPAs can obviously generate false positives. I like this idea overall and would be interested in seeing the results of others. 400 applications to 1 hire isn't a great result, and I was somewhat surprised that only about 5% of applicants were even interviewed. A few months ago I launched a side project, doing (of all things) resume review and revision services. When my clients want a review of a resume that I know won't get results, and I ask "Give me more to work with", the types of things I hear are eerily similar to the "awesome stuff" quotes in this post. I try to incorporate those things into the resume when possible. Is it the resume itself that is the problem, or is it that candidates are just less inclined to include additional details (that may seem irrelevant) that could differentiate them from others? Some resumes will list accomplishments that make it rather clear of their qualifications, but everyone doesn't have that luxury. When a candidate doesn't have a long list of work accomplishments, do they think to include this type of content that might get our attention?

vonmoltke 11y ago
> 400 applications to 1 hire isn't a great result, and I was somewhat surprised that only about 5% of applicants were even interviewed. Though Aline didn't give hard numbers on this, I would not be surprised if a majority of the applicants completely ignored Keepsafe's instructions. She mentioned that a bunch just dumped generic cover letters or links in the box instead of actually answering the question.
brettlangdon 11y ago
I can speak from my own experiences reviewing engineering candidates. Sometimes when you see someone's resume and they have great work accomplishments, like "last 4 years at [successful, engineering focused company], building their [product everyone knows about] platform", it makes deciding to interview that person fairly easy. However, there is that issue when someone is either just starting out from college, who doesn't have a lot of work experience or who hasn't had the best of work experiences (usually not their fault). For these people the main thing that I look for and try to evaluate is their ability to learn on their own. The main thing to consider when reviewing a candidate like that is, if they aren't as up to speed on the technology we need them to, how long do I think it will take to on-board them and get them self-sufficient on our platform. If someone does not have the best resume, but their Github profile is full of projects, even half finished projects, of them trying out different languages or frameworks or maybe making (or trying to make) contributions to open source, it usually makes it easier to say "lets give them a chance," especially if they have been playing around with the languages/frameworks we use. Hope that helps answer your question.

deleted 11y ago

[deleted]

blfr 11y ago

Would a Northrop Grumman engineer with a GitHub full of cool projects really be overlooked in a recruitment process? Not once, that can always happen, but regularly?

soham 11y ago
Yes, regularly. Valley has a very short attention span. NG isn't in the list of hip sexy companies. Same goes with profiles from Cisco, VMWare, Intel, Synposys, IBM, and pretty much all the big companies that were pinnacles of business and your career at one point, but they are not considered hip anymore. Looking at Github profiles, however much people talk about it, also doesn't regularly happen. It's a chore to type it out in the browser, if you're looking at a paper resume or if it's not hyper-linked. And again, if you worked at NG, what possibly interesting things you could have done? It's not Pinterest or Uber.
- ILIKEPONIES 11y ago
  This brings up a really good point. Hyperlinking to Github, or other work, in your resume is important. You should make it as easy as possible for the reviewer to say 'yes' in a short amount of time.
  - leeny 11y ago
    What sucks is that even if it's hyperlinked, people probably won't click it. I don't have clickthrough data for resumes (if someone does, that'd be awesome), but anecdotally, clicks won't happen unless you really call out to the reader that it' worth their time. When you're reading resumes you go into awful zombie mode (I do this too). If you're writing one, pretend your audience is a braindead zombie that needs to be spoonfed everything. If you have cool projects, list them and describe them concisely in a way that makes clear that they're interesting and a big deal. And link to each project if possible to increase odds of click.
    - brettlangdon 11y ago
      Obviously everyone is different in how they review resumes, but I am Software Engineer and whenever I see a resume with a github, I'll usually visit that first before even reading the rest of their resume. Since I work in and around Github and open source, usually someone's Github will tell me what I really want to know about them (not always, but really good candidates have a Github profile that stands out).

soham 11y ago

Thanks Aline, for yet another well researched article. I have to say though, that in my experience, these experiments in sourcing work quite well when your hiring is small. The moment you hit some sort of scale, it becomes very very difficult, if not impossible to run and rely on such experiments. E.g. in the first growth phase at Box, we were tasked with hiring 25 engineers a quarter. At that scale, the company deals with too many resumes and too many stakeholders in the hiring process. And at that point, you also have a group of people explicitly looking at resumes, less involvement from actual hiring managers, deadlines to meet, land to grab etc. Not saying one thing is better than the other, just that hiring at scale is an entirely different game. The other thing, which is implied in the article, but may get lost if the reader isn't careful: regardless of how a candidate is sourced, the interview bar still remains the same. i.e. AJ also must have had to clear same or similar technical interviews like other engineers that got hired there.

leeny 11y ago
Hey Soham. We have to stop meeting like this :) Yeah, scaling this stuff is hard. I do think there's a big danger in scaling it by offloading filtering powers to non-technical people because then you have to rely on proxies. Proxies aren't inherently bad, of course, but the ones we have now (school and past employment) are pretty bad, and if ultimately, we're getting things wrong more than we're getting them right, it outweighs the temptation to cut costs and time. For a company like Box with a super strong engineering brand, it's OK to have a pretty high false negative rate, of course. You can reject a lot of good people and still have a revolving door of others who want to work there. However, smaller companies often take their cues from big ones and adopt the same processes without realizing that they may not work the same way. And yes, thank you so much for calling that out. AJ had to meet the same bar as everyone else. Fortunately, he killed it.

vonmoltke 11y ago

> While AJ’s government work experience gave him a good amount of cred in the public sector, he found that making the move to industry, and startups especially, was near impossible. It wasn’t that he was blowing interviews. He just couldn’t get through the filter in the first place. ... > It was AJ, a candidate that Zouhair Belkoura, KeepSafe’s cofounder and CEO, readily admits he would have overlooked, had he come in through traditional channels. This was the story of my job search three years ago. It still kinda is.

pzxc 11y ago
I'm going through this right now as well. I ran my own business for 7 years, making content websites monetized through google adsense, building games, and freelancing, among other things. The primary source of revenue was adsense, and it paid the bills enough for me not to need a "real job" for years. 18 months ago, the adsense revenue started to dry up, and in an effort to make some additional income, I started looking for a normal job again. I ended up reluctantly taking a job working for the State of California, mostly because the schedule was flexible and it was a 5 minute commute. In the last 18 months I've gotten two promotions, including one four months after I started which is unheard of in state service (and which my bosses had to fight HR to get). I spearheaded the acquisition and implemented of a version control system, which no one hear had ever used (they were just FTPing files all the time, risking overwriting of others' work etc), and got buy-in from all the developers who now say they couldn't imagine working without it. I also now wear a variety of hats besides programming including sysadmin work, dba work, architecture design, etc. My bosses rely on me more and more everyday just for my opinions and advice, let alone the work I do. Yet now, when I send out my resume, it's almost always crickets. A year and a half ago, when the first item on my work experience list was "failed entrepreneurship", the response rate to my resume was about 80% (not even talking about interviews, just getting a response at all). Now it's more like 20%, all because the first item in my work experience section is my current job working in public service. I admit, many of my coworkers probably deserve the reputation that public sector work has. A significant number of them are clock-watchers that the bosses don't even try to assign anything important to, because they know they don't give a shit and can't easily be fired (union), they are just filling a chair waiting for their public pension to accumulate over 20-30 years. Nevertheless, I also list on my resume all of the above, including the pioneering (for us) work I've accomplished here implementing version control etc. I never intended to stay here more than a couple of years, but I also never intended for this job to have such a negative impact on my job prospects. I think I am probably going to revise my resume to leave this job off completely, and just say I've been working for myself for 9 years instead of 7. I predict sadly that this will return the response rate back to what it was before I had this job.

ammon 11y ago

Interesting post. Talking and writing about yourself well, even when not matched with programming well, probably helps get job offers (and in many cases helps be a good employee). It's difficult to tease these two thing apart. I imagine looking at speaking and writing can still miss great people (as can every filter), but I can believe that it's much better than looking at resumes.

Harj 11y ago

We have the same belief in the limited usefulness of resumes at Triplebyte. We found talking about projects with candidates both more enjoyable and interesting than looking at words on a resume. Especially when people are enthusiastic about what they built. The difficulty we had was not seeing a strong correlation between talking about projects and doing well at programming during an interview.

levlandau 11y ago
I think the other thing to look into is what "talking about projects" means. I'm willing to bet that there's some version of talking about projects that correlates well with programming ability. I've had to screen ~ 500 engineers and we basically use the same method aline describes as a first filter and then talk about the project. Imo it's other things like reliability, speed in real world conditions etc that are a bit harder to verify but those are hard to verify in a programming interview as well.
- Harj 11y ago
  That's definitely true. Do you have a particular set of questions you ask during project interviews? If so, which ones tend to be the most revealing?
  - tjradcliffe 11y ago
    Not the GP, but I focus on tradeoffs and roads not taken. Get them to talk about a project they've worked on, and then ask about alternative design decisions around some interesting feature and see what they say. Example: I developed a little state-machine framework for managing complexity in a large, legacy code-base. It allowed me to refactor a lot of ad hoc distributed logic into the transition table and clean up a lot of weird corner cases that made the code fragile and difficult to change. Questions might include: "Why did you write your own rather than use an existing state machine framework like the one in boost?" (for C++ frameworks there's pretty much always one in boost, so even if you don't know anything about the area you can throw this in for fun and see what they say). Also: "Why a state machine rather than some other approach to refactoring?" And so on. This process gets at taste and good judgement, it gives you a sense of how tolerant of alternatives they are, and so on. Additional edit: one of the things I look for in answers is people who say, "Yeah, that particular decision might have been a mistake... I always wondered what would have happened if instead I had..." Good developers are able to admit that not everything they do is perfect, and are willing to give alternative views a bit of credence.

codeonfire 11y ago

When people abandoned resumes like this it's because there is some corruption in their hiring process and lesser skilled management is attempting to hire down for political control. Their only goal is to go through the motions and stay employed. Some people don't want experts. They want to have the illusion of a functioning business unit. People with no formal training that have projects that sound impressive to the layperson are not going to make waves or quit when they find out that management really doesn't know what they're doing. Just as one can build a model airplane in their garage, those same people will never be able to build an airliner. It's not a good idea to hire hobbyists if you're in the airliner business. When you have layperson managers judging what is a strong candidate and what is important technology, you are going to get hobbyists skilled in popular tech and your company is going to get worthless hobbyist tech for your organization. For example, the article refers to an 'open source Android animation library.' To many people that sounds like a massive, great achievement. On the tech scale of importance it is a 0 or 1 out of ten. No business can be formed around an animation library, it's not a difficult or uncommon thing, and there are thousands of alternatives.

woah 11y ago
What are you talking about? The odds are that the company hiring started as a "worthless hobby"
codeonfire 11y ago
In a way, the entire premise of what a software engineer is has been destroyed. Imagine if mechanical engineers were hired based on their ability to design a simple bracket. Imagine if electrical engineers were hired because they amazed management by building a flashing LED circuit. After many years, the entire criteria for being an engineer is dumbed down to "are you able to make a flashing led or simple bracket or something equally impressive?" That's what the people chosen to manage are familiar with. That is what has happened to software engineering.
- Nacraile 11y ago
  I'm not sure I buy your implicit assertion that other engineering disciplines are somehow better at hiring than the software industry is. As far as I can tell, mechanical and electrical engineers are hired based on relevance of work experience, some discussion of that experience, and maybe a few specific technical details which should be known by anybody with the experience they claim to have. All of which is isomorphic to "Worked at Google, could explain his project in depth, and coded tree inversion on a whiteboard". The only difference I'm aware of is that a Mech or EE would probably be subjected to bullshit situational interviews by HR. This kind of feels like the general "software isn't rigorous like the real engineering disciplines" angst, which as far as I can tell is also bullshit. In reality, the "rigorous" old-school disciplines still manage to make colossal messes of complex, unprecedented projects, in the same way that software companies often make colossal messes of complex, unprecedented software projects. Sure, civil engineers can build normal roads and bridges and buildings reliably, but then again, software engineers have no problem throwing up CRUD apps and wordpress blogs. There are plenty of bridge collapses, exploding batteries, stalled tunnel-borers, and so on to match all of software's spectacular failures. It turns out that complicated things are really hard to build, regardless of your field.

s_q_b 11y ago

I agree we need an alternative to resume-based hiring, and the hiring process in general. For example, I don't do well in whiteboard interviews, which is odd because I normally don't have a public speaking issue. It feels like there's some muscle memory attached to coding that isn't well replicated with poor handwriting in a room full of people. Whiteboard lines of code are simply not the manner in which developers work once hired. That is the reason for the disconnect between speaking well about projects (easy to verbally explain and sketch) and the programming portion (bizarre.) Right the industry is doing the equivalent of interviewing lawyers by asking them to write a legal brief on a white board. We're testing the wrong thing: a proxy for the work, when we we could easily test the work itself. I much prefer work sample tests rather than whiteboard Q/A as it better replicates the actual job. Give me a few hours with problems I would actually face on the job, my dev environment, internet access, and a set of problems that truly reflect the work, and I find it much more natural. Is it too much to ask that an interview measure skills the job actually requires, in an environment that emulates the work?

nickpsecurity 11y ago

Great article. A good move. I think one of the reasons the method is successful is that it asks applicants to keep it real. The traditional channels want people to come off a certain way. People also know about their filtering rate. So, the incentive for them is to tell companies what they want to hear and in a way that conveys unreliable information. Seems your example changed the incentives, got useful information in return, and that led to a positive result. Unsurprising in hindsight. I'm going to send your article to a few people to see if I can get any to try that approach.

a-dub 11y ago

Resumes are fine, it's really in how you treat them. When I read resumes I generally ignore where people have worked and gone to school and instead look for what they have done. If there's either a good match between the general type of stuff they've done in the past and what the role is, or if there's stuff on there that is interesting enough such that I'd enjoy hearing about it, I give a thumbs up. When I interview, I tend to spend most of the time asking in depth questions about the projects I find most interesting on the resume. What was easy? What was hard? X sounds like it would be a problem, how did you solve it? What was fun? What was headbangonthewall miserable? Generally this gives a sense as to whether or not there's any bullshitting going on, and gives a sense for whether or not the candidate has a good head for thinking about hard problems. Finally, I'll ask a few questions to probe for "difficult-to-work-with" red flags and finish with a few fairly easy "technical challenges" that offer opportunity for the candidate to either walk about having solved the problem, or walk away having solved the problem and demonstrated understanding of the solution from top to bottom.

__z 11y ago
What kind of "difficult-to-work-with" questions do you ask? I'd like to filter out these people but I'm not sure what sorts of questions to ask to suss that out.
- a-dub 11y ago
  Basically I goad them into complaining about past bad work experiences and then pay close attention for subtle clues that may indicate that they are systemically disrespectful or unwilling to compromise on things that don't seem worth fighting for. Were they bothered by people, situations or outcomes? Tell me about the most frustrating time when you needed a thing, or consensus on a thing, and you had to go through way too much to get it. Tell me about the most frustrating time when you needed a thing, or consensus on a thing, and no matter how hard you pushed, you never got it. And, of course: The fastest way to be shown the door around here is to be an agitator of your colleagues on the grounds of race, sex, religion or any other attribute that has little to do with work. Nobody here is in the business of policing behavior and nobody wants to be. This requires perhaps more discipline than other places as the hammer falls harder and quicker here if things go awry, so it demands either heightened discretion or a heightened sense of self awareness and awareness of those around you. Do you think you would be able to work under such conditions?

dropit_sphere 11y ago

Is it just me, or does this still seem like a crazy hard problem? There were still 415 not-resumes to wade through for one offer.

tikhonj 11y ago

First of all, I definitely agree that resumes are woefully overused and overrated. They have structural problems and incentivize people to use "negative selection" criteria where people are eliminated based on not having a specific feature rather than selected for excelling on something¹. This article neatly demonstrates that resumes are not necessary and that not using them can unlock new sorts of candidates. However, I don't think there's a conclusion to be made about the actual method used here. I suspect that it worked because it was different, not because it carried a fundamentally strong signal. If everyone did this, project descriptions would be gamed even more than resumes—it would select for people who prepared for the selection process² more than anything else. This reminds me of various captcha strategies I've seen used by small forums to great effect—solving some math, typing a word into a text box, choosing a popular character's picture… etc. They all work, perfectly. But only because spammers don't care about the small fry: it's not worth their time to modify their bots for your little site. If any given captcha becomes used widely—or your forum grows big enough—they will bypass it trivially. Now, an essay like this isn't quite as bad as a captcha, but the idea is the same: it works because it's new and different. If everybody used it, it would probably be a step back. Ultimately, I think the real moral is that more companies should do their own thing, even if that thing is not great in the abstract. Being different carries a value of its own, and it breeds biodiversity that's healthy for the system as a whole. (Of course, many of the things companies try are really bad for various reasons, but that's a different story…) ¹ In particular, most people have a bunch of "red flags" they look for with, at best, cursory rationale—everything from passing on people who didn't go to the right school to those who have breaks in their work history, based on "common sense" or "experience" rather than anything meaningful. Most of these criteria seem counter-productive. ² I also think this is really true for college admissions and especially the admissions essay. A project blurb for hiring is more or less the same idea in a new context.

function_seven 11y ago
> This reminds me of various captcha strategies I've seen used by small forums to great effect—solving some math, typing a word into a text box, choosing a popular character's picture… etc. They all work, perfectly. But only because spammers don't care about the small fry: it's not worth their time to modify their bots for your little site. If any given captcha becomes used widely—or your forum grows big enough—they will bypass it trivially. Which reminds me of Jeff Atwood's original "captcha"[1] on the Coding Horror blog. It was a static image of the word "orange" every time. It was the most trivially defeatable captcha ever, but he didn't care because it worked. It would have been senseless for him to invest time and effort into implementing a complex captcha engine, when the existing solution was filtering spam bots just fine. [1] http://blog.codinghorror.com/captcha-is-dead-long-live-captcha/ http://blog.codinghorror.com/captcha-is-dead-long-live-captc...

roymurdock 11y ago

So, essentially the company that was hiring (KeepSafe), allowed applicants to submit highly flexible cover letters, and then they actually read the cover letters. It would be great if the majority of companies used both the resume and the cover letter effectively. It feels like most companies that require a cover letter only do so to screen out the laziest 10% who can't be bothered to write up a generic 1 page essay filled with ass-kissing and vague jargon. The cover letter is just a relic from the olden days when the application was slower and more formal. There were less applicants for each position so HR probably had more time to read/screen. This study presents an interesting alternative: Let people submit some text along with their resume on any topic of any length, and see how their personality comes through in the writing. Probably wouldn't work extremely well at a large company, but it seems like it served KeepSafe quite well.

autokad 11y ago
like the above commented, people would game the text, hire coaches, etc. I don't think there is anything fundamentally wrong with the cover letter / resume process. i just don think companies use resources properly at hand. I am biased, i think most HR are useless and most tech people aren't great at picking candidates either. they seem to have their own biases that they cant get over (for example: putting too much weight on technical skills and none on soft skills). honestly, if i were a company i'd send select employees to school. either take classes or better yet teach a class. you'd get a semester long interview process and the employee would get something out of it.

mpenn 11y ago

The writer claims to "rely heavily on data," but the punchline of the article is purely anecdotal that 1 person at 1 company got hired, was good and would have been overlooked. I am sure there were also many candidates with good resumes who were now overlooked. This article starts out with an air of science and ends with a completely unproven conclusion. While I do agree in my gut that resumes are not an amazing filter, she has completely failed to present evidence that her alternative interview process is better. And in fact, while KeepSafe still has the no resumes option open, they are now accepting resumes again -- I do not great confidence that the alternative system was anything more than a PR move by the company.

roneesh 11y ago
Shortly after she says she's come to "rely heavily on data", she says "This post, however, is going to be a bit of a departure. Rather than making broad, sweeping conclusions based on a lot of data points, I’m going to narrow in on one story that happened". I think she did a great job of doing exactly what she set out to do, and since this is just one anecdote, any qualitative or numerical data she presents won't be worth much, all the more reason to omit it and just share the story.
- mpenn 11y ago
  My bad on that -- I definitely misread the beginning. After rereading, she presents the beginning and end fairly based on an anecdote and not data. I'd still love to know why the company didn't switch to this full throttle.

wambotron 11y ago

I worked at two places where people had resumegasms over "brand name" schools. Neither of them hired anyone who was ever any better than completely average (and yes, I include myself). I don't really care where anyone went to school. It doesn't mean anything. Really, going to school at all doesn't mean much. I need to see what you've done outside of that to make any meaningful evaluation. It doesn't matter if it's a huge project. You can give me a couple 10-line things that do something useful and I'll still get to see how you name things, format code, use built-in libraries, etc. Then we can chit chat about project management and how much you love or hate it.

caublestone 11y ago

We use an alternative screening process where we prompt prospective interviewees with 2 extreme customer service complaints (this product sucks, it didn't arrive on time, i hear it's poison etc.), ask them to provide an answer directed to the customer and create a plan to prevent the issue from happening in the future. It's fascinating to see how people approach the problem of making people happy now and preventing dissatisfaction down the road. If they pass (1/10 do), we use a 30 minute phone call to identify interests and motivations which ends up being the strongest indicator of value add. This might work for B2C companies only but I'd love to see any company identify people that are truly passionate about making people happy. Edit: Clarifying that this is for non-engineering roles.

kansface 11y ago
I don't suspect strong correlation between the desire to make customers happy and productivity writing software - maybe you can tell me if my intuition is correct.
- caublestone 11y ago
  Ah, I meant to say we do this for non-engineering roles. For engineering we provide coding challenges as a pre-screen.

sjtgraham 11y ago

I screened and interviewed a lot of developers in my last position and it stood out to me that résumé quality seemed to be inversely correlated with the candidate's actual ability. I distinctly remember the absolute best developers that I hired also had the most atrociously bad résumés. The candidates with résumés that literally almost knocked me off my chair failed miserably at the most basic programming task.

dexterdog 11y ago
That's largely because so many diamonds get jobs thrown at them and wind up in the "I haven't updated my resume. No worries, just send what you have." They get the interview, knock it out of the park and don't bother to update it.
vonmoltke 11y ago
> I distinctly remember the absolute best developers that I hired also had the most atrociously bad résumés. The candidates with résumés that literally almost knocked me off my chair failed miserably at the most basic programming task. Comments like this make me wonder if one reason for the ridiculously high failure rate in programming interviews is that the resume screening stage is actually acting as a negative filter, that is disproportionately removing the candidates you want and passing the ones you don't.
rokhayakebe 11y ago
Frankly I cannot see an expert in anything other than marketing sitting down and crafting their resume. "Appearance starts where performance ends." Let me illustrate: Growth Engineer One line Resume: "I was employee 7 at Snapchat when we had 6 engineers and 400,000 users. During my tenure as the only growth engineer, our userbase grew to 10 million users over the next 6 months." SEO resume: "I joined XYZ when it was ranking at page 10 for major industry keywords. 9 months later. Google the following keyword BDHDUYD, which accounts for 40% of your market. If you find the company in the top 3 results, we should schedule an interview." Software engineer: I do not know, but I am sure you can insert a short paragraph here.
- 11y ago

AndyNemmity 11y ago

I just went through a job search, and never created a resume. I only submitted my linkedin. If anyone required a resume, I immediately responded that we weren't a good culture fit. Worked out really well. Not sure it's to be duplicated, but for me it went fantastically.

puredemo 11y ago
What if I've never had a linkedin because of all their repeated data security issues? Your comment sort of sounds like a shill post, fyi.
suyash 11y ago
LinkedIn is basically an overblown resume, so that doesn't really make a different, try without LinkedIn and resume and see how many people bothering to contact you back.
noarchy 11y ago
It often happens that recruiters will contact me on LinkedIn, on which my resume is essentially laid out in some detail, and then still demand a resume. I assume they have to enter an old-style resume into some sort of internal database, as per company policy.

deleted 11y ago

[deleted]

sytelus 11y ago

You can do much more easier, automated filtering: Ask candidate to submit link to any of the followings: 1. Github a/c 2. StackOverflow a/c 3. Their blog 4. Anything they made online If candidate fails to submit link for any of above then just don't interview them. I would guesstimate this simple check filters out 70% of the junk resume and probably 20% of the good resumes. It can scale like crazy and expanded even more (for example, use APIs to get their profile information and rank resumes).

istvan__ 11y ago
Also I would ask them to create it as a pull request on Github push it to a branch in a private repo. Another 10% of candidates gone. (There might be overlap with the guys who don't have Github presence and anything online but you get my point.)
sliverstorm 11y ago
If candidate fails to submit link for any of above then just don't interview them. Since when has every person worth hiring had their own blog? Edit: sorry, your use of "any" was ambiguous, I read it as "if missing any of the above".
throwaway12309 11y ago
* 20+ years of development experience 15 of those professionally * published author of various books * worked in systems that most startups would shit their pants with the requirements * some of the big name companies have tried to poach me based on the products I've worked * used professionally: C, C++, F#, C#, Ruby(Rails), Swift, Obj-C, Kotlin, Haskell, Scala I would never be called based on that criteria as I have no inclination to spend my free time doing stupid shit online for hipster new developers that think GitHub is the end-all. ps: to be fair, I probably wouldn't want to work in a company that has this mentality, so maybe that really does work
- Cyph0n 11y ago
  So you're saying that the people who spend their free time developing projects that you most probably use in some form or another are doing “stupid shit”. I wouldn't hire you either.

sopooneo 11y ago

All these discussion of how hiring is broken. Doesn't this imply an enormous market opportunity for the recruiting agencies or companies that can properly capitalize on either undervalued candidates or better knowledge?

shanemhansen 11y ago
Recruiting companies aren't in the business of finding good candidates, they are in the business of finding candidates their clients like. Companies have the same problem the military does. They hire a certain stereotype because they've always hired a certain stereotype and the people in charge match that stereotype. If there's any opportunity for disruption, it would be at tiny companies where there are people that 1) just need talent and 2) know that they process that made them successful is broken.
jerf 11y ago
Alternatively, it implies that there is no solution. You can obviously not make stupid mistakes like taking two months to call a candidate back, but beyond the basic competence there may be an irreducible element of rolling dice. It may be the case that evaluating humans is fundamentally difficult. I'm not advocating for that. I could argue either side of it without any cognitive dissonance. Just pointing out the additional possibility.

paulgayham 11y ago

For what position? How can you judge candidates without a position to judge how well they fit? I can just see this guy going out to Web devs, System devs, DBAs etc. and them all disagreeing because they're looking for different things (and value things differently).

pbreit 11y ago

What surprises me is how many people dislike cover letters. When I go through applicants, a decent cover letter demonstrating some enthusiasm about the company and a unique point or two is appealing.

deleted 11y ago
[deleted]
deleted 11y ago
[deleted]
deleted 11y ago
[deleted]
m3talridl3y 11y ago
It surprises me how many employers think that any of their engineers give a crap about the actual business. The better ones have some form of passion for creating good code, the actual business is an implementation detail.
- pbreit 11y ago
  I guess I've only worked for businesses that I gave a crap about.
- vonmoltke 11y ago
  The domain, product, or use case is what draws me in. But then, I'm of the "classic" engineering mindset; a maker. Code is a means to an end, not an end in itself.

ageofwant 11y ago

This is very bad news. I have always highly valued resumes as a very effective candidate filter. It works as follows: I take the pile of resumes, shuffle them thoroughly an divide roughly in half. The pile to the left goes in the bin. I repeat this process until I have the luckiest candidate's resume in my hand. This is the type of guy I want to associate with: one on whom fortune smiles, repeatedly.

bsder 11y ago

> His GitHub, full of projects spanning everything from a Python SHA-1 implementation to a tongue-in-cheek “What should I call my bro?” bromanteau generator, hinted at a different story, but most people never got there. While AJ’s government work experience gave him a good amount of cred in the public sector, he found that making the move to industry, and startups especially, was near impossible. Huh? The companies this woman hires for don't look at github? Not looking at public code that someone has published is more broken than relying on resumes. If someone has published code and it doesn't suck, I'll probably bring them in for an on-site, period. I may even tell them that "We're going to talk about "file foo.c in your code where you implemented feature Z. So be prepared." And, I suspect with startups it was more a case of "How many years were you in government? That would makes us so unhappy that we would leave. Why didn't you?" That's a different way of asking "Is this really the place for you?" As a hiring manager in a startup, when I knew I only had 9 months of runway without more funding, I'd feel REALLY bad about taking someone with a family away from their very stable job. As someone who has recruited employee single digit, I often have made a point to meet the family when recruiting someone--even if I have to fly to them. I need both the prospective employee and their partner to understand that the big probability is that the company won't be around in 24 months, there won't be any payoff, and a new employment search is likely to be the result. Yeah, there is a small probability that we'll survive and an even smaller probability that we'll get some money. It's a really delicate balance for me, at least, to properly sell the company (Startup! Options! Novel!) and reality (Bankrupt! Flameout! Layoffs!). I'd say I'm batting about 50%. For every employee I scare off, I absolutely convince one to join. Funnily enough, every single one who didn't run away said the same thing: "My wife told me I had to work with you." They were stunned that someone so important (Hah! Management in a startup is a good way to understand how unimportant you are really quickly ...) would take the time to make sure the family was informed properly about the risks and rewards.

andrewstuart 11y ago

Everyone should just give up on trying every sort of new angle to "identify great developers". It's purely subjective, there is no meaningful universal definition. It comes down to whether or not the people doing the recruiting all have the same subjective opinion.

dzhiurgis 11y ago

So you've turned down 399 people to employ 1. Not sure if that is any different when employing via resume, but that sends shivers to my spine. 399:1 ratio says to me that there is oversupply of engineers.

dsymonds 11y ago
It means there's an oversupply of people who want engineering jobs. Most people applying for software engineering positions aren't able to program even the most basic of things. That doesn't mean there's too many qualified engineers.

dools 11y ago

I do something very similar in my hiring processes on odesk. I ask each candidates opinion on something (articles usually). This allows me to eliminate 90% of applicants. Then I review the 10 applicants who's answers weren't complete gibberish and give the best 2 or 3 a programming task. Works pretty well, I've only ired one guy turned out to be inadequate for the role (out of about 20 hires over the past 4 years for php dev work).

7402 11y ago

I've always found resumes to be quite useful in figuring out who to bring in for an interview - BUT, most of the people I've been involved in hiring have had between 5 and 15 years of experience, usually at at least 2 or more real companies. I can well believe that if you're only interested in people right out of school, then it's harder to figure out candidates from their resumes. I am a little puzzled, though, about why others seem to find resumes so opaque. It seems like resume-reading is a lost art. A resume is usually a document that someone has spent a lot of effort on to make themselves look good. If you learn to read them, that can tell you a lot about the author. (Note: searching for buzzwords is not "reading.") A resume should not be regarded as simply a collection of facts - of course you'll be misled if you do that; a resume should be regarded as a document of self-expression. After a while, you can see useful patterns in what people put in resumes - a least for more-experienced applicants. Almost every resume suggests a bunch of next questions, which can be asked in a phone screen or interview to get a pretty good idea of what a person is about. It's worth recalling that absolutely all software engineers at all software companies in the world from the first ones around 1955 up to 2002 were hired without benefit of LinkedIn, StackOverflow, Github. Almost all of these engineers submitted resumes, which were reviewed prior to offering interviews. Yes, there were hiring mistakes in the old days, but I don't see a huge number of people taking about how the hiring process now is so much easier, smoother and more foolproof than it used to be.

tempestn 11y ago

> Resumes don’t have an explicit section for building rockets or Minecraft servers, and even if you stick it somewhere in “personal projects”, that’s not where the reader’s eye will go. That's true, but a good cover letter can go a long way toward helping with this. Most cover letters are generic, bland, and obviously copy-pasted from a template. (Or more often from a previous application, sometimes with info about the previous company left in!) A cover letter that talks about something exciting you've done recently, and ideally how it might be related to the job, or even just how it demonstrates skills you'll use in the job (and describes exactly how), is awesome in comparison. A letter like that would absolutely get you an interview with me, almost regardless of experience. One of our current co-op students actually had almost no programming experience on paper; he had actually switched out of a theatre degree iirc. But his cover letter was awesome (the theatre degree probably not being coincidental). Got him the interview, which got him the job, and I haven't regretted it. Just a co-op of course, but the point stands. The cover letter is probably the most important part of your application. Take the time to write a good one.

lordnacho 11y ago

The "new hiring process" experiment comes up here quite often, and I do appreciate it. However, how can we conclude anything from a procedure that only examines the hired population and none of the unhired?

fsk 11y ago
The only way you can truly be sure of your hiring process is to pick a random sample of people who fail, and hire them anyway. I see a couple of issues with this: - Could it get you in legal trouble? - Are you capable of evaluating who are the best employees AFTER they are hired? - Only a large corporation would have the resources to gamble on hiring a random sample of people who failed their interview process. Here's one half-hearted way to do it. Pick a random sample of resumes you reject, and give them phone interviews anyway. Pick a random sample of people who fail your phone interview, and give them an on-site interview anyway.

Mimu 11y ago

I fail to see how someone building games at 14 or own one of the most successful Minecraft server could not pass the resume filter. Not saying the process described in the article is bad, even though I believe anything can be gamed, but I don't really see a big difference. Main change is the way recruiters looked at what they got, resume or essay wouldn't have change a lot I think. Maybe off topic but if companies want the best people, maybe THEY should write the essay explaining why people should join instead of sitting in their high tower waiting for minions to come.

vultour 11y ago

Something else piqued my interest in this article. This lady claims that the company is fighting for candidates with Google, although the only thing they do (if i read it right) is provide an encrypted version of Dropbox. How does this require world class engineers? I've coded a file syncing app quite fast as a personal project once, and I don't think I could call myself even a regular developer. I do not believe such application would be even remotely as complex as anything Google does.

chris_wot 11y ago

My submission would be: I was a help desk pleb at a well known inkjet/scanner/camera company 15 years ago and this company "extended" their clipper database to record third party cartridges, but recorded them in .ini format. That's right, one file per record, in key=value pairs. I was bored and accidentally mentioned to the guy whose job it was to copy and paste the data from each of all 90,000 ini files into an Excel spreadsheet that Perl could do it, and I'd even use references to hashes to do it. He had no idea about that last bit, but I did it for him on the proviso that he didn't tell anyone, and reduced 10 weeks of work to 30 seconds. They unfortunately made me employee of the quarter but neglected to tell me so I missed my awards ceremony.

moises_silva 11y ago

GapJumpers seems like a good alternative to resume screening: https://www.gapjumpers.me/ https://www.gapjumpers.me/ Even if not using GapJumpers itself, you can follow the concept by requesting solving a problem or submitting a piece of original technical content along with the resume.

ilaksh 11y ago

I think allowing people to submit a link to a demo or portfolio makes a lot of sense, especially since everyone has computers and the internet now. Welcome to the amazing new world of hyperlinked multimedia, hiring people.