11 ms·
Real-life tests are THE best thing to send job candidates. It scales well (you don't have to spend personal hours on them) and you get real information. This a
by davidgerard 12y ago
Real-life tests are THE best thing to send job candidates. It scales well (you don't have to spend personal hours on them) and you get real information.
This applies even to sysadmins. We have a favourite: set up a VM with a slightly-broken application in a slightly-broken Apache and Tomcat, and get them to ssh in and document the process of fixing it. Even people who aren't a full bottle on Tomcat will give useful information, because we get an insight into their thought processes. I recommend this to all.
(I note we've just done a round of interviews where we get a nice-looking CV and conduct a technical grilling. Hideous waste of time for everyone involved. All CVs should be regarded, on the balance of probabilities, as works of fiction. Do a remote self-paced test like this. You won't regret it.)
- deleted 12y ago[deleted]
- josh33 12y agoThe risk is if someone publishes a tutorial with the secrets on the web.
- peterwaller 12y agoBut presumably that happens some time down the road long after you have made your decision. Either that, or said secrets are published and heard about by an audience which are tuned to hear about interesting things quickly, who may also be what you're after.
- davidgerard 12y agoIn this case, being able to Google how to fix a slightly-misconfigured Apache and Tomcat counts as passing the test with flying colours. Becoming a Google-Certified [whatever] Engineer at the drop of a hat is something we regard as a serious positive in a sysadmin. Open book tests are never easy.
- fsniper 12y agoSomeone still calling sysadmins instead of devops in HackerNews is a rare thing. :) I know many so called sysadmins who could not pass this test in a reasonable time. Sometimes people without even basic script-fu or networking knowledge are considered sysadmins. And this drives me crazy.
- davidgerard 12y agoYeah. I'm thinking of Joel Spolsky's 199/200 who are just the same bozos cycling through interviews until someone hires them by accident ... "devops" is fighting words in our team. Damn, if I could get our devs who'd like "devops" on their CV to give a hoot about the "-ops" half of that buzzword ... when a dev shows awareness of ops issues with their shiny new thing, I make a point of mentioning their name positively to the dev manager. "X knows their stuff. Give them more good stuff."
- fsniper 12y ago:) When devs are in the loop it always comes to "working in localhost" debate. I like to say developers are working or living in their isolated bubble. And whenever their code comes out of that bubble and reach the real world ops are on fire. Always the ops guys are in fault. I think every developer should be given sysadmin 101 and 102 lessons.
- davidgerard 12y agoI'd just like to note here that the sysadmins know damn well when a dev tries to blame them for something the dev did ... and we NEVER. FORGET. So don't do that. I'm not sure how to educate devs to think ops. Perhaps have their phone ring at 3am when stuff breaks? I think that would close the feedback loop nicely. (Let me note again: I LOVE the devs who think operational issues, who think end-user issues - what customers are actually like - who think "developer of the whole thing from go to whoa" and not just "coder on my PC." As I'd hope we all do here.)
- psychometry 12y agoIt's an effective strategy from the employer's side, but what about the applicant who is asked to spend hours on some test for every job he/she applies for? Are you paying these applicants for the time they spend jumping through these hoops? And if your "technical grilling" fails to identify strong candidates, you're obviously asking the wrong questions or, at the very least, not asking the right ones.
- sillysaurus3 12y agoI would rather spend hours on some test than be judged on a whiteboard problem, or any other problem which has nothing to do with programming on an actual computer that has an internet connection.
- vault11th 12y agoNobody puts a gun to applicant's head and forces him to take a test, so... Business is not for people to have jobs but to make profit.
- kstenerud 12y agoThe point is that you'll only get applicants up to a certain level. Once an applicant has enough skill and knows it, and also values his time, he'll take one look at your 14 hour challenge and laugh. So in the end you're only hurting yourself because you'll mostly get naive and easily manipulated people who are fine for the lower ranks but absolute poison once they move up. That's if they don't wise up and leave first, costing you time and effort to find and train replacements and scramble to rediscover and distribute the lost knowledge they left with. But then again, such a long duration challenge is already very telling of the company culture. If you can't respect other peoples' time, then your company is already on shaky footing.
- vault11th 12y agoBut the process described in the post above is clearly about proficiency, so a challenge is not supposed to last 14 hours. ;) And I admit you have a point. The thing is, people really have well-looking CVs and they can't code BFS in Python. Probably balancing of 'technical grilling' is a hard to master skill in and of itself.
- B5geek 12y agoA nice simple test that I like to give as a first-wave elimination: Have a computer setup and running (all properly configured). Pull the network cable out of the jack a little bit (so it looks like it's plugged in but isn't). Ask the person being interviewed to show me an IP being used by microsoft or google. (so ping/dig/nslookup/etc) Let the person know that (a) the computer is in working condition (i.e. no drivers are missing) (b) the network works (i.e cables are good, switch is good, DHCP is enabled, etc.) (c) tell them that this is a test to determine their troubleshooting skills It is always disappointing to see how few ever open a term/cmd window (depending on the OS). 90% of participants just try to open a web-browser and type in "what is the IP address of google"
- icebraining 12y agoIf I answer without touching the computer, do I pass? :) Google DNS comes in handy!
- walshemj 12y agoI know even with my poor short term memory 8.8.8.8 is memorable
- chris_mahan 12y agoWhat if they just tell you, because they've memorized the IP address of Google? You hire on the spot then?
- incision 12y agoI have mixed feelings about this sort of thing. As an interviewer I've done something like this when needing to hire for a very straightforward job with very little time and an effectively endless supply of candidates. This addressed the problem of our time crunch, but I doubt it helped the selection process overall. I think this sort of testing is subject to all the same issues as most any other method of interviewing, but limited to a much smaller sample size. Basically, it's arguably good for the interviewers and a coin flip for everyone else. As an interviewee, I haven't encountered a test like this, but I expect my response would almost surely involve questioning the architecture decisions that led to such a problem in the first place.
- davidgerard 12y agoCertainly, as user:psychometry pointed out also. In this case, as a sysadmin I was pleased to be able to show my chops in a realistic test. And our previous hire was also quite pleased to get a real test. But of course, there is potential to abuse it.
- Tarrosion 12y agoSeems like this sort of test should be very effective at finding people who are already reasonably good at the job you're hiring for, but gives you little to no information about how quickly they learn, their willingness to try new things, etc. Given how fast things change, even if you imagine the person staying for only a year or three, learning abilities are nontrivial.
- deleted 12y ago[deleted]
- davidgerard 12y agoTotally. YMMV. That's why you look for stuff on the CV like willingness to experiment, play with OSes at home, "describe your home network", etc. This doesn't replace the interview, it just verifies claimed competence and helps show thinking ability.
- UrMomReadsHN 12y ago"Describe your home network" is an interview question now? Jeez, guess I shouldn't leave my job, I'll never get another one. I am going to go ahead and challenge the assumption that your willingness and eagerness to learn on the job is not really correlated with how you spend your time outside your working hours.
- davidgerard 12y agoI mean as extra points that help the candidate look interesting. It's like the "other interests" section on a CV: you should treat it as an opportunity to market yourself and add a hook to catch the interviewer's interest.
- jpindar 12y agoI think it's an excellent question. Not because a candidate needs to have something special about their network (although if they do, that's a plus and a chance for them to talk about their interests) but because if they can't describe something they own, why should anyone think they will have the communications skills to describe something they're going to be working on?
- michaelchisari 12y ago> Real-life tests are THE best thing to send job candidates I agree, but only if you're allowed to use references/google/etc and given a reasonable amount of time to accomplish it. I've had a "real-life" test where I wasn't allowed to verify or look up information, or where I'm giving a very short time to execute, and I've always thought those were absurd. The idea that you would ever have to do something like this in a bomb-diffusing type of scenario with no internet and a ticking clock, would only test if you have already solved the problem before, not whether you could investigate a new problem.
- davidgerard 12y agoYeah, this is an open-book test. Of course, there's tests like user:B5Geek's, where the candidate gets a PC with a slightly-unplugged network cable and has to realise this fact.
- michaelchisari 12y agoYeah, but B5Geek's is a pretty entry-level thing. I've encountered whiteboard tests where you have to recreate an algorithm that someone spent their PhD thesis creating. It seems like a good way to find "geniuses", but it just isn't practical. In a lot of ways, it's a matter of identity crisis for software developers. We're all, industry included, not quite sure whether we're mechanics, carpenters, architects or scientists.
- davidgerard 12y agoTrue. Testing for sysadmins is easier: you're after a way of thinking. So a competence test ("can you do what you claimed?") with a freeform "keep notes on your thinking along the way" is quite informative. You still need to interview, of course. I confess I don't know how I'd apply this to developers. They pass fizzbuzz, OK - what do you do next?
- michaelchisari 12y agoFor developers that have the academics, but not the experience, I've always said that the industry needs to adopt the apprenticeship programs that other trades have. Where you do a mixture of work for the company, and trade schooling run by the industry to teach you the specific skills you need. For example, an IBEW apprentice will do a lot of gruntwork on the job site. Pulling wire, for instance. But they'll get exposed to how things are done, to trade industry expectations, and what a project looks like from start to end. Part of the week, they spend in a classroom learning the specifics of their trade. Not just the high level theory, but the nitty gritty, based on real world applications and problems. This apprenticeship lasts from 2-5 years depending. I don't see why this model wouldn't work really well with programming.
- Dove 12y agoI love this approach. I am good at what I do, and appreciate the opportunity to prove it directly. Spending a couple hours solving a technical problem or writing an essay is much easier for me, not to mention sending a more reliable signal, than trying to figure out what networks / buzzwords / whatnot various people in an opaque organization are going to want to see. Not to mention, it communicates to me that the people doing the hiring know what they want and how to get it, which is encouraging. I believe there is a happy synergy here, by the way. A candidate who welcomes this type of test -- who sees a direct work sample as the easy and efficient way to prove themselves to you -- is probably someone who knows their work is good.
- davidgerard 12y agoIt depends on the job (see elsewhere in this thread), but if it works then it works really well.
- korzun 12y ago> Set up a VM with a slightly-broken application in a slightly-broken Apache and Tomcat, and get them to ssh in and document the process of fixing it. That might work on college students. To me that screams that you can't properly interview your candidates. > I note we've just done a round of interviews where we get a nice-looking CV and conduct a technical grilling. You need a better screening process and a better layered.
- agumonkey 12y agoI remember a website providing failing VMs for this purpose. I'm not even sure it wasn't just a suggestion .. but if someone knows such a site, feel free to leave the url.