3 ms·
At my place of work we have a coding test that we issue to all applicants for the software engineering roles. Here are a few responses from our point of view...
by danpalmer 11y ago
At my place of work we have a coding test that we issue to all applicants for the software engineering roles. Here are a few responses from our point of view...
1. We know you can probably code, you've already passed a phone conversation about your coding experience, but you may not have a GitHub profile (you might not be allowed to at your current place of work, or you might have experienced harassment through GitHub), so we're not going to penalise you for that.
2. We want to see your solution to a problem that we know REALLY well. We know the pitfalls, the places people often trip up, we know how to tell if your solution is going to be maintainable over the long term. That's why we want to see your solution to our problem, not just 'portfolio' code, or even a 'bugfix'. Not to mention the fact that the bugfix should be already done by the team, finding tasks that are low enough priority that we can give them to someone outside the team is actually difficult to do.
3. Our coding test is quite long (3 hours), but roughly capped, we ask candidates not to take longer than this. This is significantly less commitment for most than taking a day off work for a day of working/pair programming/etc. We do pair program later in the interview process, but not until we're pretty sure that bringing the candidate on-site isn't a waste of their time, and ours.
I realise that there are some devs who reject the idea of coding tests, but we have found it (in the form we do it) to provide some of the highest signal to noise ratio information in the interview process, and find it to be a really good balance of time commitments on both sides. Very few candidates drop out because of it.
- onion2k 11y agoWe want to see your solution to a problem that we know REALLY well. Does that give you an expectation of the sorts of things that someone new to the problem will spot? If so, how do you manage that expectation - eg if the user misses something in their 3 hour test that you only spotted after using the same problem for 6 months, how can you ignore the fact they still missed something?
- danpalmer 11y agoMissing something doesn't rule them out, nothing in the test is a hard failure. That said, the candidates that really shine are the ones who do spot the things that others don't. It allows for the really good candidates to differentiate themselves. Mostly though, we're focusing on the assumptions that the candidate is making about the problem, and the data structures they use to solve it, we just wouldn't be able to analyse and compare to the same extent between two pieces of portfolio code from two different candidates GitHub profiles.
- taurath 11y agoSounds like the ideal candidate or at least the one most likely to succeed is someone who has seen the problem or a version of it before. Hope your other problems are in the same class.
- danpalmer 11y agoThe problem is sufficiently random and fictional that people won't have seen it specifically before, and a lot of the focus is on code readability and maintainability, and I don't think that's something you can fake having seen a problem before.
- tossaway1 11y agoAs an interviewer you shouldn't compare a candidate's answers/solutions to what you think you would have done, you should conpare them to other candidates. This is the main reason I have a standard set of questions I make sure to cover in an interview -- not because they're awesome questions but because they allow me to "callibrate" my evaluation.
- akvadrako 11y agoDo you have any evidence, like a study, to back up those claims of effectiveness? I'm sure you can find someone claiming any sort of useless hiring process works for them. I mean, how do you even know how things would have turned out with an alternative process?
- danpalmer 11y agoI'm not sure what you want a study for, I assume you want a study about the effectiveness of coding tests as a measure of ability, but that's not the point I was making. My points were addressing the rejection from the author of the blog post. I feel we get a large amount of information from coding tests, but it is only a part of a larger interview process that covers communication abilities, and many other important aspects, so it's very difficult to test in isolation. I don't have data to back up that claim of effectiveness, but I have nearly 2 years of seeing this particular hiring process (and seeing the 'borderline' test candidates come in and always fail the interviews - tests are anonymous, we only find out after interviews). Our CTO has quite a bit more experience with this process and appears to hold similar opinions based on our conversations about it in the past.
- bojo 11y agoI have no idea who you are or what you do, but in my experience #2 up to this point tends to be "here is a github gist assignment which defines a simple REST interface, please write it and add some tests." This may be useful to weed out junior candidates with ~1-2 years experience, but those of us with 10+ it's an incredible waste of time. What exactly do you* want to see? Your homework problem doesn't even come close to covering the tools I'd use to build a highly scalable and responsive application, including but not limited to specific coding practices which offer great performance, clean API design, full blown test coverage, as well as attention to server infrastructure design and careful selection of tools which will scale with the application from 10k to 10M users (or whatever other performance characteristics your app requires). One of the more interesting interview techniques I've experienced is when they engage me in a prior/personal project and give me the chance to completely go 100% on about something I've been passionate about (maybe even sending in a code sample and doing a code/project review). It tends to be win/win because the interviewer can clearly see the true developer experience through this method without arbitrary imposed design decisions for homework-level candidates. They can typically guide the conversation towards some of the deeper gems of interest which only an experienced developer can understand and both parties can relate to at some level. This is what I feel the true essence of an interview can and should be. By all means use a homework assignment to weed out the junior programmers, especially if their resume indicates as much. But if they say they have 10 years of experience treat them like they do and give them the benefit of a doubt with a 30-45 minute phone screen which concretely proves that. I've written a publicly available open source database driver full of tests and documentation which is at some level an incredibly strong indicator of higher level programming skill, yet when submitted it's been overlooked and "please write a RESTful web app" is typically the answer. That is borderline insulting, why should I work for your company? This makes me question the point of submitting my GitHub profile as a resource at all. * The general "you" as in "people with these hiring practices." I'm not picking on danpalmer specifically.
- danpalmer 11y ago> here is a github gist assignment which defines a simple REST interface, please write it and add some tests I've seen this many times too, but I like to think ours is a little different, and has a bit of a different focus. Without giving away too much about our problem (we've had it leaked online before), it's essentially a simple simulation problem. With some relatively simple data structures knowledge, and some careful thought, it's possible to complete in the 3 hours we give. It's nothing about REST, nothing about any specific technologies, nothing specialised at all, because we want to see maintainable code, in a nice structure, that the rest of the team could pick up and work with easily - not the fact you know React or something. I'll emphasise, this really only works when we know the problem really well, and can compare solutions to those from other developers. It gives us a good feel for what the passing grade should be. We also aren't looking for things like testing tools, or deployment strategies, or infrastructure stuff, as we would typically interview about that, and it's stuff that's a little more ingrained in a team, so not something that is going to matter from day 1, it's more something that the candidate could input into the direction of in the future - their code on the other hand will run in production from day 1, so that is vitally important.
- michaelt 11y agoOur coding test is quite long (3 hours) [...] This is significantly less commitment for most than taking a day off work for a day of working/pair programming/etc. Whether that saves candidates' time depends on the stage in the recruitment pipeline when companies send the test out. If a company sends me a 3-hour test, then the hiring manager rejects my CV because I don't have enough Foo experience, they've wasted 3 hours of my time with the test as I wouldn't have been invited to interview anyway.
- LoSboccacc 11y agowe do ask for a coding exercise, but it takes from then to 20 minutes max, and it's done in the interview because we're just interested in hearing the candidate thinking about a fairly well known problem with no single correct solution (an elevator controller)
- danpalmer 11y agoWe send our 3 hour test after CV/GitHub/etc screening, and after a phone call, the next stage after it would be an on-site technical interview. I agree that sending it before the CV screen would be bad.