6 ms·
I've not seen weak candidates that got hired get much better even after extensive coaching and pairing. It may be that coaching, pairing and code reviews are s
by barrkel 10y ago
I've not seen weak candidates that got hired get much better even after extensive coaching and pairing.
It may be that coaching, pairing and code reviews are simply ineffective in teaching people. They are also very expensive, since they sap the productivity of the rest of the team, and make planning and sequencing less predictable.
Overall, in my experience it's a lot less costly to turn down a candidate that might be good than to take one on.
(FWIW, the main metric in our company's hiring after meeting a minimum bar of an online coding test (~15 minutes to solution) is pairing with them on a problem that can be solved in several different ways. Getting a key insight into how to solve the problem isn't the metric; it's communicating different approaches and seeing how much explanation is needed to get the implementation idea across and how fluent they are at turning the idea into code. It's 2 hours of working together; it is not 8 hours of trivia.)
- tptacek 10y agoI have. In fact: it became sort of a norm. Want to talk more about it? I'm sure there's a definition of "weak candidate" that perfectly captures "uncoachable". But my definition, and I think it's the common-sense definition, is that a "weak candidate" is one who gets as much negative feedback as positive (Joel Spolsky famously advocates for "if you have to ask, the answer is NO HIRE"). Most of the people we hired at Matasano for several years fell into that category, and most of them wound up outpacing the strong candidates (none of them did badly).
- Futurebot 10y agoAre there any other companies that you know of that do what you did at Matasano (offer post-hire specialist-type training to non-specialists/generalists is how I understood your posts about it)? The moneyball/trainingball idea seems like such an obvious way to get the people a company needs that one would think it would be somewhat common by now. For all the alleged shortages, can it really be that all these years later companies are still unwilling to gamble on training because they feel like people will leave too quickly?
- dagw 10y agoI know lots of the big name consulting companies in London at least used to do it. A friend of mine with a physics degree from Oxford got job offers for all kinds of things on the assumption that if you can get a physics degree from Oxford, you can probably learn other stuff as well. He ended up doing some sort of forensic accounting.
- jacalata 10y agoA friend of mine in Australia became a management consultant with a PhD in physiotherapy, they essentially were saying a PhD in anything is a good base.
- calinet6 10y agoCompanies have too incomplete a notion of the value of training and a positive environment to retention and productivity. People really do think skill is innate and people 'are who they are' more or less. It's probably not true in 80% of cases, is true in 20% of cases, and the 10% of cases where people got burned by a poor candidate who couldn't improve are the ones they remember and say "never again;" likewise the 10% of cases where they are amazed by a highly skilled candidate and think all of them should be that way. They're missing the majority effect that impacts the productivity of 80% of their workforce. This is why it's crucial to have an innate sense of human variation and psychology, a la W. Edwards Deming. So, in that sense, it's not the employees that need the training, but the managers.
- krmboya 10y agoFrom my perspective, I see that several other skill-based professions assume novice hires, and emphasize a some sort of on-the-job training as part of career progression. In software dev, as much as we have stack overflow and such where we can figure out trivia of the tools we have to work with, I guess we'll come to the realization that some kind of formalized training post-hire will be useful, especially as development becomes more specialized or domain specific.
- barrkel 10y agoI know you've talked about your case as a counterexample, but I think it's because it's not coding ability you were finding hard to get, but security and pen-testing knowledge and experience. To my mind, that's domain knowledge, and domain knowledge can definitely be taught, and much faster than how to write good code in good time. I think sending some pen-testing books to a bored and under-exercised VB programmer may well get good results; but put the same guy in front of a problem that is almost completely abstract, and if he's able to understand the solution you present to him and create code (in any language he likes; we don't care about specific language experience either) that implements the solution, then he'd get hired at my company too. Because we don't test for domain knowledge; we don't expect it. We test for ability to communicate, understand and turn concrete implementation ideas into code.
- tptacek 10y agoIt's a programming job. Matasano wasn't a netpen firm; every project involved us digesting and processing some giant blob of other people's code. Everyone on the team had to write tooling and, more importantly, quickly understand other people's code well enough to outguess the authors. I don't accept the argument that we were hiring for an easier job than "programmer"; I think we were looking for a subset of the population that ordinary software development teams look for. The bored and under-exercised VB programmer, by the way, helped found the NCC Cryptography practice, and moved on to one of the greatest crypto software development jobs in the industry. It's not my place to say exactly which job, but if you make a mental list of the 5 coolest crypto software development jobs that might exist, it's probably one of those five. The problem is everyone else runs hiring processes that assume the bored VB programmers are just VB programmers. They are wrong. Joel Spolsky was also wrong about this (despite the other nice things he said about VB).
- barrkel 10y agoOh, I agree about VB; once upon a time, a long time ago, I too was a VB programmer (before I discovered Turbo Pascal and then became aware that I found the distinction between Let and Set disturbing; I ended up loving Pascal so much I spent 5 years at Borland on the Delphi compiler). While at Borland, another engineer (Allen Bauer) shared his personal theory, that some engineers were great at reading code, more were great at writing code, and not so many were great at both. Sounds like your job required people that were very good at reading code. It intuitively feels very testable.
- dagw 10y agoI've not seen weak candidates that got hired get much better even after extensive coaching and pairing. I was very much a weak candidate when hired for my first programming job after university. No experience with shipping code or working on large code bases, virtually no experience with C++ or OpenGL/3D programming for an application where the core was doing 3D visualisation in C++, minimal relevant domain knowledge etc. etc. And I'd really like to think that not only did I get much better in the first 6 month I was there on a temp contract, but that I kept getting better in every aspect of the profession for the rest of the 3 years I was at that job. And even being on the other side of the 'table' I've seen people who could hardly code at all turn out perfectly acceptable code 6 month later. It doesn't even take "extensive coaching and pairing", just a lot of nudging and pointing in the right direction, some well timed pieces of good advice and the occasional bollocking when they're being just a bit extra dense.
- barrkel 10y agoEverybody's weak the first time they do something for reals; that's not really what I'd classify as a weak programmer, but rather junior. When interviewing a junior, ideally they'll have been programming since before college and are actually intermediate devs in disguise. But when actually hiring a confirmed junior, you look for a spark of intelligence but also go in on the understanding that (a) their potential is unknown, and (b) there are only so many juniors you can allocate across teams and maintain productivity. Juniors are a speculative investment.
- lovich 10y agoWhy is it ideal that a candidate has been coding since before they were in college? I understand wanting people who have an interest in the job, but how many other industries feel that the only good candidate is one whose been involved in it since they were a child? Sports are the only one that come readily to mind
- ern 10y agoNow that there is a push to get all kids coding in school, this criterion will have to be tightened: I wouldn't be surprised to see: "had first PR merged into a Github project before college". Of course these arbitrary criteria to filter junior programmers add no value, but are another symptom of how broken the entire business is.
- chamakits 10y agoI know this is all very anecdotal, but I certainly have seen developers that I wonder for months how they could have ever been hired, and given time become quite capable. I first started to notice with out of college developers, but eventually saw beyond that too. In my experience, the ones that were 'capable' from the get go either had a lot of experience programming on their own, or had some internship or other real world work experience. The ones less capable didn't. Then I realized that my measure of 'capable' was probably unfair, because I expected people without having worked before to be aware of all the complex details that working with other developers entail. All the annoyances you have to deal with about writing real code that needs to evolve and change that only a stable work environment provides. Then I took it one step further. What if the ones that weren't out of college were just never expected to really improve. They never had the chance to be exposes to a stable work environment that required growth. They just either were let go from previous jobs before being allowed to grow, or weren't put in an environment that neither required nor promoted growing? I think not all jobs do a good job of demanding and providing space to grow. And that's a problem. It's possible to 'get good' on your own, but it shouldn't be expected; and it's far from the only way. So maybe we are being a bit unfair when we say a candidate is weak. Maybe they just haven't been in an environment that would require and allow them to improve.
- barrkel 10y agoIn all walks of life, people with more experience are expected to be more formed, more of a known quantity, than people with less experience. The distinction between a junior candidate and a weak programmer is mostly down to the length of their CV. If someone has been programming for a few years, it's a bad sign if they're not very good at it.
- exBarrelSpoiler 10y agoYou can spend ten thousand hours doing the first hour of the same work, at a bad job. There's a lot of bullshit positions with poor management that don't exactly lead to programmer development.
- 10y ago
- ido 10y agoMy latest hire (I hired them almost 1 year ago) was very green (no job experience, didn't study CS, autodidact programmer). But I saw enough drive, motivation & willingness to learn that I gave them a shot. It has proven a very advantageous hire for both parties. I invested a lot in them and the investment quickly bore fruit. A previous person I hired (in a previous job, for a different company) had a lot more industry experience as well as appropriate education. They were quite capable but I had a slightly bad gut feeling, ended up hiring anyway cause I didn't want to discriminate against a seemingly capable candidate. Ended up being a poor fit: argumentative and not willing to accept that they do not always get to call the shots about how stuff is done. I think this was disadvantageous to both parties. In the future I will definitely prefer hiring someone showing a lot of potential than someone skilled who I suspect may not be a "team player" (for lack of a better term).