5 ms·

No one can ever "solve" the recruiting issue. We are exactly where we were 15 years ago - not one thing has changed except certain screen-share coding tests, wh

No one can ever "solve" the recruiting issue. We are exactly where we were 15 years ago - not one thing has changed except certain screen-share coding tests, which are basically the same as white boards. Can you reduce hiring an airplane engine designer to a set of tests? Or a custom home builder? If so, why is there not just a stack ranking of best ones, or an algorithmic way of hiring one? Software engineering at elite levels is an art. It is conducted by master craftsmen. On top of it the best people are often quirky / weird and the social aspect when communication of needs is so essential makes it even more difficult to hire "the best". The business models of recruiting is always either like sales (recruiting firms) or marketing (LinkedIn). I would stay away from the recruiting industry as a software startup - difficult to make any money here in a novel way.

solardev 3y ago
I wish we had industry-standard certifications that actually meant something, and were widely accepted. The closest I've seen in the real world is just "degree in comp sci". Other orgs have tried to come up with various certs, but they never seem to have any sticking power. I guess it's a hard thing to certify when the landscape itself is constantly evolving and skills are obsoleted in 3-4 years. It seems even the pace of change is itself speeding up, and every year I see more technologies come to popularity (if not quite maturity) than I can even remember, much less learn. And it's not just languages and frameworks, but entire architectures... today's best practices are tomorrow's tech debt. There should always be room for innovators to push the bleeding edge forward, but many businesses don't really need that. Instead, if there was just some shared certification for "good enough engineering" that can deliver a reliable, maintainable product somewhat on time, reproducibly, hiring and job-hunting would be so much easier. I'm so tired of trying to charm recruiters, impress managers, placate HR, all while hoping against hope that the actual dev 2 or 3 interviews later happen to have some overlap in languages, toolsets, shared pain points, etc. It's all such a dog and pony show, with many teams having ill-fitting skillsets and cultures between members because nobody really knows what they're signing up for or getting.
- monero-xmr 3y ago
  It’s simply too free-form and the industries software is used in are… all of them. I believe it is better to certify the outputs (product features, uptime SLAs, security) than the inputs and how the sausage is made. Software engineering is building a factory. The inputs are data. The outputs are data. All of the stuff inside the application are the different conveyor belts, elevators, gears, rollers, pounders, etchers, etc. to move and manipulate the data to make the output valuable. The factory can be constructed infinite ways without any care about physical dimensions, and the raw material can be duplicated to infinite amounts. The situation is beyond regulation and it should not be regulated. Only the product output should be regulated.
- arp242 3y ago
  Credentialism is not good either. And in general I've seen little correlation between credentials and quality, both in the software industry and elsewhere, so I don't think any such certifications will really solve anything either.
  - solardev 3y ago
    I think a more nuanced take on this is that very few companies need top quality, but nobody wants bottom quality. Top-end devs make a name for themselves and don't really have to worry about hiring. Companies compete for them with crazy high offers. Bottom-end devs try to make up for their lack of experience or skill by "hacking" the application or interview process somehow. In between, "good enough" devs power the world's bog-standard apps. They're not going to win any awards or lead any talks, but they can do the jobs well enough to make the company money. They know they aren't top-end and can't just say "I'm quitting, who's hiring?" on twitter, but they are also competing with the hordes of less-skilled devs because there's no easy way to tell them apart. I've never heard of "credentialism", but why doesn't it work in this instance, when it does for doctors, lawyers, engineers, etc.?
    - arp242 3y ago
      > I've never heard of "credentialism", but why doesn't it work in this instance, when it does for doctors, lawyers, engineers, etc.? There are loads of bad doctors though, and bad lawyers, and bad engineers. "Credentialism" refers to over-reliance on credentials, rather than just looking at what someone can do. I'm not against education, but education is not the same as credentials. People that can hack the interview process can also hack the credentials. I've seen people with college degrees in IT employed as programmers that literally cannot program. I'm not talking "bad programmers" or "brain fart moment", I'm talking "consistently being unable to grasp even basic control flow over a period of months or years". Also some good programmers can be hard to work with for reasons other than their programming ability.[1] Plus not every programmer is suitable for every position or every type of programming or "jibes" with every other programmer in their sensibilities and approach.[2] The hiring process is just as much about that as testing basic skills. No one thinks credentials are a guarantee of quality (people will always slip through the cracks no matter what), but I'm not sure they're even a strong indicator of quality either, certainly not for programming. This is hard to quantify of course, and just based on my own observations over my career. [1]: https://news.ycombinator.com/item?id=21585958 https://news.ycombinator.com/item?id=21585958 [2] Simple example: I'm firmly in team "short names", but I know some people really hate it, but I'm pragmatic if nothing else and don't mind doing whatever works well for the entire team. On one extreme end I've worked in a team that insisted on the longest possible name for everything: every variable had to spell out the full name of the class where it came from, often >20 chars, otherwise people considered it "unclear what anything refers to!" I once added a mkdir() function, this was considered "obscure" so it had to be "CreateDir()", then someone said it may not be clear what "dir" is so "CreateDirectory()", and then another remarked it actually makes all intermediate directories too, so we ended up with "CreateRecusiveDirectoryTree()" – I'm not saying this is "wrong", but it does not jibe well with my sensibilities. Several other things also didn't, so I didn't last too long).
xyzzy_plugh 3y ago
Indeed. I think tptacek's perspective on hiring is quite radical, certainly for established shops where doing things differently than the rest of the industry just adds friction elsewhere. My read on the approach is that it works very well for certain types of engagements, but other types, such as where you are trying to build a solid team of peers, requires something _closer_ to the traditional methods. That being said the Starfighter approach is probably right most of the time -- teams are almost certainly never stable, management usually treats ICs as fungible cogs and moves them around, in which case individual value should trump things like team fit or culture. But we're human. Appropaches like Google's work better (n.b. I am far from a proponent of what they do) because the interviewers are human and despite the whole blind-panel thing if the interviewer and interviewee can't get along the interview won't go well. So culture and team fit _is_ evaluated significantly more than something like Starfighter. People care a lot about that implicit bit, in my experience.
- tptacek 3y ago
  Fly.io is several solid teams of peers, built on resume-blind work-sample hiring.
  - xyzzy_plugh 3y ago
    For sure! But the company was built that way. An existing 1000 person corp works differently. Everyone has to be on board for it to work, I'd wager.
- wglb 3y ago
  I used this approach at a company of about 750 folks, bending HR to bypass some of their stuff. E.g., they insisted on collecting resumes, but I generally discarded them. Instead, I did a one-on-one phone call for the purpose of them to ask me questions--not to grill them, and to ask them if they would like go to the next step and do a work sample test. I tailored the work sample test to the role. I did it for the compliance hires and for the project manager. The results were unexpectedly excellent. As a result of this process, and having a solid company culture, there are around ten or twelve folks that have a new career in the security field that would likely not have had another opportunity in the classical hiring scheme. I think the state of the industry with respect to hiring and evaluating folks is broken across the whole spectrum. It is like once someone becomes established in a field, they invent all manner of gatekeeping tests that they didn't pass to get where they are. The whole story of sabermetrics is another example.
dmoy 3y ago
Yea it's a problem. You're trying to hire from a pool that includes very competent people, and also people who legitimately can't write any code at all and are just trying to sneak in for 6mo to get $100k before they get fired, or think that "fake it til you make it" is a real thing. But at the same time, unlike other engineering disciples there's no PE licensing exams (or whatever the FAA does, I forget), no equivalent degree requirements (for good reason - one can learn to code without it, and a huge portion of a CS degree is very theoretical), but at the same time is qjite difficult, nothing like a plumber/electrician/etc certification level with accompanying work requirements and an apprenticeship program, and has very high pay and demand. For very small shops willing to incur legal risk or limit the application pool, you can do take home tests and/or lean on side projects, with an hour of explaining the nitty gritty details and choices. At scale it becomes untenable, because you open yourself up for discrimination lawsuits, and you can't convince a lot of people who already have jobs and families to do lengthier take home stuff. If I had to bet, I'd bet some day it ends up like plumbing. You work as an apprentice, learn real stuff on the job, and then there's some kind of state certification which itself is a joke, but requires you to have gone through an apprenticeship program already where all the real shit happens. Companies trying to hire "the best" or whatever are still going to have problems, so as long as there's not much regulation increasing cost of dev and uniformity by an alarming degree, we'll still have insane hiring (and insane pay too, regs will make it unprofitable eventually).
- jvickers 3y ago
  > because you open yourself up for discrimination lawsuits Is that because when looking to make a possibly subjective judgement on the performance in a test and especially what a side project shows, it then becomes more difficult to prove that the judgement was not instead made because of some protected characteristic of the candidate?
  - dmoy 3y ago
    That, and a big chunk of it is kids. In the same way you can't decline to hire someone because they're pregnant or may soon become pregnant (or are the father side of that equation) - requiring side projects is a very thin/loose proxy for "doesn't have kids". It's not a big enough problem to really stop people, but a big company's lawyers will stop it internally.
tracerbulletx 3y ago
Software interviews feel more like an audition which isn't necessarily a bad approach, except we're using random people sometimes with two years of experience or who do not even want to be there as casting directors.

solardev 3y ago

I wish we had industry-standard certifications that actually meant something, and were widely accepted. The closest I've seen in the real world is just "degree in comp sci". Other orgs have tried to come up with various certs, but they never seem to have any sticking power. I guess it's a hard thing to certify when the landscape itself is constantly evolving and skills are obsoleted in 3-4 years. It seems even the pace of change is itself speeding up, and every year I see more technologies come to popularity (if not quite maturity) than I can even remember, much less learn. And it's not just languages and frameworks, but entire architectures... today's best practices are tomorrow's tech debt. There should always be room for innovators to push the bleeding edge forward, but many businesses don't really need that. Instead, if there was just some shared certification for "good enough engineering" that can deliver a reliable, maintainable product somewhat on time, reproducibly, hiring and job-hunting would be so much easier. I'm so tired of trying to charm recruiters, impress managers, placate HR, all while hoping against hope that the actual dev 2 or 3 interviews later happen to have some overlap in languages, toolsets, shared pain points, etc. It's all such a dog and pony show, with many teams having ill-fitting skillsets and cultures between members because nobody really knows what they're signing up for or getting.

monero-xmr 3y ago
It’s simply too free-form and the industries software is used in are… all of them. I believe it is better to certify the outputs (product features, uptime SLAs, security) than the inputs and how the sausage is made. Software engineering is building a factory. The inputs are data. The outputs are data. All of the stuff inside the application are the different conveyor belts, elevators, gears, rollers, pounders, etchers, etc. to move and manipulate the data to make the output valuable. The factory can be constructed infinite ways without any care about physical dimensions, and the raw material can be duplicated to infinite amounts. The situation is beyond regulation and it should not be regulated. Only the product output should be regulated.
arp242 3y ago
Credentialism is not good either. And in general I've seen little correlation between credentials and quality, both in the software industry and elsewhere, so I don't think any such certifications will really solve anything either.
- solardev 3y ago
  I think a more nuanced take on this is that very few companies need top quality, but nobody wants bottom quality. Top-end devs make a name for themselves and don't really have to worry about hiring. Companies compete for them with crazy high offers. Bottom-end devs try to make up for their lack of experience or skill by "hacking" the application or interview process somehow. In between, "good enough" devs power the world's bog-standard apps. They're not going to win any awards or lead any talks, but they can do the jobs well enough to make the company money. They know they aren't top-end and can't just say "I'm quitting, who's hiring?" on twitter, but they are also competing with the hordes of less-skilled devs because there's no easy way to tell them apart. I've never heard of "credentialism", but why doesn't it work in this instance, when it does for doctors, lawyers, engineers, etc.?
  - arp242 3y ago
    > I've never heard of "credentialism", but why doesn't it work in this instance, when it does for doctors, lawyers, engineers, etc.? There are loads of bad doctors though, and bad lawyers, and bad engineers. "Credentialism" refers to over-reliance on credentials, rather than just looking at what someone can do. I'm not against education, but education is not the same as credentials. People that can hack the interview process can also hack the credentials. I've seen people with college degrees in IT employed as programmers that literally cannot program. I'm not talking "bad programmers" or "brain fart moment", I'm talking "consistently being unable to grasp even basic control flow over a period of months or years". Also some good programmers can be hard to work with for reasons other than their programming ability.[1] Plus not every programmer is suitable for every position or every type of programming or "jibes" with every other programmer in their sensibilities and approach.[2] The hiring process is just as much about that as testing basic skills. No one thinks credentials are a guarantee of quality (people will always slip through the cracks no matter what), but I'm not sure they're even a strong indicator of quality either, certainly not for programming. This is hard to quantify of course, and just based on my own observations over my career. [1]: https://news.ycombinator.com/item?id=21585958 https://news.ycombinator.com/item?id=21585958 [2] Simple example: I'm firmly in team "short names", but I know some people really hate it, but I'm pragmatic if nothing else and don't mind doing whatever works well for the entire team. On one extreme end I've worked in a team that insisted on the longest possible name for everything: every variable had to spell out the full name of the class where it came from, often >20 chars, otherwise people considered it "unclear what anything refers to!" I once added a mkdir() function, this was considered "obscure" so it had to be "CreateDir()", then someone said it may not be clear what "dir" is so "CreateDirectory()", and then another remarked it actually makes all intermediate directories too, so we ended up with "CreateRecusiveDirectoryTree()" – I'm not saying this is "wrong", but it does not jibe well with my sensibilities. Several other things also didn't, so I didn't last too long).

xyzzy_plugh 3y ago

Indeed. I think tptacek's perspective on hiring is quite radical, certainly for established shops where doing things differently than the rest of the industry just adds friction elsewhere. My read on the approach is that it works very well for certain types of engagements, but other types, such as where you are trying to build a solid team of peers, requires something _closer_ to the traditional methods. That being said the Starfighter approach is probably right most of the time -- teams are almost certainly never stable, management usually treats ICs as fungible cogs and moves them around, in which case individual value should trump things like team fit or culture. But we're human. Appropaches like Google's work better (n.b. I am far from a proponent of what they do) because the interviewers are human and despite the whole blind-panel thing if the interviewer and interviewee can't get along the interview won't go well. So culture and team fit _is_ evaluated significantly more than something like Starfighter. People care a lot about that implicit bit, in my experience.

tptacek 3y ago
Fly.io is several solid teams of peers, built on resume-blind work-sample hiring.
- xyzzy_plugh 3y ago
  For sure! But the company was built that way. An existing 1000 person corp works differently. Everyone has to be on board for it to work, I'd wager.
wglb 3y ago
I used this approach at a company of about 750 folks, bending HR to bypass some of their stuff. E.g., they insisted on collecting resumes, but I generally discarded them. Instead, I did a one-on-one phone call for the purpose of them to ask me questions--not to grill them, and to ask them if they would like go to the next step and do a work sample test. I tailored the work sample test to the role. I did it for the compliance hires and for the project manager. The results were unexpectedly excellent. As a result of this process, and having a solid company culture, there are around ten or twelve folks that have a new career in the security field that would likely not have had another opportunity in the classical hiring scheme. I think the state of the industry with respect to hiring and evaluating folks is broken across the whole spectrum. It is like once someone becomes established in a field, they invent all manner of gatekeeping tests that they didn't pass to get where they are. The whole story of sabermetrics is another example.

dmoy 3y ago

Yea it's a problem. You're trying to hire from a pool that includes very competent people, and also people who legitimately can't write any code at all and are just trying to sneak in for 6mo to get $100k before they get fired, or think that "fake it til you make it" is a real thing. But at the same time, unlike other engineering disciples there's no PE licensing exams (or whatever the FAA does, I forget), no equivalent degree requirements (for good reason - one can learn to code without it, and a huge portion of a CS degree is very theoretical), but at the same time is qjite difficult, nothing like a plumber/electrician/etc certification level with accompanying work requirements and an apprenticeship program, and has very high pay and demand. For very small shops willing to incur legal risk or limit the application pool, you can do take home tests and/or lean on side projects, with an hour of explaining the nitty gritty details and choices. At scale it becomes untenable, because you open yourself up for discrimination lawsuits, and you can't convince a lot of people who already have jobs and families to do lengthier take home stuff. If I had to bet, I'd bet some day it ends up like plumbing. You work as an apprentice, learn real stuff on the job, and then there's some kind of state certification which itself is a joke, but requires you to have gone through an apprenticeship program already where all the real shit happens. Companies trying to hire "the best" or whatever are still going to have problems, so as long as there's not much regulation increasing cost of dev and uniformity by an alarming degree, we'll still have insane hiring (and insane pay too, regs will make it unprofitable eventually).

jvickers 3y ago
> because you open yourself up for discrimination lawsuits Is that because when looking to make a possibly subjective judgement on the performance in a test and especially what a side project shows, it then becomes more difficult to prove that the judgement was not instead made because of some protected characteristic of the candidate?
- dmoy 3y ago
  That, and a big chunk of it is kids. In the same way you can't decline to hire someone because they're pregnant or may soon become pregnant (or are the father side of that equation) - requiring side projects is a very thin/loose proxy for "doesn't have kids". It's not a big enough problem to really stop people, but a big company's lawyers will stop it internally.

tracerbulletx 3y ago

Software interviews feel more like an audition which isn't necessarily a bad approach, except we're using random people sometimes with two years of experience or who do not even want to be there as casting directors.