6 ms·

A major problem with data in the job market is that there are bad actors on both sides, employer and employee, with incentives to lie about themselves; ie compa

by vonnik 3y ago

A major problem with data in the job market is that there are bad actors on both sides, employer and employee, with incentives to lie about themselves; ie companies with toxic cultures won't reveal that, nor will malicious or incompetent job candidates. Many readers of HN know that companies can be bad, but don't recognize that, say, competent software engineers are not the median candidate applying for most roles. The other side has a filtering problem, too. A second major problem with data is that it's often hard to know why something didn't work out. Yeah, not everybody's a good fit for a role. But where would you get an objective source of truth about an individual's performance. Or ... transpose it to dating: would you trust a group of ex-girlfriends' or ex-boyfriends' to give an accurate assessment of someone as a partner? Might they have incentives to distort the record? Where does good data come from? So it's not just that AI lacks the data. It's that there are structural problems with ever gathering the data. It's not like the data's out there and we just don't have access. If it even exists, it's poisoned at the source.

ipaddr 3y ago
"competent software engineers are not the median candidate applying for most roles" Most developers with a few years of experience are pretty competent. Most employers are not toxic waste dumps. Give the worst interview candidates a try and they will do well. Give the best interview candidates a try and they will fail or leave early. Interviewing is a skill, anyone who is extremely good at it most likely practiced more often and that could mean they needed to. The more experienced and poor interview they give the more of an unearthed gem you've discovered
- zdragnar 3y ago
  I just interviewed someone who I am certain lied on their resume; they could barely write valid code during their interview, despite supposedly being a senior engineer with nearly 10 years of experience. Randomly hiring the first people to apply (or "giving them a shot") is a terrible strategy unless you have tons of money to burn and no deadlines to meet. We have internship positions for people willing to learn, but when we are trying to fill a senior position, we expect people to be at a certain level.
  - bodge5000 3y ago
    I'd like to think I'm a competent enough engineer, but I usually don't perform well in those live coding interviews, whereas I tend to do well enough in the "take home" tests. I wouldn't even necessarily say its a problem with live-coding, I've done things like pair programming before and its always gone fine, but an interview is an entirely different setting.
    - zdragnar 3y ago
      There's nothing I hate more than coding exercises in technical interviews, but the team wanted it, so there's a pretty simple one. I try to make as clear as possible in the preamble before we get to that part of the interview that I consider it to be a casual exercise, and have will offer to pair program with candidates who freeze up. I honestly don't care if they know the answer to some leetcode problem (also something I state in the preamble), I want to see how they break down a problem, and if they get stumped, how they work with other people on it. In the example I gave before, I had to explain to the candidate what the code they copy / pasted from the internet did. That might be fine for a junior candidate, but for someone with "10 years" of experience with the language, it's a non-starter for a senior programmer.
  - ipaddr 3y ago
    I would hire that person based on the reasons I gave before. Not being able to perform to your standards in the space you allowed doesn't mean that person is a fraud or that the person couldn't do the job. You were biased by your own test. You thought you were measuring how good a programmer someone was when in reality you were testing how good your test was.
    - wiseowise 3y ago
      That’s a lot of words to say nothing.
  - lisasays 3y ago
    It's amazing how fast coding skills can turn to shit (or in more polite terms, atrophy) if they haven't been accessed for, say, a year or so (and you were not super burned in with that language). Take that, add a bit of performance anxiety - that is to say: basic humanness - and there you go. Doesn't mean you have to hire them. Still, sometimes we just don't have enough data to tell whether someone is lying or not.
- wiseowise 3y ago
  > Most developers with a few years of experience are pretty competent. I’ve reviewed hundreds of assignments in last two years. No, that’s not true, chief.
SoftTalker 3y ago
The best hires I've seen in my life have come through word of mouth referrals. No data, no recruiters, no leetcode, no HR screenings. Good people know other good people. They trust one another, they are motivated to discuss opportunities of mutual benefit. Of course that doesn't scale when you're trying to fill in a large team quickly, but at that point you have to realize that "average" is about the best you can hope to get.
- artsytrashcan 3y ago
  Some of the worst hires come through word-of-mouth referrals, too. That's how you get unqualified nepotism hires. In fact, networking can be thought of as a form of nepotism or cronyism (depending on how touchy a given person is about "nepotism" being applied to the manner in which they got their position). I know I'm likely alone in viewing such hiring practices - feelings over process - as signal of an impending collapse. I think it's a sound heuristic, though, at least for previously-established companies. If you're not connected to eminently-qualified potential hires (and you're probably overestimating the quality of your network), "hire my friends" is a sure way to end up with an increasingly incompetent workforce.
  - alfalfasprout 3y ago
    I've gone back and forth on this and now am firmly in the feelings > process camp if you have competent folks making the hiring decisions and you train everyone well in avoiding unconscious bias. The reality is that strong soft-skills are crucial to hiring the best talent and those skills cannot be "objectively" tested. Even software engineering to a large degree requires a lot of design-related traits that require subjectivity in assessment.
    - mrkurt 3y ago
      You can test soft skills. Evaluations don't need to be objective, they just need to be consistent.
      - 3y ago
sublinear 3y ago
> Where does good data come from? Arbitrary opinions that cannot be decided by mere bias in data aggregated by an AI of any sample size or source. This is the fundamental flaw and why it's all horseshit. You know artists many years ago already covered all this. Impressionism: https://en.m.wikipedia.org/wiki/Impressionism https://en.m.wikipedia.org/wiki/Impressionism

by vonnik 3y ago

ipaddr 3y ago

"competent software engineers are not the median candidate applying for most roles" Most developers with a few years of experience are pretty competent. Most employers are not toxic waste dumps. Give the worst interview candidates a try and they will do well. Give the best interview candidates a try and they will fail or leave early. Interviewing is a skill, anyone who is extremely good at it most likely practiced more often and that could mean they needed to. The more experienced and poor interview they give the more of an unearthed gem you've discovered

zdragnar 3y ago
I just interviewed someone who I am certain lied on their resume; they could barely write valid code during their interview, despite supposedly being a senior engineer with nearly 10 years of experience. Randomly hiring the first people to apply (or "giving them a shot") is a terrible strategy unless you have tons of money to burn and no deadlines to meet. We have internship positions for people willing to learn, but when we are trying to fill a senior position, we expect people to be at a certain level.
- bodge5000 3y ago
  I'd like to think I'm a competent enough engineer, but I usually don't perform well in those live coding interviews, whereas I tend to do well enough in the "take home" tests. I wouldn't even necessarily say its a problem with live-coding, I've done things like pair programming before and its always gone fine, but an interview is an entirely different setting.
  - zdragnar 3y ago
    There's nothing I hate more than coding exercises in technical interviews, but the team wanted it, so there's a pretty simple one. I try to make as clear as possible in the preamble before we get to that part of the interview that I consider it to be a casual exercise, and have will offer to pair program with candidates who freeze up. I honestly don't care if they know the answer to some leetcode problem (also something I state in the preamble), I want to see how they break down a problem, and if they get stumped, how they work with other people on it. In the example I gave before, I had to explain to the candidate what the code they copy / pasted from the internet did. That might be fine for a junior candidate, but for someone with "10 years" of experience with the language, it's a non-starter for a senior programmer.
- ipaddr 3y ago
  I would hire that person based on the reasons I gave before. Not being able to perform to your standards in the space you allowed doesn't mean that person is a fraud or that the person couldn't do the job. You were biased by your own test. You thought you were measuring how good a programmer someone was when in reality you were testing how good your test was.
  - wiseowise 3y ago
    That’s a lot of words to say nothing.
- lisasays 3y ago
  It's amazing how fast coding skills can turn to shit (or in more polite terms, atrophy) if they haven't been accessed for, say, a year or so (and you were not super burned in with that language). Take that, add a bit of performance anxiety - that is to say: basic humanness - and there you go. Doesn't mean you have to hire them. Still, sometimes we just don't have enough data to tell whether someone is lying or not.
wiseowise 3y ago
> Most developers with a few years of experience are pretty competent. I’ve reviewed hundreds of assignments in last two years. No, that’s not true, chief.

SoftTalker 3y ago

The best hires I've seen in my life have come through word of mouth referrals. No data, no recruiters, no leetcode, no HR screenings. Good people know other good people. They trust one another, they are motivated to discuss opportunities of mutual benefit. Of course that doesn't scale when you're trying to fill in a large team quickly, but at that point you have to realize that "average" is about the best you can hope to get.

artsytrashcan 3y ago
Some of the worst hires come through word-of-mouth referrals, too. That's how you get unqualified nepotism hires. In fact, networking can be thought of as a form of nepotism or cronyism (depending on how touchy a given person is about "nepotism" being applied to the manner in which they got their position). I know I'm likely alone in viewing such hiring practices - feelings over process - as signal of an impending collapse. I think it's a sound heuristic, though, at least for previously-established companies. If you're not connected to eminently-qualified potential hires (and you're probably overestimating the quality of your network), "hire my friends" is a sure way to end up with an increasingly incompetent workforce.
- alfalfasprout 3y ago
  I've gone back and forth on this and now am firmly in the feelings > process camp if you have competent folks making the hiring decisions and you train everyone well in avoiding unconscious bias. The reality is that strong soft-skills are crucial to hiring the best talent and those skills cannot be "objectively" tested. Even software engineering to a large degree requires a lot of design-related traits that require subjectivity in assessment.
  - mrkurt 3y ago
    You can test soft skills. Evaluations don't need to be objective, they just need to be consistent.
    - 3y ago

sublinear 3y ago

> Where does good data come from? Arbitrary opinions that cannot be decided by mere bias in data aggregated by an AI of any sample size or source. This is the fundamental flaw and why it's all horseshit. You know artists many years ago already covered all this. Impressionism: https://en.m.wikipedia.org/wiki/Impressionism https://en.m.wikipedia.org/wiki/Impressionism