6 ms·

I will never forget. Some startup claimed they could find the best candidates for the job using "AI." They had a whole bunch of BS selling points. I think th

by robswc 4y ago

I will never forget. Some startup claimed they could find the best candidates for the job using "AI." They had a whole bunch of BS selling points. I think they even convinced a fair amount of companies to get on board too. I was thinking "how can people be so stupid!?" I then found one of the "features" was that they have the managers/bosses take the exam. They then offer a package that can weigh candidates based on correlation to their boss. Well, that's what it did. The marketing pitch was something something, view if they're compatible with your culture, something something. I then realized its genius. Charge a ton of money to do basic linear regression and spend 90% of your money on giving your clients an excuse to hire whoever they want without appearing bias. They would even appear "virtuous" because they let the "AI decide." Just awful, lol. Of course, I never got to see the internals and I don't want to slander but the "test the boss, correlate with new hire" was 100% real.

duxup 4y ago
At least as far as technical knowledge goes... that maybe works pretty well.
- robswc 4y ago
  I mean, it works incredibly well if you're a boss and want to hire a bunch of clones... but if you're a bad boss... lol
  - duxup 4y ago
    Is suspect that is often the case. Is it optimal? Nope. But if you're offering candidates / filtering them for someone... you give them what they want / like right?
  - gojomo 4y ago
    Can anything save a bad boss? Maybe every boss deserves to be matched with the team that makes all involved most-likely to mutually-succeed, even if it's a peculiar & unrepresentative bunch of people who can only work with each other. And further, if that best-effort assemblage still consistently malfunctions, the boss, & the whole team, should be encouraged to find other roles and teams that have a better chance of succeeding.
    - robswc 4y ago
      I totally agree. I just feel like a bad boss hiring a bunch of clones would only be worse for the company.
mistrial9 4y ago
a PhD scientist in California was awarded a multi-year $5m USD grant based on similar principles, around 2015, through the University of California. I sat through her lunch talk to a group of researchers. The merit of the idea was that she could find under-represented demographics of young potential coders by matching various online activity with their resume, and assuming a lot of low-income education credits are not going to look very good on a typical boilerplate resume. I was mildly annoyed at the time.
- robswc 4y ago
  Wow. Yea. I think the hiring processes is broken in a lot of ways. I also think trying to get AI to solve it or adding even more layers into the process is only going to make things worse.
- tikhonj 4y ago
  There's a qualitative difference between using AI for sourcing—that is, finding candidates you would not find normally—and using it for assessment. The impact of a biased source is attenuated by having other sources and by the rest of your assessment process. Perhaps you end up with a marginally "worse" pool of candidates at the top of your hiring funnel... but, frankly, it's already hard to do much worse than the mix of LinkedIn keyword searches, resume screens and ad hoc channels teams use today. Crucially, if this happens, the impact on any given candidate is minimal—a bit more competition that, if the rest of the process is any good, should be negligible. The impact of bias in how you reject candidates, on the other hand, fundamentally cannot be compensated for in the remaining process. A rejected candidate is already out! Any bias in rejecting steps is going to be directly reflected in the sort of teams you can recruit. Mixing together sourcing methods that are biased in different ways can cancel out the bias. Mixing together assessment steps that are biased in different ways compounds the bias. Traditional resumes are pretty awful any way you look at it[1] and developing new ways of sourcing qualified candidates that get overlooked by the traditional approach doesn't create the sorts of problems that new assessment methods do, so the research you're describing seems unambiguously useful. [1]: interviewing.io did a cool study looking at how resumes compare against the mock interview performance of users on their site. They just don't work well. This isn't a peer-reviewed study, of course, but, frankly, the data and approach are more compelling than some of the peer-reviewed research I've seen on the subject. https://blog.interviewing.io/resumes-suck-heres-the-data/ https://blog.interviewing.io/resumes-suck-heres-the-data/
fallingknife 4y ago
> I don't want to slander It is never slander to speak the truth
- fsckboy 4y ago
  he didn't mean "I don't want to technical-meaning-of-slander", he meant "I don't want to badmouth perhaps unfairly since I probably don't have all the facts"
- robswc 4y ago
  True. I guess I'd rather not create drama for drawing implications from marketing material, haha.
  - fallingknife 4y ago
    Fair enough, though I think it's pretty fair to assume that the truth is either the same or worse than the marketing material!
- cercatrova 4y ago
  Except in Japan where you can be prosecuted even if what you spoke about someone else was true. They see it as destroying the their person's honor.
  - aliqot 4y ago
    For the other 99.87% of us on this board, this is good info.
    - cercatrova 4y ago
      Depends on your jurisdiction, really. Even in Western countries the laws differ on what is slander and what isn't.
      - hutzlibu 4y ago
z_zetetic_z 4y ago
How problematic. Not least because hiring for 'culture fit' is an anti-pattern, and orgs should be hiring for 'culture add'.
humanistbot 4y ago
Sounds like https://pymetrics.ai https://pymetrics.ai ?
- robswc 4y ago
  Looks _very_ close, if not it. It has honestly been so long I can't say for sure. It was early 2010s. Seems a lot of the marketing material is vague and you need to request a demo. Looks pretty silly, regardless. Play a bunch of "games" as part of the interview? I bet HR loves it tho, lol.
salawat 4y ago
The term is "math-washing". It's use as a means to facilitate accountability laundering has been a sticking point for a while.
- bheadmaster 4y ago
  The excuse used to be "it's God's will". Now it's "it's AI's will".
  - deleted 4y ago
    [deleted]
- deleted 4y ago
  [deleted]
thedracle 4y ago
HireVue AI Assessment.
- cuteboy19 4y ago
  Pymetric Assessment™
tomp 4y ago
Well, that's literally what US colleges have been doing for decades. Basically outsourced IQ tests (since IQ tests during the hiring process are illegal). High time someone else got in on the action.
- eru 4y ago
  IQ tests aren't all that illegal. (And you can approximate them pretty well with other tests, like coding tests etc.) What you can't check for in a short interview is conformity and tenacity. You can however check for that via college.
- MichaelCollins 4y ago
  IQ tests during hiring are illegal if that IQ test has a disparate impact against a protected class. (It turns out they pretty much all do, even those specifically designed not to, like Raven's Progressive Matrices: https://en.wikipedia.org/wiki/Raven%27s_Progressive_Matrices https://en.wikipedia.org/wiki/Raven%27s_Progressive_Matrices) A "totally not an IQ test" IQ test is safer, particularly if it is bespoke, because new/obscure tests are less likely to have been studied, and consequently, any forbidden trends in test outcomes are less likely to be discovered.
  - sokoloff 4y ago
    > IQ tests during hiring are illegal if that IQ test has a disparate impact against a protected class. Are you sure about that? That seems unreasonable and everything that I can find in 5 minutes of Googling suggests that they are legal-but-restricted, to cases where the skill/aptitude being tested is relevant to the job being hired for. (You couldn't use an IQ test to hire a fry cook, as an example.)
    - MichaelCollins 4y ago
      Hiring practices which discriminate based on attributes that would otherwise be illegal to consider may be permitted if those attributes are Bona fide occupational qualifications (BFOQ.) For instance, blind people can't be bus drivers. IQ tests are verboten after Griggs v. Duke Power Co. As far as I'm aware, claiming a BFOQ exemption to this will fail, or at best, is unproven. IQ tests discriminate by race (even if unintentionally) and BFOQ exemptions don't apply to racial discrimination. Please correct me if I'm wrong.
hinkley 4y ago
Perfect echo chamber greedy algorithm. I love it. I also hate it. Also to quote J Jonah Jameson: it can’t be slander, because it’s in print. So it’s libel.

by robswc 4y ago

duxup 4y ago

At least as far as technical knowledge goes... that maybe works pretty well.

robswc 4y ago
I mean, it works incredibly well if you're a boss and want to hire a bunch of clones... but if you're a bad boss... lol
- duxup 4y ago
  Is suspect that is often the case. Is it optimal? Nope. But if you're offering candidates / filtering them for someone... you give them what they want / like right?
- gojomo 4y ago
  Can anything save a bad boss? Maybe every boss deserves to be matched with the team that makes all involved most-likely to mutually-succeed, even if it's a peculiar & unrepresentative bunch of people who can only work with each other. And further, if that best-effort assemblage still consistently malfunctions, the boss, & the whole team, should be encouraged to find other roles and teams that have a better chance of succeeding.
  - robswc 4y ago
    I totally agree. I just feel like a bad boss hiring a bunch of clones would only be worse for the company.

mistrial9 4y ago

a PhD scientist in California was awarded a multi-year $5m USD grant based on similar principles, around 2015, through the University of California. I sat through her lunch talk to a group of researchers. The merit of the idea was that she could find under-represented demographics of young potential coders by matching various online activity with their resume, and assuming a lot of low-income education credits are not going to look very good on a typical boilerplate resume. I was mildly annoyed at the time.

robswc 4y ago
Wow. Yea. I think the hiring processes is broken in a lot of ways. I also think trying to get AI to solve it or adding even more layers into the process is only going to make things worse.
tikhonj 4y ago
There's a qualitative difference between using AI for sourcing—that is, finding candidates you would not find normally—and using it for assessment. The impact of a biased source is attenuated by having other sources and by the rest of your assessment process. Perhaps you end up with a marginally "worse" pool of candidates at the top of your hiring funnel... but, frankly, it's already hard to do much worse than the mix of LinkedIn keyword searches, resume screens and ad hoc channels teams use today. Crucially, if this happens, the impact on any given candidate is minimal—a bit more competition that, if the rest of the process is any good, should be negligible. The impact of bias in how you reject candidates, on the other hand, fundamentally cannot be compensated for in the remaining process. A rejected candidate is already out! Any bias in rejecting steps is going to be directly reflected in the sort of teams you can recruit. Mixing together sourcing methods that are biased in different ways can cancel out the bias. Mixing together assessment steps that are biased in different ways compounds the bias. Traditional resumes are pretty awful any way you look at it[1] and developing new ways of sourcing qualified candidates that get overlooked by the traditional approach doesn't create the sorts of problems that new assessment methods do, so the research you're describing seems unambiguously useful. [1]: interviewing.io did a cool study looking at how resumes compare against the mock interview performance of users on their site. They just don't work well. This isn't a peer-reviewed study, of course, but, frankly, the data and approach are more compelling than some of the peer-reviewed research I've seen on the subject. https://blog.interviewing.io/resumes-suck-heres-the-data/ https://blog.interviewing.io/resumes-suck-heres-the-data/

fallingknife 4y ago

> I don't want to slander It is never slander to speak the truth

fsckboy 4y ago
he didn't mean "I don't want to technical-meaning-of-slander", he meant "I don't want to badmouth perhaps unfairly since I probably don't have all the facts"
robswc 4y ago
True. I guess I'd rather not create drama for drawing implications from marketing material, haha.
- fallingknife 4y ago
  Fair enough, though I think it's pretty fair to assume that the truth is either the same or worse than the marketing material!
cercatrova 4y ago
Except in Japan where you can be prosecuted even if what you spoke about someone else was true. They see it as destroying the their person's honor.
- aliqot 4y ago
  For the other 99.87% of us on this board, this is good info.
  - cercatrova 4y ago
    Depends on your jurisdiction, really. Even in Western countries the laws differ on what is slander and what isn't.
    - hutzlibu 4y ago

z_zetetic_z 4y ago

How problematic. Not least because hiring for 'culture fit' is an anti-pattern, and orgs should be hiring for 'culture add'.

humanistbot 4y ago

Sounds like https://pymetrics.ai https://pymetrics.ai ?

robswc 4y ago
Looks _very_ close, if not it. It has honestly been so long I can't say for sure. It was early 2010s. Seems a lot of the marketing material is vague and you need to request a demo. Looks pretty silly, regardless. Play a bunch of "games" as part of the interview? I bet HR loves it tho, lol.

salawat 4y ago

The term is "math-washing". It's use as a means to facilitate accountability laundering has been a sticking point for a while.

bheadmaster 4y ago
The excuse used to be "it's God's will". Now it's "it's AI's will".
- deleted 4y ago
  [deleted]
deleted 4y ago
[deleted]

thedracle 4y ago

HireVue AI Assessment.

cuteboy19 4y ago
Pymetric Assessment™

tomp 4y ago

Well, that's literally what US colleges have been doing for decades. Basically outsourced IQ tests (since IQ tests during the hiring process are illegal). High time someone else got in on the action.

eru 4y ago
IQ tests aren't all that illegal. (And you can approximate them pretty well with other tests, like coding tests etc.) What you can't check for in a short interview is conformity and tenacity. You can however check for that via college.
MichaelCollins 4y ago
IQ tests during hiring are illegal if that IQ test has a disparate impact against a protected class. (It turns out they pretty much all do, even those specifically designed not to, like Raven's Progressive Matrices: https://en.wikipedia.org/wiki/Raven%27s_Progressive_Matrices https://en.wikipedia.org/wiki/Raven%27s_Progressive_Matrices) A "totally not an IQ test" IQ test is safer, particularly if it is bespoke, because new/obscure tests are less likely to have been studied, and consequently, any forbidden trends in test outcomes are less likely to be discovered.
- sokoloff 4y ago
  > IQ tests during hiring are illegal if that IQ test has a disparate impact against a protected class. Are you sure about that? That seems unreasonable and everything that I can find in 5 minutes of Googling suggests that they are legal-but-restricted, to cases where the skill/aptitude being tested is relevant to the job being hired for. (You couldn't use an IQ test to hire a fry cook, as an example.)
  - MichaelCollins 4y ago
    Hiring practices which discriminate based on attributes that would otherwise be illegal to consider may be permitted if those attributes are Bona fide occupational qualifications (BFOQ.) For instance, blind people can't be bus drivers. IQ tests are verboten after Griggs v. Duke Power Co. As far as I'm aware, claiming a BFOQ exemption to this will fail, or at best, is unproven. IQ tests discriminate by race (even if unintentionally) and BFOQ exemptions don't apply to racial discrimination. Please correct me if I'm wrong.

hinkley 4y ago

Perfect echo chamber greedy algorithm. I love it. I also hate it. Also to quote J Jonah Jameson: it can’t be slander, because it’s in print. So it’s libel.