4 ms·

Judge your Resume with AI: What's the worst that can happen?

17 points by dmezzetti 3y ago

gmerc 3y ago
The worst that can happen is that you will get sued about using unsuitable technology with built in biases to assess or grade candidates, creating systemic discrimination in your hiring pipeline. The FTC just tore a retailer a 50mil new hole along with a 5 year order to ban the use of facial recognition software because they failed to ensure a non discriminatory environment by failing to validate AI software supplier claims about its performance. This article does not even cover the most basic mitigation to take (which very well may be unsuitable) like removing PII (name, address). It’s trivial to prove that leaving these elements in can bias the output in various ways. Can’t wait for every company to adopt the same AI snakeoil supplier for filtering applicants and then getting dragged into the inevitable class action with hot ugly discovery.
- postalrat 3y ago
  Nah, thats not so bad.
- dmezzetti 3y ago
  Hiring has been beyond broken well before LLMs came around. Most applicants don't even get a response when they apply to a job, especially with large/popular companies. I'm more optimistic on this you and think AI can help with the hiring process. There's just too many applicants and good applicants fall through the cracks. This is also not all that new. HR has long had systems to try to bubble up good candidates. This will just make those better. Currently, the best way to get in with a big tech company is knowing someone and working around the standard hiring process. How is that fair? Or how do we accept that most candidates never get a response? I'd argue there is a lot of room for improvement.
  - gmerc 3y ago
    I think it’s wishful thinking. How do you exactly envision current AI could make the process better? Technology rarely solves problems, at best it replaces them with different problems. You also assume that the ultimate goal of hiring is supposed to be fair - but fair is just one parameter, a ballot would be fair, but it would fail to meet the companies only goal.
    - dmezzetti 3y ago
      LLMs can help with data extraction. Resumes are notoriously tough to parse given the randomness of the formats. It's an unsolved problem. LLMs can also be used to run natural language queries against your candidate database (i.e. RAG). Those are just two, plenty of other things. No one is going to read every incoming resume, ATS systems have been around a long time. I'm sure existing ATS systems have already added the term "AI-powered" to their marketing. While AI is an overused term, there are certainly plenty of use cases to improve existing processes that are already happening.
fxtentacle 3y ago
The title seems to imply problems, but this is actually just a txtai tutorial.
- dmezzetti 3y ago
  That is part of it but this article is also covering how different prompts can lead to different results.

17 points by dmezzetti 3y ago

The worst that can happen is that you will get sued about using unsuitable technology with built in biases to assess or grade candidates, creating systemic discrimination in your hiring pipeline. The FTC just tore a retailer a 50mil new hole along with a 5 year order to ban the use of facial recognition software because they failed to ensure a non discriminatory environment by failing to validate AI software supplier claims about its performance. This article does not even cover the most basic mitigation to take (which very well may be unsuitable) like removing PII (name, address). It’s trivial to prove that leaving these elements in can bias the output in various ways. Can’t wait for every company to adopt the same AI snakeoil supplier for filtering applicants and then getting dragged into the inevitable class action with hot ugly discovery.

postalrat 3y ago
Nah, thats not so bad.
dmezzetti 3y ago
Hiring has been beyond broken well before LLMs came around. Most applicants don't even get a response when they apply to a job, especially with large/popular companies. I'm more optimistic on this you and think AI can help with the hiring process. There's just too many applicants and good applicants fall through the cracks. This is also not all that new. HR has long had systems to try to bubble up good candidates. This will just make those better. Currently, the best way to get in with a big tech company is knowing someone and working around the standard hiring process. How is that fair? Or how do we accept that most candidates never get a response? I'd argue there is a lot of room for improvement.
- gmerc 3y ago
  I think it’s wishful thinking. How do you exactly envision current AI could make the process better? Technology rarely solves problems, at best it replaces them with different problems. You also assume that the ultimate goal of hiring is supposed to be fair - but fair is just one parameter, a ballot would be fair, but it would fail to meet the companies only goal.
  - dmezzetti 3y ago
    LLMs can help with data extraction. Resumes are notoriously tough to parse given the randomness of the formats. It's an unsolved problem. LLMs can also be used to run natural language queries against your candidate database (i.e. RAG). Those are just two, plenty of other things. No one is going to read every incoming resume, ATS systems have been around a long time. I'm sure existing ATS systems have already added the term "AI-powered" to their marketing. While AI is an overused term, there are certainly plenty of use cases to improve existing processes that are already happening.

fxtentacle 3y ago

The title seems to imply problems, but this is actually just a txtai tutorial.

dmezzetti 3y ago
That is part of it but this article is also covering how different prompts can lead to different results.