So it depends how you set up the experiment right? I would assume the question you are posing to AI is not how much the resume in question resembles the set of

So it depends how you set up the experiment right? I would assume the question you are posing to AI is not how much the resume in question resembles the set of resumes of hired engineers, but rather: given a resume, what is the probability that candidate will eventually be hired? So the classification function should take into account the resumes of rejected engineers, rather than the pool of resumes of hired employees at Amazon. If someone is seeking a position as an engineer, it is not relevant how much their resume resembles that of HR people, but it is very relevant how much it resembles that of rejected engineering candidates. If that's the case, then something like having the phrase "women's chess club" in one's resume should not be a meaningful factor for the classifier unless it disproportionately leads to rejection in the current process.