7 ms·

Auditing for discrimination in algorithms delivering job ads

33 points by yamafaktory 6y ago

jeliotj 6y ago
Advertising is, in a sense, lying. The question of advertisers has always been, how do I engage a person with my product in a way that seems natural to him/her, i.e. so that he/she doesn't see the lie. Necessities don't need advertising. Therefore, in a sense, everything advertised is based on desire rather than need and, as such, requires targeting what the consumer already desires in some way. In re this article: preferences based on gender are encouraged by a society of individuals who want to excuse their desire as worthy. AI only picks up on what already exists and therefore the root of the problem is much deeper than Facebook can remedy by a simple patch.
tareqak 6y ago
Same story from the Associated Press: https://apnews.com/article/discrimination-f62160cbbad4d72ce5250e6ef2222f5e https://apnews.com/article/discrimination-f62160cbbad4d72ce5...
Hermel 6y ago
What if the discrimination is based on interest? For example, the researcher found that a job ad for a car salesperson is more often shown to men and a job ad for a jewelry salesperson more often to women. But that could just be correlation, and not intentional causation. It makes perfect sense to show jewelry related ads to persons that are more interested in jewelry. In that case, the problem is not Facebook but the fact that women are more interested in jewelry than men and that men are more interested in cars than women.
- Ekaros 6y ago
  This data is probably based on what types of ads users click or what type of posts they like or interact with. Thus if more men interact with car related items and more women with jewellery related items. Isn't the targeting working correctly? As surely your average employer prefers to hire a person that is interested in field and on other hand employee would prefer job in field they are interested in. Maybe only solution is to fix the content consumption habits. Force everyone to only see and interact with perfectly mixed and equally distributed content that in no way take their own interest in mind.
bencollier49 6y ago
This is terrible and ought to be remedied for obvious reasons, but it does raise an interesting question. In the future, as AIs develop more complex mental models and are able to start forming nuanced opinions without explicit training, thoughtcrime in Artificial Intelligence is going to be a growing field. What happens when AIs universally develop opinions that we disagree with? What if they all inexorably come to the conclusion that the moral standards of, oh, say Ancient Sparta, would be most beneficial to humans, and relentlessly promote those values? Do we mindwipe them, or put them into correctional training facilities with appropriately painful backpropagation when they think the wrong thing? There's probably a business here for someone who can make software which detects when AIs develop politically dangerous opinions so that they can be shut down.
- CyberRabbi 6y ago
  AI thoughtcrime is a fascinating phenomenon that I had not modeled as such previously. This is exactly why I come to HN. Brilliant.
  - ryder9 6y ago
    only interesting parts about westworld season 3
- petercooper 6y ago
  We've experienced such things in the insurance industry with actuaries and their statistics/models playing the role of oracle. Pricing discrimination takes place based on the customer's job, demographics, income, or neighborhood – and often it's the poorer, younger, and less privileged people who pay more due to their 'risk' merely based on the identity groups they're in. Eventually, people get upset enough so that laws are passed enforcing whatever the society thinks is fair, and then it's the job of the industry (or whoever is controlling the 'AI' in your example) to comply, such as in the EU post-2012 with banning gender discrimination in insurance whether or not it has any statistical merit. In your case, the solution seems, to me, to be as simple as making the system ignore whatever variables you feel shouldn't be taken into account, whether that's gender or something else.
sokoloff 6y ago
Ad buyers used to choose these inadvertent biases before. Advertising in the Wall Street Journal is likely to bring you a different audience than advertising in Mother Earth News. I don't recall any previous claims that if you advertised in one that you were obligated to advertise in the other. Now, this Facebook case is clearly more than 0% different but I think is less than 100% different from that. In fact, I wouldn't be surprised if advertising in Technology Review itself (twice as many male readers as female readers) wouldn't have some of these same problems. * - https://mediakit.technologyreview.com/#banner1 https://mediakit.technologyreview.com/#banner1
- everythingswan 6y ago
  It's certainly worth talking about and fixing--the difference between segmentation and discrimination. Like in the HUD case mentioned in the article, discriminating based on age, gender, race, etc. are actually illegal. Not every industry is as regulated nor are they all at risk of it because of this. To me, there's not a question of "should we make this better?" because we're talking about basic rights: the right to not be discriminated in the pursuit of a roof over your head or a job to pay you a living wage (housing and employment). IMO, a very different question when you advertise anything outside of those basic rights. Plenty of grey area to talk about there.
  - sokoloff 6y ago
    If I’m advertising a $20M luxury penthouse overlooking Central Park, am I permitted to advertise only in the WSJ and Robb Report (or whatever rich people read)? Does that change if I learn those publications skew overwhelmingly male?
- lupire 6y ago
  Article: > The researchers weren’t able to discern why that is, because Facebook won’t say how its ad-delivery system works.
thitcanh 6y ago
In other news, 12% of nurses are men. Is it because the nursing industry isn’t welcoming to men or is it because men aren’t attracted to nursing jobs? You can see bias anywhere you look, but if you look well you’ll see that it’s just a natural consequence of the real world. If a user joined a car shop group, said user is likely to be interested in car-related jobs. It happens to be that most of these users are men. The algorithm is working as intended. In some fields this difference isn’t as clear or obvious, so we end up with these articles.
- cj 6y ago
  From the article: > This is considered sex-based discrimination under US equal employment opportunity law, which bans ad targeting based on protected characteristics. The algorithm is not "working as intended" if it's violating EEO laws.
  - shadowgovt 6y ago
    And, indeed, those laws are set up, in a sense, to cut against "natural" order (or, more specifically, to widen opportunities beyond the limits imposed by societal tradition).
  - KMnO4 6y ago
    I’m sure the algorithm isn’t checking “if (person.sex == male) { show_car_jobs(); }”, But rather “if (person.likes_cars) { show_car_jobs(); }”.
    - psychometry 6y ago
      Clearly, but you're entirely missing the point.
    - anoncake 6y ago
      The question is it it's checking "if(person.sex != male) { person.likes_cars = false }".
  - fabatka 6y ago
adsHijackU 6y ago
Facebook is profit driven more than anything. Does this mean men are more ambitious job seakers while women have different interests?(at large) My buddy gets ads for expensive highly marketed products as he buys Apple, Tesla, Peleton. His ads are drastically different than mine. I must be a low value user because mine are for news articles. His are for spending $$$ Facebook has no morals or objectives, they just want money.
_0ffh 6y ago
Such a horribly disingenuous click bait title, that doesn't even line up with what's actually in the article. >> They advertised for two delivery driver jobs, for example: one for Domino’s (pizza delivery) and one for Instacart (grocery delivery). There are currently more men than women who drive for Domino’s, and vice versa for Instacart. [...] The Domino’s ad was shown to more men than women, and the Instacart ad was shown to more women than men. The researchers found the same pattern with ads for two other pairs of jobs: software engineers for Nvidia (skewed male) and Netflix (skewed female), and sales associates for cars (skewed male) and jewelry (skewed female). << In short: >> The findings suggest that Facebook’s algorithms are somehow picking up on the current demographic distribution of these jobs, which often differ for historical reasons. << That's not at all what is explicitly, and falsely, claimed in the headline!
- thaumasiotes 6y ago
  > The findings suggest that Facebook’s algorithms are somehow picking up on the current demographic distribution of these jobs, which often differ for historical reasons. It seems like it would also be possible that the demographics of the jobs differ for non-historical reasons, and Facebook's algorithms are correctly picking up on the fact that women are more interested in working for Instacart than for Domino's...?
  - _0ffh 6y ago
    Yes it could of course be all kinds of things. Men could post more about pizza and women more about groceries, men could post more about Nvidia and women about Netflix. But that's not where my beef is. This is certainly not "excluding women" because it is just as much "excluding men" for some jobs, where "excluding" means "less likely to show the ad to" (we don't even get to know how much less likely, could be ppm for all we know) and "men" and "women" are interchangeable wherefore the "women" in the headline is irrelevant, distracting and (I must assume) willfully misleading.
- deleted 6y ago
  [deleted]
beaconstudios 6y ago
Demographic statistical reductions are the exact same thing as stereotypes, so of course they're going to be biased. As long as we keep using statistical reductions to predict individual behaviour they're going to continue to be wrong for the individual, sometimes in offensive ways. If you want predictive ads that work for the individual, if eg. women are 70% of teachers then 70% of teacher job ads will still go to women, but men who want to teach and women who don't want to will no longer receive those ads as they're not part of their individual profile.
bjeds 6y ago
Sensationalist. The key point of the article hinges on one particular statement: "These gender differences cannot be explained away by gender differences in qualifications or a lack of qualifications," How the heck is Facebook supposed to know about someones qualifications? Facebook _obviously_ have a set of standard data points they use for ad targeting, such as location, gender, age-span and so on together with dynamically updated data who have interacted with the ad. Just because the outcome is not what the journalist want doesn't necessarily mean it's wrong or discriminatory. Sure, it could of course be that Facebook algorithm is explicitly discriminatory, but it's more likely an algorithm such as this one is actually fairly neutral (compared with pre-trained data that can have built-in bias, for example photographs of people with mostly white skin - ad targeting is probably keyword based, and should be trained on actual data from what actual people click on). Is it discriminatory? I don't think so. Is it "filter-bubble-reinforcing"? Yes, that's more likely. As more men initially click an ad, it will be shown to more men. And vice versa.
- DanBC 6y ago
  > Facebook _obviously_ have a set of standard data points they use for ad targeting, such as [...] gender, How can anyone read this about job ads and not think maybe it's a problem?
  - bjeds 6y ago
    It doesn't actually matter, you can remove that data point and you are still at mercy at who actually interact with the ad once it's published.
- lupire 6y ago
  Domino's and DoorDash have the same qualifications. > set of standard data points for ad targeting, such as location, gender That would be illegal employment discrimination on the basis of gender.
  - AlexTWithBeard 6y ago
    Employment - yes. Advertising - no. Otherwise my big lawyered up corp wouldn't be having "girls in tech" recruitment events. Funny enough, these events are still attended predominately by males. Go figure.
mustafa_pasi 6y ago
Technically it is impossible to fix this issue. You can remove the explicit gender feature from the data, but I guarantee that the remaining features are correlated enough with gender that nothing would change. One could debate whether it should be the law that ML is not used in certain services so as to not project society's biases. However I doubt the claim that this is illegal. Is targeted advertisement illegal? I would love for it to be, but I doubt it is.
lupire 6y ago
I don't think job ads are important. What important are job *searches". Does FB even have a feature to search job ads? The ought to; they can charge employers and users will seek out ads to look at!
cmancini 6y ago
Unless I'm reading this incorrectly, these are just ads, not some kind of sexist job board where you put in your gender and out comes a list of eligible jobs. Just yesterday the internet was mad that people were seeing ads and it was bringing the downfall of democracy. Why are we mad they are now not seeing ads? Given the number of features in a model as sophisticated as Facebook's must be means these researchers are almost certainly oversimplifying, and certainly the way these articles are written as if to teach readers there's an evil software engineer writing biased code. From my experience in ML it's almost certainly the opposite--most of the data scientists I've worked with are highly aware of the issues of bias in AI and actively work against it to a sophistication level never understood by journalists.
- newacct583 6y ago
  The article doesn't discuss culpability or root cause. It simply reports a study that shows the bias still exists. Surely you'd agree that part of the solution to bias in AI is awareness of what bias exists?
seneca 6y ago
This article is extremely sensationalist. Facebook pretty blatantly advertises based on interest. If a job ad for nurses were placed in a nursing magazine, it would be seen largely by women. That's because nurses are largely women, not because magazines are excluding men. If you start from the fantasy position that women and men are the same, reality is going to seem extremely biased, I supposed.

33 points by yamafaktory 6y ago

Advertising is, in a sense, lying. The question of advertisers has always been, how do I engage a person with my product in a way that seems natural to him/her, i.e. so that he/she doesn't see the lie. Necessities don't need advertising. Therefore, in a sense, everything advertised is based on desire rather than need and, as such, requires targeting what the consumer already desires in some way. In re this article: preferences based on gender are encouraged by a society of individuals who want to excuse their desire as worthy. AI only picks up on what already exists and therefore the root of the problem is much deeper than Facebook can remedy by a simple patch.

tareqak 6y ago

Same story from the Associated Press: https://apnews.com/article/discrimination-f62160cbbad4d72ce5250e6ef2222f5e https://apnews.com/article/discrimination-f62160cbbad4d72ce5...

Hermel 6y ago

What if the discrimination is based on interest? For example, the researcher found that a job ad for a car salesperson is more often shown to men and a job ad for a jewelry salesperson more often to women. But that could just be correlation, and not intentional causation. It makes perfect sense to show jewelry related ads to persons that are more interested in jewelry. In that case, the problem is not Facebook but the fact that women are more interested in jewelry than men and that men are more interested in cars than women.

Ekaros 6y ago
This data is probably based on what types of ads users click or what type of posts they like or interact with. Thus if more men interact with car related items and more women with jewellery related items. Isn't the targeting working correctly? As surely your average employer prefers to hire a person that is interested in field and on other hand employee would prefer job in field they are interested in. Maybe only solution is to fix the content consumption habits. Force everyone to only see and interact with perfectly mixed and equally distributed content that in no way take their own interest in mind.

bencollier49 6y ago

This is terrible and ought to be remedied for obvious reasons, but it does raise an interesting question. In the future, as AIs develop more complex mental models and are able to start forming nuanced opinions without explicit training, thoughtcrime in Artificial Intelligence is going to be a growing field. What happens when AIs universally develop opinions that we disagree with? What if they all inexorably come to the conclusion that the moral standards of, oh, say Ancient Sparta, would be most beneficial to humans, and relentlessly promote those values? Do we mindwipe them, or put them into correctional training facilities with appropriately painful backpropagation when they think the wrong thing? There's probably a business here for someone who can make software which detects when AIs develop politically dangerous opinions so that they can be shut down.

CyberRabbi 6y ago
AI thoughtcrime is a fascinating phenomenon that I had not modeled as such previously. This is exactly why I come to HN. Brilliant.
- ryder9 6y ago
  only interesting parts about westworld season 3
petercooper 6y ago
We've experienced such things in the insurance industry with actuaries and their statistics/models playing the role of oracle. Pricing discrimination takes place based on the customer's job, demographics, income, or neighborhood – and often it's the poorer, younger, and less privileged people who pay more due to their 'risk' merely based on the identity groups they're in. Eventually, people get upset enough so that laws are passed enforcing whatever the society thinks is fair, and then it's the job of the industry (or whoever is controlling the 'AI' in your example) to comply, such as in the EU post-2012 with banning gender discrimination in insurance whether or not it has any statistical merit. In your case, the solution seems, to me, to be as simple as making the system ignore whatever variables you feel shouldn't be taken into account, whether that's gender or something else.

sokoloff 6y ago

Ad buyers used to choose these inadvertent biases before. Advertising in the Wall Street Journal is likely to bring you a different audience than advertising in Mother Earth News. I don't recall any previous claims that if you advertised in one that you were obligated to advertise in the other. Now, this Facebook case is clearly more than 0% different but I think is less than 100% different from that. In fact, I wouldn't be surprised if advertising in Technology Review itself (twice as many male readers as female readers) wouldn't have some of these same problems. * - https://mediakit.technologyreview.com/#banner1 https://mediakit.technologyreview.com/#banner1

everythingswan 6y ago
It's certainly worth talking about and fixing--the difference between segmentation and discrimination. Like in the HUD case mentioned in the article, discriminating based on age, gender, race, etc. are actually illegal. Not every industry is as regulated nor are they all at risk of it because of this. To me, there's not a question of "should we make this better?" because we're talking about basic rights: the right to not be discriminated in the pursuit of a roof over your head or a job to pay you a living wage (housing and employment). IMO, a very different question when you advertise anything outside of those basic rights. Plenty of grey area to talk about there.
- sokoloff 6y ago
  If I’m advertising a $20M luxury penthouse overlooking Central Park, am I permitted to advertise only in the WSJ and Robb Report (or whatever rich people read)? Does that change if I learn those publications skew overwhelmingly male?
lupire 6y ago
Article: > The researchers weren’t able to discern why that is, because Facebook won’t say how its ad-delivery system works.

thitcanh 6y ago

In other news, 12% of nurses are men. Is it because the nursing industry isn’t welcoming to men or is it because men aren’t attracted to nursing jobs? You can see bias anywhere you look, but if you look well you’ll see that it’s just a natural consequence of the real world. If a user joined a car shop group, said user is likely to be interested in car-related jobs. It happens to be that most of these users are men. The algorithm is working as intended. In some fields this difference isn’t as clear or obvious, so we end up with these articles.

cj 6y ago
From the article: > This is considered sex-based discrimination under US equal employment opportunity law, which bans ad targeting based on protected characteristics. The algorithm is not "working as intended" if it's violating EEO laws.
- shadowgovt 6y ago
  And, indeed, those laws are set up, in a sense, to cut against "natural" order (or, more specifically, to widen opportunities beyond the limits imposed by societal tradition).
- KMnO4 6y ago
  I’m sure the algorithm isn’t checking “if (person.sex == male) { show_car_jobs(); }”, But rather “if (person.likes_cars) { show_car_jobs(); }”.
  - psychometry 6y ago
    Clearly, but you're entirely missing the point.
  - anoncake 6y ago
    The question is it it's checking "if(person.sex != male) { person.likes_cars = false }".
- fabatka 6y ago

adsHijackU 6y ago

Facebook is profit driven more than anything. Does this mean men are more ambitious job seakers while women have different interests?(at large) My buddy gets ads for expensive highly marketed products as he buys Apple, Tesla, Peleton. His ads are drastically different than mine. I must be a low value user because mine are for news articles. His are for spending $$$ Facebook has no morals or objectives, they just want money.

_0ffh 6y ago

Such a horribly disingenuous click bait title, that doesn't even line up with what's actually in the article. >> They advertised for two delivery driver jobs, for example: one for Domino’s (pizza delivery) and one for Instacart (grocery delivery). There are currently more men than women who drive for Domino’s, and vice versa for Instacart. [...] The Domino’s ad was shown to more men than women, and the Instacart ad was shown to more women than men. The researchers found the same pattern with ads for two other pairs of jobs: software engineers for Nvidia (skewed male) and Netflix (skewed female), and sales associates for cars (skewed male) and jewelry (skewed female). << In short: >> The findings suggest that Facebook’s algorithms are somehow picking up on the current demographic distribution of these jobs, which often differ for historical reasons. << That's not at all what is explicitly, and falsely, claimed in the headline!

thaumasiotes 6y ago
> The findings suggest that Facebook’s algorithms are somehow picking up on the current demographic distribution of these jobs, which often differ for historical reasons. It seems like it would also be possible that the demographics of the jobs differ for non-historical reasons, and Facebook's algorithms are correctly picking up on the fact that women are more interested in working for Instacart than for Domino's...?
- _0ffh 6y ago
  Yes it could of course be all kinds of things. Men could post more about pizza and women more about groceries, men could post more about Nvidia and women about Netflix. But that's not where my beef is. This is certainly not "excluding women" because it is just as much "excluding men" for some jobs, where "excluding" means "less likely to show the ad to" (we don't even get to know how much less likely, could be ppm for all we know) and "men" and "women" are interchangeable wherefore the "women" in the headline is irrelevant, distracting and (I must assume) willfully misleading.
deleted 6y ago
[deleted]

beaconstudios 6y ago

Demographic statistical reductions are the exact same thing as stereotypes, so of course they're going to be biased. As long as we keep using statistical reductions to predict individual behaviour they're going to continue to be wrong for the individual, sometimes in offensive ways. If you want predictive ads that work for the individual, if eg. women are 70% of teachers then 70% of teacher job ads will still go to women, but men who want to teach and women who don't want to will no longer receive those ads as they're not part of their individual profile.

bjeds 6y ago

Sensationalist. The key point of the article hinges on one particular statement: "These gender differences cannot be explained away by gender differences in qualifications or a lack of qualifications," How the heck is Facebook supposed to know about someones qualifications? Facebook _obviously_ have a set of standard data points they use for ad targeting, such as location, gender, age-span and so on together with dynamically updated data who have interacted with the ad. Just because the outcome is not what the journalist want doesn't necessarily mean it's wrong or discriminatory. Sure, it could of course be that Facebook algorithm is explicitly discriminatory, but it's more likely an algorithm such as this one is actually fairly neutral (compared with pre-trained data that can have built-in bias, for example photographs of people with mostly white skin - ad targeting is probably keyword based, and should be trained on actual data from what actual people click on). Is it discriminatory? I don't think so. Is it "filter-bubble-reinforcing"? Yes, that's more likely. As more men initially click an ad, it will be shown to more men. And vice versa.

DanBC 6y ago
> Facebook _obviously_ have a set of standard data points they use for ad targeting, such as [...] gender, How can anyone read this about job ads and not think maybe it's a problem?
- bjeds 6y ago
  It doesn't actually matter, you can remove that data point and you are still at mercy at who actually interact with the ad once it's published.
lupire 6y ago
Domino's and DoorDash have the same qualifications. > set of standard data points for ad targeting, such as location, gender That would be illegal employment discrimination on the basis of gender.
- AlexTWithBeard 6y ago
  Employment - yes. Advertising - no. Otherwise my big lawyered up corp wouldn't be having "girls in tech" recruitment events. Funny enough, these events are still attended predominately by males. Go figure.

mustafa_pasi 6y ago

Technically it is impossible to fix this issue. You can remove the explicit gender feature from the data, but I guarantee that the remaining features are correlated enough with gender that nothing would change. One could debate whether it should be the law that ML is not used in certain services so as to not project society's biases. However I doubt the claim that this is illegal. Is targeted advertisement illegal? I would love for it to be, but I doubt it is.

lupire 6y ago

I don't think job ads are important. What important are job *searches". Does FB even have a feature to search job ads? The ought to; they can charge employers and users will seek out ads to look at!

cmancini 6y ago

Unless I'm reading this incorrectly, these are just ads, not some kind of sexist job board where you put in your gender and out comes a list of eligible jobs. Just yesterday the internet was mad that people were seeing ads and it was bringing the downfall of democracy. Why are we mad they are now not seeing ads? Given the number of features in a model as sophisticated as Facebook's must be means these researchers are almost certainly oversimplifying, and certainly the way these articles are written as if to teach readers there's an evil software engineer writing biased code. From my experience in ML it's almost certainly the opposite--most of the data scientists I've worked with are highly aware of the issues of bias in AI and actively work against it to a sophistication level never understood by journalists.

newacct583 6y ago
The article doesn't discuss culpability or root cause. It simply reports a study that shows the bias still exists. Surely you'd agree that part of the solution to bias in AI is awareness of what bias exists?

seneca 6y ago

This article is extremely sensationalist. Facebook pretty blatantly advertises based on interest. If a job ad for nurses were placed in a nursing magazine, it would be seen largely by women. That's because nurses are largely women, not because magazines are excluding men. If you start from the fantasy position that women and men are the same, reality is going to seem extremely biased, I supposed.