9 ms·

Nobody is actually smarter or better than anyone at anything. It's a result of biases and privilege. And if all biases were taken into consideration, everyone w

by daenz 5y ago

Nobody is actually smarter or better than anyone at anything. It's a result of biases and privilege. And if all biases were taken into consideration, everyone would (and should) score exactly the same on everything, regardless of age, sex, race, gender, ethnicity, orientation, etc etc. That's the underlying message about ML initiatives that touch sensitive topics. Yes, we should be very careful not to encode biases into our models, and these biases are definitely real. Absolutely 100%! But what happens if the models still show disparity? Are we prepared for that possibility? Does it mean there are more biases yet to uncover and we must root them out? Or should we hide anything that reveals disparity in the absence of biases? Or are all biases impossible to remove, so they must forever remain the explanation for all disparity? Is disparity a proof of bias? Because if that is true, then the only conclusion is that every person is exactly functionally equal and the only differences are human biases. I've seen smart tech people unable to say that men are on average taller than women because (I believe) they were afraid to say that some "favorable" outcome (tall) was a result of some component that could be genetic. Their argument was anecdotes of tall women, and saying there is no such thing as an "average" man or "average" woman.

j7ake 5y ago
It would be nice to do a well-designed experiment to ask what are the biological differences in different categories of humans (eg old vs young). Then use that ground truth to assess whether an ML model is generating the biases that deviate from the ground truth. That way there is no a priori assumption of what is technical bias and what is real signal. With the level of investment people are putting into ML technologies, these types of ground truth calibrations would cost only a fraction of the total budget.
joe_the_user 5y ago
Nobody is actually smarter or better than anyone at anything. It's a result of biases and privilege. Where is the linked article claiming this? Your post seems like complete derail of the article discussion for the purposes of making broader political points. [insert hn caveat about being more nuanced in potentially more flamebait topic]
spoonjim 5y ago
Yes, it’s a common “argumentation“ strategy on Twitter. “Do you know that someone was once born with a horse’s head? Clearly the concept of ‘human’ does not exist!”
NicoJuicy 5y ago
I usually mention that nature experiments more with men than with woman. That's an explanation for why top talent are usually men, but the lesser ones are usually men too. ( + I was it was an explanation of a study)
3grdlurker 5y ago
> But what happens if the models still show disparity? You have yet to qualify exactly what you mean by disparity here—it doesn’t help that the example you used (i.e gender-height correlation) isn’t an ML model.
- tomrod 5y ago
  I agree with your main point, but note that a gender-height correlation is effectively the basis of a linear regression, the most basic of all ML models.
mustafa_pasi 5y ago
That is not my understanding of the ethical dilemma. People are different, and belonging to groups is highly predictive, and that is what ML algorithms depend upon to classify you (every feature is a group). The ethical dilemma is, should you be doomed to be assessed based on your belonging to an arbitrary set of groups? It is highly predictive, but also by definition, discriminatory. Business entities benefit, because on average the predictions are right, but individuals are being discriminated against, and btw, besides the protected categories, other types of discrimination are also unfair.
- biasedbrain 5y ago
  You should be allowed to think and perceive and notice things, and use tools to help you do so. It is ridiculous to mandate algorithms should be modified to hide statistical facts about the world. by all means, educate people that just because some population is perhaps on average more criminal, it doesn't imply any individual belonging to that population is also more criminal. To hide the statistics would also hurt efforts to find solutions. That can not be a good thing. Edit: if you don't like the criminal example, what about noticing some populations are on average poorer than others? Would that be a good thing, because government money could be diverted their way, or a bad thing, because banks would be less inclined to lend them money, and landlords would be less likely to rent them a home?
  - mustafa_pasi 5y ago
    The point of contention is not whether ugly statistical data should be hidden away. That is a stupid conspiracy theory. Sociologists have no problem publishing all kinds of ugly statistical data. The problems start when for profit entities or governments, start deciding to discriminate against individuals because they belong to certain groups. Statistical data applied to individuals is always going to result in injustice. The thought of the economy at large, or the government reducing an individual to a few datapoints and deciding their fate based on group statistics is just about the scariest communist dystopian nightmare, as far as I am concerned. It robs people of their own humanity.
nxpnsv 5y ago
The political debate is endless and not very enlightening. However, this stuff max constitute useful tools to make better models in a non political way. A model that is biased because of too narrow test data will perform worse than a model trained on a suitable dataset. It can be hard to spot these issues, and having some checks and bounds to avoid common sources of bias is a good thing. Furthermore, an analysis might expose more info about the users than intended. This can have pretty bad consequences. For this purpose I thought the one about "collecting sensitive information" was a pretty good start.
phreeza 5y ago
> And if all biases were taken into consideration, everyone would (and should) score exactly the same on everything, regardless of age, sex, race, gender, ethnicity, orientation, etc etc. "Should" is the operational word here. It is a normative decision that these (protected in many jurisdictions) classes should not receive any different treatment due to membership in the class. I agree with this goal, it seems you do not? Or are you objecting to a normative decision being disguised as a descriptive one? In that case I think it is largely a straw man, it's not how most people actually think.
- bordercases 5y ago
  > In that case I think it is largely a straw man, it's not how most people actually think. Why do you believe this? Even the notion of bias, which you must conceive as real for your claim to hold any weight, is a version of the descriptive being confused for the normative. And for biases to be as problematic as they are considered, they must be common in key places. Why couldn't researchers have an unreasonable bias against reasonable biases?
Jack000 5y ago
It shouldn't be controversial to say that on average, men are taller than women. However it is a problem to say that "since you are a woman, I infer that you must be short". The difference is (not really) subtle but the first statement can be true while the latter statement is obviously fallacious. With an objective quantity like height it's easy to see the error, but harder with less well-defined quantities like intelligence, character, etc.. It's frustrating to see people talk past each other with ill-defined terms.
- daenz 5y ago
  >It shouldn't be controversial to say that on average, men are taller than women. However it is a problem to say that "since you are a woman, I infer that you must be short". The difference is (not really) subtle but the first statement can be true while the latter statement is obviously fallacious. I agree, but the person I was speaking about specifically was the former, not the latter. They were extremely resistant to the idea of measurable biological differences. This is an intelligent, science-believing person. I think there are a few main groups that people fall into around these kinds of subjects: 1) People who suspect that there may be differences in groups of people, but know that acknowledging them will probably lead to arguments that support sexism, racism, homophobia, transphobia, etc (because they have historically), and so they avoid acknowledging any differences that can't be explained by bias. 2) People who want there to be differences in people so that they can justify their biases against groups of people. 3) People who don't want there to be differences in people, because it doesn't align with a fair worldview that they want to be real. 4) Everyone else who just want people to be honest about potentially uncomfortable truths.
  - jimmygrapes 5y ago
    Where are all the #4s?
    - deleted 5y ago
      [deleted]
    - mhuffman 5y ago
rocknor 5y ago
No need to spread FUD here. What you're saying isn't really happening and is not a problem. Mountains are being made out of molehills to spread FUD and derail actual useful discussion for the purposes of making a political point. Show us some proof and then we can talk.
nl 5y ago
> Nobody is actually smarter or better than anyone at anything.. <snip> That's the underlying message about ML initiatives that touch sensitive topics. No it's not, and if you think that you are misreading it. The general point of ML ethics is that a machine learning system should not reinforce bias caused by historically biased training data. This is good statistical practice, but unfortunately often gets lost in the "magic AI" bubble. Avoiding this is like saying "a salary of $110K/year makes you a rich person". It's sort of based on something true (it does put you in the the top quartile of US earners), but is completely useless without correcting at least for location and number of dependants. That's what ML Ethics is about. > I've seen smart tech people unable to say that men are on average taller than women because (I believe) they were afraid to say that some "favorable" outcome (tall) was a result of some component that could be genetic. So you've taken that dumb argument and said all ML ethics arguments are the same?
- tbrownaw 5y ago
  > The general point of ML ethics is that a machine learning system should not reinforce bias caused by historically biased training data. When I think of "biased training data", I think of data that systematically misrepresents reality. Like say taking the prevalence of news reports as a proxy for the prevalence of what's being reported on. In contrast, the AI ethics people seem to use that term to mean data that doesn't reflect just (as opposed to unjust) causality. I think there's a deeper issue, which is summed up in the saying "when a measure becomes a target, it ceases to be a good measure": simple correlation-finders - which is what current ML/AI models are - are not fit for the purpose of building control loops. And the "ethics" people are trying to paper this over by manually fixing up all the correlations that people notice aren't (or shouldn't be) casual.
  - nl 5y ago
    The post linked to for this story gives the example of searching for "CEO" only showing white men. The reason for that is almost entirely the same as your example of biased training data ("taking the prevalence of news reports as a proxy for the prevalence of what's being reported on"). CEOs from large US companies get reported on the most in English language press. Therefore searching in English for "CEO" finds those people.
DoreenMichele 5y ago
I've seen smart tech people unable to say that men are on average taller than women because (I believe) they were afraid to say that some "favorable" outcome (tall) was a result of some component that could be genetic. I've taken some pertinent college classes, like Intro to Psychology and Social Psychology. These are really hard things to debate in good faith because people look around them at the world, see the reality that men are, on average, taller than women on average and assume genetics when we don't actually know the reason. So if you have good faith reasons to wonder put loud or speculate about such things, you are likely to get treated dismissively and not taken seriously. There is a huge component of "them that has, gets" and it's really hard to sort out exactly why x person got y outcome. If you are interested in social justice -- or even just making your own life work when you are the "wrong" demographic -- then these are really frustrating discussions to try to have. It may be some hypothetical argument for someone whose life works well, but it's often a desperate attempt to get a fair shake of some sort by marginalized people. So there's a lot to unpack there and it's hard to unpack it and there are legitimate reasons why some people feel strongly compelled to not go along with "the obvious answer" that "everyone" knows to be true, even though they may not know how to effectively argue against it.
- lambdaphagy 5y ago
  In surveys of hundreds of human cultures, I don't believe anthropologists have found a single one in which women are, on average, even as tall as men. Though average height differs considerably between groups, there appears to be a consistent bias favoring men (see Fig 1. https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.1558-5646.1992.tb01997.x https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.1558-5646....). We also know that although secular changes in diet can shift population averages over time, the heritability of height is pretty high: most estimates end up at about 0.8. If your parents were short, getting adopted into a tall family is not going to help you become taller. Height is a highly polygenic trait and no single gene controls more than a few percent of the variance. When people say that we don't know the "genes for height", that's what they mean, not that the jury is still out on whether height has any genetic basis at all.
  - 5y ago
trhway 5y ago
>Nobody is actually smarter or better than anyone at anything. It's a result of biases and privilege. that statement is just incompatible with the basics of natural selection.
thefz 5y ago
> Nobody is actually smarter or better than anyone at anything. It's a result of biases and privilege. And if all biases were taken into consideration, everyone would (and should) score exactly the same on everything, regardless of age, sex, race, gender, ethnicity, orientation, etc etc. I wholeheartedly disagree. You cannot wipe away genetic predisposition and natural talent like that.
dekhn 5y ago
I think it's more likely heritable attributes follow a normal distribution, rather than have a variance of 0. I wouldn't expect everybody to score the same- there seems to be (biologically speaking) a process which leads to variable levels of attainability of intelligence. Anybody who actively denies that men and women have different height distributions is just being non-scientific to avoid pissing off somebody.

by daenz 5y ago

j7ake 5y ago

It would be nice to do a well-designed experiment to ask what are the biological differences in different categories of humans (eg old vs young). Then use that ground truth to assess whether an ML model is generating the biases that deviate from the ground truth. That way there is no a priori assumption of what is technical bias and what is real signal. With the level of investment people are putting into ML technologies, these types of ground truth calibrations would cost only a fraction of the total budget.

joe_the_user 5y ago

Nobody is actually smarter or better than anyone at anything. It's a result of biases and privilege. Where is the linked article claiming this? Your post seems like complete derail of the article discussion for the purposes of making broader political points. [insert hn caveat about being more nuanced in potentially more flamebait topic]

spoonjim 5y ago

Yes, it’s a common “argumentation“ strategy on Twitter. “Do you know that someone was once born with a horse’s head? Clearly the concept of ‘human’ does not exist!”

NicoJuicy 5y ago

I usually mention that nature experiments more with men than with woman. That's an explanation for why top talent are usually men, but the lesser ones are usually men too. ( + I was it was an explanation of a study)

3grdlurker 5y ago

> But what happens if the models still show disparity? You have yet to qualify exactly what you mean by disparity here—it doesn’t help that the example you used (i.e gender-height correlation) isn’t an ML model.

tomrod 5y ago
I agree with your main point, but note that a gender-height correlation is effectively the basis of a linear regression, the most basic of all ML models.

mustafa_pasi 5y ago

That is not my understanding of the ethical dilemma. People are different, and belonging to groups is highly predictive, and that is what ML algorithms depend upon to classify you (every feature is a group). The ethical dilemma is, should you be doomed to be assessed based on your belonging to an arbitrary set of groups? It is highly predictive, but also by definition, discriminatory. Business entities benefit, because on average the predictions are right, but individuals are being discriminated against, and btw, besides the protected categories, other types of discrimination are also unfair.

biasedbrain 5y ago
You should be allowed to think and perceive and notice things, and use tools to help you do so. It is ridiculous to mandate algorithms should be modified to hide statistical facts about the world. by all means, educate people that just because some population is perhaps on average more criminal, it doesn't imply any individual belonging to that population is also more criminal. To hide the statistics would also hurt efforts to find solutions. That can not be a good thing. Edit: if you don't like the criminal example, what about noticing some populations are on average poorer than others? Would that be a good thing, because government money could be diverted their way, or a bad thing, because banks would be less inclined to lend them money, and landlords would be less likely to rent them a home?
- mustafa_pasi 5y ago
  The point of contention is not whether ugly statistical data should be hidden away. That is a stupid conspiracy theory. Sociologists have no problem publishing all kinds of ugly statistical data. The problems start when for profit entities or governments, start deciding to discriminate against individuals because they belong to certain groups. Statistical data applied to individuals is always going to result in injustice. The thought of the economy at large, or the government reducing an individual to a few datapoints and deciding their fate based on group statistics is just about the scariest communist dystopian nightmare, as far as I am concerned. It robs people of their own humanity.

nxpnsv 5y ago

The political debate is endless and not very enlightening. However, this stuff max constitute useful tools to make better models in a non political way. A model that is biased because of too narrow test data will perform worse than a model trained on a suitable dataset. It can be hard to spot these issues, and having some checks and bounds to avoid common sources of bias is a good thing. Furthermore, an analysis might expose more info about the users than intended. This can have pretty bad consequences. For this purpose I thought the one about "collecting sensitive information" was a pretty good start.

phreeza 5y ago

> And if all biases were taken into consideration, everyone would (and should) score exactly the same on everything, regardless of age, sex, race, gender, ethnicity, orientation, etc etc. "Should" is the operational word here. It is a normative decision that these (protected in many jurisdictions) classes should not receive any different treatment due to membership in the class. I agree with this goal, it seems you do not? Or are you objecting to a normative decision being disguised as a descriptive one? In that case I think it is largely a straw man, it's not how most people actually think.

bordercases 5y ago
> In that case I think it is largely a straw man, it's not how most people actually think. Why do you believe this? Even the notion of bias, which you must conceive as real for your claim to hold any weight, is a version of the descriptive being confused for the normative. And for biases to be as problematic as they are considered, they must be common in key places. Why couldn't researchers have an unreasonable bias against reasonable biases?

Jack000 5y ago

It shouldn't be controversial to say that on average, men are taller than women. However it is a problem to say that "since you are a woman, I infer that you must be short". The difference is (not really) subtle but the first statement can be true while the latter statement is obviously fallacious. With an objective quantity like height it's easy to see the error, but harder with less well-defined quantities like intelligence, character, etc.. It's frustrating to see people talk past each other with ill-defined terms.

daenz 5y ago
>It shouldn't be controversial to say that on average, men are taller than women. However it is a problem to say that "since you are a woman, I infer that you must be short". The difference is (not really) subtle but the first statement can be true while the latter statement is obviously fallacious. I agree, but the person I was speaking about specifically was the former, not the latter. They were extremely resistant to the idea of measurable biological differences. This is an intelligent, science-believing person. I think there are a few main groups that people fall into around these kinds of subjects: 1) People who suspect that there may be differences in groups of people, but know that acknowledging them will probably lead to arguments that support sexism, racism, homophobia, transphobia, etc (because they have historically), and so they avoid acknowledging any differences that can't be explained by bias. 2) People who want there to be differences in people so that they can justify their biases against groups of people. 3) People who don't want there to be differences in people, because it doesn't align with a fair worldview that they want to be real. 4) Everyone else who just want people to be honest about potentially uncomfortable truths.
- jimmygrapes 5y ago
  Where are all the #4s?
  - deleted 5y ago
    [deleted]
  - mhuffman 5y ago

rocknor 5y ago

No need to spread FUD here. What you're saying isn't really happening and is not a problem. Mountains are being made out of molehills to spread FUD and derail actual useful discussion for the purposes of making a political point. Show us some proof and then we can talk.

nl 5y ago

> Nobody is actually smarter or better than anyone at anything.. <snip> That's the underlying message about ML initiatives that touch sensitive topics. No it's not, and if you think that you are misreading it. The general point of ML ethics is that a machine learning system should not reinforce bias caused by historically biased training data. This is good statistical practice, but unfortunately often gets lost in the "magic AI" bubble. Avoiding this is like saying "a salary of $110K/year makes you a rich person". It's sort of based on something true (it does put you in the the top quartile of US earners), but is completely useless without correcting at least for location and number of dependants. That's what ML Ethics is about. > I've seen smart tech people unable to say that men are on average taller than women because (I believe) they were afraid to say that some "favorable" outcome (tall) was a result of some component that could be genetic. So you've taken that dumb argument and said all ML ethics arguments are the same?

tbrownaw 5y ago
> The general point of ML ethics is that a machine learning system should not reinforce bias caused by historically biased training data. When I think of "biased training data", I think of data that systematically misrepresents reality. Like say taking the prevalence of news reports as a proxy for the prevalence of what's being reported on. In contrast, the AI ethics people seem to use that term to mean data that doesn't reflect just (as opposed to unjust) causality. I think there's a deeper issue, which is summed up in the saying "when a measure becomes a target, it ceases to be a good measure": simple correlation-finders - which is what current ML/AI models are - are not fit for the purpose of building control loops. And the "ethics" people are trying to paper this over by manually fixing up all the correlations that people notice aren't (or shouldn't be) casual.
- nl 5y ago
  The post linked to for this story gives the example of searching for "CEO" only showing white men. The reason for that is almost entirely the same as your example of biased training data ("taking the prevalence of news reports as a proxy for the prevalence of what's being reported on"). CEOs from large US companies get reported on the most in English language press. Therefore searching in English for "CEO" finds those people.

DoreenMichele 5y ago

I've seen smart tech people unable to say that men are on average taller than women because (I believe) they were afraid to say that some "favorable" outcome (tall) was a result of some component that could be genetic. I've taken some pertinent college classes, like Intro to Psychology and Social Psychology. These are really hard things to debate in good faith because people look around them at the world, see the reality that men are, on average, taller than women on average and assume genetics when we don't actually know the reason. So if you have good faith reasons to wonder put loud or speculate about such things, you are likely to get treated dismissively and not taken seriously. There is a huge component of "them that has, gets" and it's really hard to sort out exactly why x person got y outcome. If you are interested in social justice -- or even just making your own life work when you are the "wrong" demographic -- then these are really frustrating discussions to try to have. It may be some hypothetical argument for someone whose life works well, but it's often a desperate attempt to get a fair shake of some sort by marginalized people. So there's a lot to unpack there and it's hard to unpack it and there are legitimate reasons why some people feel strongly compelled to not go along with "the obvious answer" that "everyone" knows to be true, even though they may not know how to effectively argue against it.

lambdaphagy 5y ago
In surveys of hundreds of human cultures, I don't believe anthropologists have found a single one in which women are, on average, even as tall as men. Though average height differs considerably between groups, there appears to be a consistent bias favoring men (see Fig 1. https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.1558-5646.1992.tb01997.x https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.1558-5646....). We also know that although secular changes in diet can shift population averages over time, the heritability of height is pretty high: most estimates end up at about 0.8. If your parents were short, getting adopted into a tall family is not going to help you become taller. Height is a highly polygenic trait and no single gene controls more than a few percent of the variance. When people say that we don't know the "genes for height", that's what they mean, not that the jury is still out on whether height has any genetic basis at all.
- 5y ago

trhway 5y ago

>Nobody is actually smarter or better than anyone at anything. It's a result of biases and privilege. that statement is just incompatible with the basics of natural selection.

thefz 5y ago

> Nobody is actually smarter or better than anyone at anything. It's a result of biases and privilege. And if all biases were taken into consideration, everyone would (and should) score exactly the same on everything, regardless of age, sex, race, gender, ethnicity, orientation, etc etc. I wholeheartedly disagree. You cannot wipe away genetic predisposition and natural talent like that.

dekhn 5y ago

I think it's more likely heritable attributes follow a normal distribution, rather than have a variance of 0. I wouldn't expect everybody to score the same- there seems to be (biologically speaking) a process which leads to variable levels of attainability of intelligence. Anybody who actively denies that men and women have different height distributions is just being non-scientific to avoid pissing off somebody.