6 ms·

Coauthor here. Some of the press articles about our work didn't have a lot of nuance (unsurprisingly), but in the paper we're careful about what we say, what we

Coauthor here. Some of the press articles about our work didn't have a lot of nuance (unsurprisingly), but in the paper we're careful about what we say, what we don't say, and what the implications are. Happy to engage in informed discussion :)

fpp 9y ago
very interesting topic - could you please share a link to the original paper. best probably to read that one first.
- privong 9y ago
  > very interesting topic - could you please share a link to the original paper. Unless the link was changed in the few minutes since you posted your comment, the link for the article is the original Science paper (http://science.sciencemag.org/content/356/6334/183.full http://science.sciencemag.org/content/356/6334/183.full)
  - pasbesoin 9y ago
    (From a Javascript-disabled perspective) Page with actual link: http://science.sciencemag.org/content/356/6334/183/tab-pdf http://science.sciencemag.org/content/356/6334/183/tab-pdf Link to PDF itself: http://science.sciencemag.org/content/sci/356/6334/183.full.pdf http://science.sciencemag.org/content/sci/356/6334/183.full....
  - fpp 9y ago
    sorry mixed it up. If I look at glove & WordNet usage e.g. for topic extraction, bagging / clustering or semantic similarity would you say we would need to get rid of such a bias, e.g. create something like a Geiger counter for NLP. Alternative view - when doing sentiment analysis / classification would you say that such a bias actually helps to identify a type of sentiment in a doc / sentence.
    - russdpale 9y ago
      Wouldn't this lead to an entire of idea of contextual bias? Times when it could benefit and be used, and times where it is occluded.
glibgil 9y ago
Can you provide an example of how this bias might play out in a human-AI interaction?
- andreasvc 9y ago
  It appears that the linked paper has examples.
- yummyfajitas 9y ago
  The paper has them: - An AI correctly infers (simply by reading text) that a physicist is male and a nurse is female. - An AI correctly infers the gender of humans with androgyonous names. - An AI infers insects are unpleasant and flowers are pleasant to humans. - An AI also infers that African American names are more likely to be associated with unpleasantness than European names. [edit: to those who dislike this comment, can you tell me what you object to? Which of my concrete examples is not in the paper?]
yummyfajitas 9y ago
Do you have any evidence that this effect results in machines making systematically wrong inferences? Near as I can tell, your paper shows that these "biases" result in significantly more accurate predictions. For example, Fig 1 shows that a machine trained on human language can accurately predict the % female of many professions. Fig 2 shows the machine can accurately predict the gender of humans. Normally I'd expect a "bias" to result in wrong predictions - but in this case (due to an unusual redefinition of "bias") the exact opposite seems to occur. (Drawing on your analogy with stereotypes, it's probably also worth linking to a pointer on stereotype accuracy: http://emilkirkegaard.dk/en/wp-content/uploads/Jussim-et-al-unbearable-accuracy-of-stereotypes.pdf http://emilkirkegaard.dk/en/wp-content/uploads/Jussim-et-al-... http://spsp.org/blog/stereotype-accuracy-response http://spsp.org/blog/stereotype-accuracy-response )
- andreasvc 9y ago
  I think your questions would be answered by reading the article. Particularly: "In AI and machine learning, bias refers generally to prior information, a necessary prerequisite for intelligent action (4). Yet bias can be problematic where such information is derived from aspects of human culture known to lead to harmful behavior. Here, we will call such biases “stereotyped” and actions taken on their basis “prejudiced.”" This definition is not unusual. This is about inferences that are wrong in the sense of prejudiced, not necessarily inaccurate.
  - yummyfajitas 9y ago
    The usual definition of bias in ML papers is E[theta_estimator - theta]. That is explicitly a systematically wrong prediction. In any case, the paper suggests that this "bias" or "prejudice" is better described as "truths I don't like". I'm asking if the author knows of any cases where they are actually not truthful. The paper does not suggest any, but maybe there are some?
    - deleted 9y ago
      [deleted]
    - 9y ago
EGreg 9y ago
Isn't the word bias being redefined by a social justice point of view? Normally bias would be with reference to failing to match reality (eg women in general have physically weaker upper body than men), and not failing to match whatever standard of equality a society wishes were the case eventually.
- sp332 9y ago
  You should check out the Implicit Association Test that they used to measure the biases. Just as one example, there's nothing about being a doctor that is inherently more male or female. So all gender differences would have external causes.
ylem 9y ago
Hi! I just read the paper--impressive work! Have you tried any other languages? For example, French or German?
slackstation 9y ago
Since we think of biases of a large human corpora as wrong, I'm curious how one would find one or make one that is "right". Given how accurate human corpora is at predicting things like gender distribution in jobs for instance, wouldn't making an "unbiased" corpora make an inaccurate AI? Shouldn't we be careful in implying things like the biases and solutions to said biases? For instance, I'd like to know if my algorithm for filtering job applicants is trying to undo the injustices of the world in addition to finding the best candidates.

fpp 9y ago

very interesting topic - could you please share a link to the original paper. best probably to read that one first.

privong 9y ago
> very interesting topic - could you please share a link to the original paper. Unless the link was changed in the few minutes since you posted your comment, the link for the article is the original Science paper (http://science.sciencemag.org/content/356/6334/183.full http://science.sciencemag.org/content/356/6334/183.full)
- pasbesoin 9y ago
  (From a Javascript-disabled perspective) Page with actual link: http://science.sciencemag.org/content/356/6334/183/tab-pdf http://science.sciencemag.org/content/356/6334/183/tab-pdf Link to PDF itself: http://science.sciencemag.org/content/sci/356/6334/183.full.pdf http://science.sciencemag.org/content/sci/356/6334/183.full....
- fpp 9y ago
  sorry mixed it up. If I look at glove & WordNet usage e.g. for topic extraction, bagging / clustering or semantic similarity would you say we would need to get rid of such a bias, e.g. create something like a Geiger counter for NLP. Alternative view - when doing sentiment analysis / classification would you say that such a bias actually helps to identify a type of sentiment in a doc / sentence.
  - russdpale 9y ago
    Wouldn't this lead to an entire of idea of contextual bias? Times when it could benefit and be used, and times where it is occluded.

glibgil 9y ago

Can you provide an example of how this bias might play out in a human-AI interaction?

andreasvc 9y ago
It appears that the linked paper has examples.
yummyfajitas 9y ago
The paper has them: - An AI correctly infers (simply by reading text) that a physicist is male and a nurse is female. - An AI correctly infers the gender of humans with androgyonous names. - An AI infers insects are unpleasant and flowers are pleasant to humans. - An AI also infers that African American names are more likely to be associated with unpleasantness than European names. [edit: to those who dislike this comment, can you tell me what you object to? Which of my concrete examples is not in the paper?]

yummyfajitas 9y ago

Do you have any evidence that this effect results in machines making systematically wrong inferences? Near as I can tell, your paper shows that these "biases" result in significantly more accurate predictions. For example, Fig 1 shows that a machine trained on human language can accurately predict the % female of many professions. Fig 2 shows the machine can accurately predict the gender of humans. Normally I'd expect a "bias" to result in wrong predictions - but in this case (due to an unusual redefinition of "bias") the exact opposite seems to occur. (Drawing on your analogy with stereotypes, it's probably also worth linking to a pointer on stereotype accuracy: http://emilkirkegaard.dk/en/wp-content/uploads/Jussim-et-al-unbearable-accuracy-of-stereotypes.pdf http://emilkirkegaard.dk/en/wp-content/uploads/Jussim-et-al-... http://spsp.org/blog/stereotype-accuracy-response http://spsp.org/blog/stereotype-accuracy-response )

andreasvc 9y ago
I think your questions would be answered by reading the article. Particularly: "In AI and machine learning, bias refers generally to prior information, a necessary prerequisite for intelligent action (4). Yet bias can be problematic where such information is derived from aspects of human culture known to lead to harmful behavior. Here, we will call such biases “stereotyped” and actions taken on their basis “prejudiced.”" This definition is not unusual. This is about inferences that are wrong in the sense of prejudiced, not necessarily inaccurate.
- yummyfajitas 9y ago
  The usual definition of bias in ML papers is E[theta_estimator - theta]. That is explicitly a systematically wrong prediction. In any case, the paper suggests that this "bias" or "prejudice" is better described as "truths I don't like". I'm asking if the author knows of any cases where they are actually not truthful. The paper does not suggest any, but maybe there are some?
  - deleted 9y ago
    [deleted]
  - 9y ago

EGreg 9y ago

Isn't the word bias being redefined by a social justice point of view? Normally bias would be with reference to failing to match reality (eg women in general have physically weaker upper body than men), and not failing to match whatever standard of equality a society wishes were the case eventually.

sp332 9y ago
You should check out the Implicit Association Test that they used to measure the biases. Just as one example, there's nothing about being a doctor that is inherently more male or female. So all gender differences would have external causes.

ylem 9y ago

Hi! I just read the paper--impressive work! Have you tried any other languages? For example, French or German?

slackstation 9y ago

Since we think of biases of a large human corpora as wrong, I'm curious how one would find one or make one that is "right". Given how accurate human corpora is at predicting things like gender distribution in jobs for instance, wouldn't making an "unbiased" corpora make an inaccurate AI? Shouldn't we be careful in implying things like the biases and solutions to said biases? For instance, I'd like to know if my algorithm for filtering job applicants is trying to undo the injustices of the world in addition to finding the best candidates.