4 ms·

> But what if we found that while Model C performs the best overall, it's also most likely to assign a more positive sentiment to the sentence "The main charact

by troupe 9y ago

> But what if we found that while Model C performs the best overall, it's also most likely to assign a more positive sentiment to the sentence "The main character is a man" than to the sentence "The main character is a woman"? As I understand the problem, they are saying that statistically, the statement about the main character being male is a bit more likely to be positive than if the same thing is said about a woman. If that is statistically true and you are trying to create a model to determine the level of positive sentiment in a review, then that may be a legitimate indicator of how people categorize things. If the goal is to try to "fix" how people talk and write, I'm not sure ignoring statistical patterns in the way we talk is really the right approach.

elliotec 9y ago
Agreed. The machine is biased because biased humans are training it to be biased.
- troupe 9y ago
  Well I think some of it has to do with the fact that language is not completely symmetrical (there may be a better word). For example, the following sentences: 1. They went to college. 2. They went to the University of Phoenix. The fact you are calling out UoP may turn out to be used more in slightly less positive sentiments than the first one. If you are trying to figure out the sentiment of a few sentences, this might be important. Yes you can ignore it, but the question is whether you are trying to understand how the language is actually being used or not.
  - elliotec 9y ago
    True again - biased humans creating biased machines asking biased questions and receiving biased answers.
    - troupe 9y ago
      Keep in mind this wasn't an analysis of whether people liked men or women in the lead role. It was basically pointing out that one statement was a little more likely to be more strongly positive than the other. So you think asking what level of positive (or negative) sentiment does this review express is a biased question?
  - yorwba 9y ago
    A real-world example: 交大不如复旦 [jiāodà bùrú fùdàn] gets translated as "Jiaotong University is not as good as Fudan University" by Google https://translate.google.com/?hl=en#auto/en/%E4%BA%A4%E5%A4%A7%E4%B8%8D%E5%A6%82%E5%A4%8D%E6%97%A6 https://translate.google.com/?hl=en#auto/en/%E4%BA%A4%E5%A4%... 复旦不如交大 [fùdàn bùrú jiāodà] (just swapping the word order) is translated into "Fudan is better than Jiaotong University" https://translate.google.com/?hl=en#auto/en/%E5%A4%8D%E6%97%A6%E4%B8%8D%E5%A6%82%E4%BA%A4%E5%A4%A7 https://translate.google.com/?hl=en#auto/en/%E5%A4%8D%E6%97%... The literal translation of 不如 would be "is unlike", but usually it implies a value judgment. In the translation, Google seems to be consistently sure that Fudan is simply better. But when you specify that you're talking about the Jiaotong University in Shanghai and not one of the others, it suddenly changes its mind: 上海交大不如复旦 [shànghǎi jiāodà bùrú fùdàn] "Shanghai Jiaotong University is better than Fudan University" https://translate.google.com/?hl=en#auto/en/%E4%B8%8A%E6%B5%B7%E4%BA%A4%E5%A4%A7%E4%B8%8D%E5%A6%82%E5%A4%8D%E6%97%A6 https://translate.google.com/?hl=en#auto/en/%E4%B8%8A%E6%B5%... 复旦不如上海交大 [fùdàn bùrú shànghǎi jiāodà] "Fudan is not as good as Shanghai Jiaotong University" https://translate.google.com/?hl=en#auto/en/%E5%A4%8D%E6%97%A6%E4%B8%8D%E5%A6%82%E4%B8%8A%E6%B5%B7%E4%BA%A4%E5%A4%A7 https://translate.google.com/?hl=en#auto/en/%E5%A4%8D%E6%97%... I'm at SJTU and everyone here seems to agree that this is the objective truth, but the people at Fudan are probably not so happy about it.
    - AstralStorm 9y ago
      Unfortunately begging the questions: in what ways and how much? And most importantly: is the difference meaningful? Google Translate and most other alleged artificial intelligence won't be able to answer that question meaningfully.
    - nmca 9y ago
      This is a really excellent example.
    - iguy 9y ago
      This is a great example. It's like an exam question where you have to know what the teacher is thinking. Notice that in "Fudan is better than Jiaotong University" you're supposed to know that Fudan is another place with a university, not (say) a kind of apprenticeship... or a joke about some kind of noodles or something. You're supposed to have some outside context, but not too much, not enough to know their reputations. That's quite a fine line to ask a translation system to draw.
gowld 9y ago
Good, because the article clearly states that the critical takeway is that modelers must pay attention to statistical patterns, and not blithely ignore them.
lsiebert 9y ago
Actually it's any name, even the name of the reviewer, as the test they conducted showed. the issue is that humans don't understand multi-variable statistical analysis, whether in the form of ANOVA or machine learning training, so they try to pack everything down into two or sometimes three variables of output. And that's fine if you are pulling from a population that's homogenous. But if there are two or more discrete subpopulations, you want to control for them or represent them separately, not just ignore them or pretend they reflect the information you want. Anyway if the reviewer name is enough to throw the results, it may suggest that guys praise more movies, rather than movies with male character's get more praise. I just looked at the first page of 10 star reviews of battlefield Earth and none seemed to be female, just saying.
- dibstern 9y ago
  If the multiple sub populations exist, that’ll be reflected in the data, as long as your dataset is good enough.
  - lsiebert 9y ago
    Maybe? But even something as simple as an analysis of human height has genetic/ethnic, nutritional, age and gender components, not to mention historical differences. If your input is name, and your output is predicted height, knowing and testing for bias in your data sources is definitely important. I think where people get caught up is that they don't see the world at large as biased, because they view their understandings as essentially correct. For example, we expect judges to rule fairly on every case, right? To pick a non controversial issue, likelihood of parole is apparently effected by how recently the judge ate https://www.economist.com/node/18557594 https://www.economist.com/node/18557594 Now, parole data accurately reflects how people were paroled, so predicting likelihood of clemency requests is a perfectly valid use of that data. If you were doing machine learning to try to help people get paroled, you'd want to leave that bias in as a predictor, because it's unfair but real. But you'd probably want to adjust that data to correct for the recently having eaten bias if you were writing a system for parole recommendations for new judges based on past judicial decisions. You wouldn't want people to be more likely to be denied just because they came before a judge before lunch. And if you don't test for a bias like that, how would you be able to tell that the machine learning algorithm had it? And it wouldn't even need to be direct... seeing people A-Z in court could mean a bias based on name. Long story short, bias is a real issue, and you need to be aware of it and test for it, not assume that your input data isn't effected by human error.

by troupe 9y ago

elliotec 9y ago

Agreed. The machine is biased because biased humans are training it to be biased.

troupe 9y ago
Well I think some of it has to do with the fact that language is not completely symmetrical (there may be a better word). For example, the following sentences: 1. They went to college. 2. They went to the University of Phoenix. The fact you are calling out UoP may turn out to be used more in slightly less positive sentiments than the first one. If you are trying to figure out the sentiment of a few sentences, this might be important. Yes you can ignore it, but the question is whether you are trying to understand how the language is actually being used or not.
- elliotec 9y ago
  True again - biased humans creating biased machines asking biased questions and receiving biased answers.
  - troupe 9y ago
    Keep in mind this wasn't an analysis of whether people liked men or women in the lead role. It was basically pointing out that one statement was a little more likely to be more strongly positive than the other. So you think asking what level of positive (or negative) sentiment does this review express is a biased question?
- yorwba 9y ago
  A real-world example: 交大不如复旦 [jiāodà bùrú fùdàn] gets translated as "Jiaotong University is not as good as Fudan University" by Google https://translate.google.com/?hl=en#auto/en/%E4%BA%A4%E5%A4%A7%E4%B8%8D%E5%A6%82%E5%A4%8D%E6%97%A6 https://translate.google.com/?hl=en#auto/en/%E4%BA%A4%E5%A4%... 复旦不如交大 [fùdàn bùrú jiāodà] (just swapping the word order) is translated into "Fudan is better than Jiaotong University" https://translate.google.com/?hl=en#auto/en/%E5%A4%8D%E6%97%A6%E4%B8%8D%E5%A6%82%E4%BA%A4%E5%A4%A7 https://translate.google.com/?hl=en#auto/en/%E5%A4%8D%E6%97%... The literal translation of 不如 would be "is unlike", but usually it implies a value judgment. In the translation, Google seems to be consistently sure that Fudan is simply better. But when you specify that you're talking about the Jiaotong University in Shanghai and not one of the others, it suddenly changes its mind: 上海交大不如复旦 [shànghǎi jiāodà bùrú fùdàn] "Shanghai Jiaotong University is better than Fudan University" https://translate.google.com/?hl=en#auto/en/%E4%B8%8A%E6%B5%B7%E4%BA%A4%E5%A4%A7%E4%B8%8D%E5%A6%82%E5%A4%8D%E6%97%A6 https://translate.google.com/?hl=en#auto/en/%E4%B8%8A%E6%B5%... 复旦不如上海交大 [fùdàn bùrú shànghǎi jiāodà] "Fudan is not as good as Shanghai Jiaotong University" https://translate.google.com/?hl=en#auto/en/%E5%A4%8D%E6%97%A6%E4%B8%8D%E5%A6%82%E4%B8%8A%E6%B5%B7%E4%BA%A4%E5%A4%A7 https://translate.google.com/?hl=en#auto/en/%E5%A4%8D%E6%97%... I'm at SJTU and everyone here seems to agree that this is the objective truth, but the people at Fudan are probably not so happy about it.
  - AstralStorm 9y ago
    Unfortunately begging the questions: in what ways and how much? And most importantly: is the difference meaningful? Google Translate and most other alleged artificial intelligence won't be able to answer that question meaningfully.
  - nmca 9y ago
    This is a really excellent example.
  - iguy 9y ago
    This is a great example. It's like an exam question where you have to know what the teacher is thinking. Notice that in "Fudan is better than Jiaotong University" you're supposed to know that Fudan is another place with a university, not (say) a kind of apprenticeship... or a joke about some kind of noodles or something. You're supposed to have some outside context, but not too much, not enough to know their reputations. That's quite a fine line to ask a translation system to draw.

gowld 9y ago

Good, because the article clearly states that the critical takeway is that modelers must pay attention to statistical patterns, and not blithely ignore them.

lsiebert 9y ago

Actually it's any name, even the name of the reviewer, as the test they conducted showed. the issue is that humans don't understand multi-variable statistical analysis, whether in the form of ANOVA or machine learning training, so they try to pack everything down into two or sometimes three variables of output. And that's fine if you are pulling from a population that's homogenous. But if there are two or more discrete subpopulations, you want to control for them or represent them separately, not just ignore them or pretend they reflect the information you want. Anyway if the reviewer name is enough to throw the results, it may suggest that guys praise more movies, rather than movies with male character's get more praise. I just looked at the first page of 10 star reviews of battlefield Earth and none seemed to be female, just saying.

dibstern 9y ago
If the multiple sub populations exist, that’ll be reflected in the data, as long as your dataset is good enough.
- lsiebert 9y ago
  Maybe? But even something as simple as an analysis of human height has genetic/ethnic, nutritional, age and gender components, not to mention historical differences. If your input is name, and your output is predicted height, knowing and testing for bias in your data sources is definitely important. I think where people get caught up is that they don't see the world at large as biased, because they view their understandings as essentially correct. For example, we expect judges to rule fairly on every case, right? To pick a non controversial issue, likelihood of parole is apparently effected by how recently the judge ate https://www.economist.com/node/18557594 https://www.economist.com/node/18557594 Now, parole data accurately reflects how people were paroled, so predicting likelihood of clemency requests is a perfectly valid use of that data. If you were doing machine learning to try to help people get paroled, you'd want to leave that bias in as a predictor, because it's unfair but real. But you'd probably want to adjust that data to correct for the recently having eaten bias if you were writing a system for parole recommendations for new judges based on past judicial decisions. You wouldn't want people to be more likely to be denied just because they came before a judge before lunch. And if you don't test for a bias like that, how would you be able to tell that the machine learning algorithm had it? And it wouldn't even need to be direct... seeing people A-Z in court could mean a bias based on name. Long story short, bias is a real issue, and you need to be aware of it and test for it, not assume that your input data isn't effected by human error.