25 ms·

"say you're training a model to detect criminality based on facial structure. This has come up as a real world example, papers have been published on this topic

"say you're training a model to detect criminality based on facial structure. This has come up as a real world example, papers have been published on this topic. What does a "good" dataset look like?" I don't think anybody who is respected says "here is this data set of criminals, we have trained the algorithm on it, and therefore it is proven that such and such facial features predict criminality". I mean yeah this mistake has been made over and over again (even before the invention of computers), but it has long been debunked. Also presumably "black skin" is a good predictor for criminality - in the current day, the crime rate is higher for black people. The algorithm only detects that, it doesn't interpret it. It is up to the humans who use the algorithm to interpret it. If you interpret it as "black people have a genetic disposition to criminality", you are wrong. But it wouldn't be the fault of the algorithm. What is insanity, but basically what the "AI ethics" people demand, is to tweak the algorithms to make them pretend the prevalence of criminality is not higher in certain populations. "But when such "bugs" aren't prioritized because people don't think they are bugs, you have to debate whether or not they are bugs at all!" Nobody says they are not bugs. You are creating an imaginary problem here. You really think, say, researchers at Amazon said "let's make it so that women are ranked down by the algorithm"? Likewise I don't think anybody says "the algorithm should rank black people worse for crime". It is also not a novel idea to look out for bias int he algorithms, delivered to us by the woke crowd. The whole field is about treating bias - a machine learning algorithm is all about training some bias.

joshuamorton 5y ago
> Nobody says they are not bugs. You are creating an imaginary problem here. You really think, say, researchers at Amazon said "let's make it so that women are ranked down by the algorithm"? Likewise I don't think anybody says "the algorithm should rank black people worse for crime". They did though, at least until Gebru and those like her came along and forced the issue. It's really sad to see people say that this was never a concern as though bias and ethics were taken seriously by the field as a whole more than, say, 5 years ago. They weren't. Idk if you're new to the field or weren't paying attention, but it just wasn't a thing. Like most of the foundational papers in terms of racial misclassification and such are from 2017 and 2018.[2] It's more recent than...GANs or AlphaZero. Not to mention that there's attempts to publish garbage like this[1] every year! > You really think, say, researchers at Amazon said "let's make it so that women are ranked down by the algorithm"? Likewise I don't think anybody says "the algorithm should rank black people worse for crime". No, I already said this. Someone failing to notice a bug isn't malice. But there issue is that no one even considered that these kinds of things were bugs so they didn't get noticed or researched. > The whole field is about treating bias - a machine learning algorithm is all about training some bias. Yes, but thinking about race as a particular category where we should avoid unintended bias (and indeed prefer generalization across categories) was a novel idea when proposed by those ethicists! > But it wouldn't be the fault of the algorithm. What is insanity, but basically what the "AI ethics" people demand, is to tweak the algorithms to make them pretend the prevalence of criminality is not higher in certain populations. But...you're making the algorithm. If your goal is to build a model that tries to detect "racial criminality", I'm going to suggest that you probably are doing something racist, because there isn't really a useful, non-racist, reason to train a model that incorrectly classifies people as criminal based on their skin color. On the other hand, if you're having to do additional interpretation of the model output, why aren't you integrating that additional interpretation into the model? And if you can't, then is the model even adding any value? Probably not. And that's not even ignoring questions like what "prevalence of criminality". I think you mean "are arrested more often". We often think that that correlates with criminality, and for some crimes it may, but for e.g. drug crimes we know that it doesn't. The point is, if you don't have at least thoughtful answers to all of those questions and more, you have no business trying to do "criminality" prediction, because your algorithm is not doing whatever you think its doing. [1]: https://www.bbc.com/news/technology-53165286 https://www.bbc.com/news/technology-53165286 [2]: Seriously, Gender Shades is 2018, Debiasing word embeddings is 2016 which I think is the earliest you could argue people were taking this stuff seriously, and it cites "Unequal Representation and Gender Stereotypes in Image Search Results for Occupations" from 2015, which is kind of it.
- horrified 5y ago
  "> Nobody says they are not bugs. You are creating an imaginary problem here. You really think, say, researchers at Amazon said "let's make it so that women are ranked down by the algorithm"? Likewise I don't think anybody says "the algorithm should rank black people worse for crime". They did though, at least until Gebru and those like her came along and forced the issue." What do you mean? Can you point out examples? I think you are flat out wrong when you claim nobody would have cared for racist algorithms before "Ai ethics" was invented. " Like most of the foundational papers in terms of racial misclassification and such are from 2017 and 2018.[2] " Because there isn't anything special about "racial misclassification". It is just "misclassification". There don't need to be any papers about "racial misclassification". You can point out that if people at some university train an algorithm on their mostly white peers for starters, it will not learn to recognise black skinned faces. But that is not specifically an issue of racism. "AI Ethics" people can provide some limited value by providing better data sets for people to work with. But that would have happened without the "ethics" part, simply because people want their algorithms to be as good as possible. At first it is an achievement if it can recognise your white friends. But then you want more, so you will start to collect more data and so on. There is nothing racist about it, unless you want to accuse people with predominantly white friends of racism. Which is exactly why this "AI ethics" stuff is toxic. "But there issue is that no one even considered that these kinds of things were bugs so they didn't get noticed or researched." DO you have a citation? When was it not considered a problem id an algorithm misclassified people by race or gender? "Yes, but thinking about race as a particular category where we should avoid unintended bias (and indeed prefer generalization across categories) was a novel idea when proposed by those ethicists!" And that, as said above, is simply nonsense. That is exactly what is wrong about these people and their approach. It is just grievance politics, aiming for control and cushy extra jobs (every AI department now needs to hire some "AI ethics" people to avoid being called racist and to stave off government regulation). "If your goal is to build a model that tries to detect "racial criminality", I'm going to suggest that you probably are doing something racist, because there isn't really a useful, non-racist, reason to train a model that incorrectly classifies people as criminal based on their skin color." Sure - but you don't need "AI ethics" people to point that out. In fact "AI ethics" people wouldn't even help with that. If somebody sets out to prove some race is genetically predisposed to criminality, they are beyond "AI ethics" territory. You cited " if you're having to do additional interpretation of the model output," I don't follow, what are you referring to? I said "AI ethics" people want to tweak the algorithms to make them blind to actual prevalences of criminality. That is a different thing - I don't think that should be done. You cited [1] of police trying to detect criminality in a face, but they don't set out to show black people are more criminal. It is a different thing, with pitfalls, no doubt. But the problem is not in the AI. Assume black people are more often criminal, what is the proper reaction. Is it OK for police to pay higher attention to black people or not? That is not an AI question or "AI ethics". I don't think there is a simple answer, either. "I think you mean "are arrested more often"." No, I think violent crimes show the prevalence is higher. Like I think black people are six times more likely to be murderers than other races. You can not simply explain that away with "black people are more likely to be investigated for murder". There is a valid discussion about unfair arrests, but there is still a clear indication of higher prevalence of criminality.

joshuamorton 5y ago

> Nobody says they are not bugs. You are creating an imaginary problem here. You really think, say, researchers at Amazon said "let's make it so that women are ranked down by the algorithm"? Likewise I don't think anybody says "the algorithm should rank black people worse for crime". They did though, at least until Gebru and those like her came along and forced the issue. It's really sad to see people say that this was never a concern as though bias and ethics were taken seriously by the field as a whole more than, say, 5 years ago. They weren't. Idk if you're new to the field or weren't paying attention, but it just wasn't a thing. Like most of the foundational papers in terms of racial misclassification and such are from 2017 and 2018.[2] It's more recent than...GANs or AlphaZero. Not to mention that there's attempts to publish garbage like this[1] every year! > You really think, say, researchers at Amazon said "let's make it so that women are ranked down by the algorithm"? Likewise I don't think anybody says "the algorithm should rank black people worse for crime". No, I already said this. Someone failing to notice a bug isn't malice. But there issue is that no one even considered that these kinds of things were bugs so they didn't get noticed or researched. > The whole field is about treating bias - a machine learning algorithm is all about training some bias. Yes, but thinking about race as a particular category where we should avoid unintended bias (and indeed prefer generalization across categories) was a novel idea when proposed by those ethicists! > But it wouldn't be the fault of the algorithm. What is insanity, but basically what the "AI ethics" people demand, is to tweak the algorithms to make them pretend the prevalence of criminality is not higher in certain populations. But...you're making the algorithm. If your goal is to build a model that tries to detect "racial criminality", I'm going to suggest that you probably are doing something racist, because there isn't really a useful, non-racist, reason to train a model that incorrectly classifies people as criminal based on their skin color. On the other hand, if you're having to do additional interpretation of the model output, why aren't you integrating that additional interpretation into the model? And if you can't, then is the model even adding any value? Probably not. And that's not even ignoring questions like what "prevalence of criminality". I think you mean "are arrested more often". We often think that that correlates with criminality, and for some crimes it may, but for e.g. drug crimes we know that it doesn't. The point is, if you don't have at least thoughtful answers to all of those questions and more, you have no business trying to do "criminality" prediction, because your algorithm is not doing whatever you think its doing. [1]: https://www.bbc.com/news/technology-53165286 https://www.bbc.com/news/technology-53165286 [2]: Seriously, Gender Shades is 2018, Debiasing word embeddings is 2016 which I think is the earliest you could argue people were taking this stuff seriously, and it cites "Unequal Representation and Gender Stereotypes in Image Search Results for Occupations" from 2015, which is kind of it.

horrified 5y ago
"> Nobody says they are not bugs. You are creating an imaginary problem here. You really think, say, researchers at Amazon said "let's make it so that women are ranked down by the algorithm"? Likewise I don't think anybody says "the algorithm should rank black people worse for crime". They did though, at least until Gebru and those like her came along and forced the issue." What do you mean? Can you point out examples? I think you are flat out wrong when you claim nobody would have cared for racist algorithms before "Ai ethics" was invented. " Like most of the foundational papers in terms of racial misclassification and such are from 2017 and 2018.[2] " Because there isn't anything special about "racial misclassification". It is just "misclassification". There don't need to be any papers about "racial misclassification". You can point out that if people at some university train an algorithm on their mostly white peers for starters, it will not learn to recognise black skinned faces. But that is not specifically an issue of racism. "AI Ethics" people can provide some limited value by providing better data sets for people to work with. But that would have happened without the "ethics" part, simply because people want their algorithms to be as good as possible. At first it is an achievement if it can recognise your white friends. But then you want more, so you will start to collect more data and so on. There is nothing racist about it, unless you want to accuse people with predominantly white friends of racism. Which is exactly why this "AI ethics" stuff is toxic. "But there issue is that no one even considered that these kinds of things were bugs so they didn't get noticed or researched." DO you have a citation? When was it not considered a problem id an algorithm misclassified people by race or gender? "Yes, but thinking about race as a particular category where we should avoid unintended bias (and indeed prefer generalization across categories) was a novel idea when proposed by those ethicists!" And that, as said above, is simply nonsense. That is exactly what is wrong about these people and their approach. It is just grievance politics, aiming for control and cushy extra jobs (every AI department now needs to hire some "AI ethics" people to avoid being called racist and to stave off government regulation). "If your goal is to build a model that tries to detect "racial criminality", I'm going to suggest that you probably are doing something racist, because there isn't really a useful, non-racist, reason to train a model that incorrectly classifies people as criminal based on their skin color." Sure - but you don't need "AI ethics" people to point that out. In fact "AI ethics" people wouldn't even help with that. If somebody sets out to prove some race is genetically predisposed to criminality, they are beyond "AI ethics" territory. You cited " if you're having to do additional interpretation of the model output," I don't follow, what are you referring to? I said "AI ethics" people want to tweak the algorithms to make them blind to actual prevalences of criminality. That is a different thing - I don't think that should be done. You cited [1] of police trying to detect criminality in a face, but they don't set out to show black people are more criminal. It is a different thing, with pitfalls, no doubt. But the problem is not in the AI. Assume black people are more often criminal, what is the proper reaction. Is it OK for police to pay higher attention to black people or not? That is not an AI question or "AI ethics". I don't think there is a simple answer, either. "I think you mean "are arrested more often"." No, I think violent crimes show the prevalence is higher. Like I think black people are six times more likely to be murderers than other races. You can not simply explain that away with "black people are more likely to be investigated for murder". There is a valid discussion about unfair arrests, but there is still a clear indication of higher prevalence of criminality.