4 ms·

> De-biasing models is necessary. Anyone who claims otherwise... I just don't think that's a defensible position to take if you've ever worked with large datase

> De-biasing models is necessary. Anyone who claims otherwise... I just don't think that's a defensible position to take if you've ever worked with large datasets. I don't know how to be more clear: nowhere in this thread have I argued for using raw models without any de-biasing. Twice now you have ascribed this position to me. Are you reading what I have actually written? > but it is extremely reasonable for AI researches to say "Internet data biases against certain groups and we would like to correct for that bias when training models." The idea that "Internet data biases against certain groups" seems itself a biased statement. I am pretty sure that Internet data is biased against all groups. I bet you can get a raw LLM to say offensive things about any group if you give it the right prompt.

danShumway 4y ago
> I don't know how to be more clear: nowhere in this thread have I argued for using raw models without any de-biasing. Okay, great. But then we are arguing about an implementation, aren't we? We agree that de-biasing is necessary. It's just that you seem to think that calling out specific biases is the wrong way to approach that. But I'm curious how you're supposed to de-bias data without drawing any attention to or acknowledging the groups that the data is biased against? > The idea that "Internet data biases against certain groups" seems itself a biased statement. If a data set is biased, it's biased in a direction. By definition, that's what bias is. > I am pretty sure that Internet data is biased against all groups. I bet you can get a raw LLM to say offensive things about any group if you give it the right prompt. That's not what bias is. Bias is not "can you get it to say something offensive" -- bias is a systematic predisposition in a direction. Bias is not stuff like, "it made a dirty joke", bias is stuff like, "this hiring model is more likely to hire you if your name sounds white." And the goal here (believe it or not when I say this) is not to make models that never make offensive jokes. The goal is to avoid making models that reproduce and reinforce systemic issues in society. It's entirely appropriate when looking at the problem of "what social inequities are being reproduced by a model" to identify specific social inequities and inequalities that exist in society. ---- Part of the reason why I've been assuming that you're against de-biasing in general is that you keep saying stuff like "the idea that the Internet biases against certain groups is itself a biased statement." That is nonsensical, it seems like you're suggesting that companies should be correcting against bias without determining what the direction of that bias is or who it affects. How exactly should they do that? What does it mean to correct a systemic skew without identifying what the skew is? Again, look at the Reddit example. Yes, you can find offensive stuff on Reddit for everyone: atheists, vegans, Conservatives, Liberals, whoever. But de-biasing is not about whether or not something is offensive, it's about identifying and correcting a predisposition or tendency towards certain attitudes and thoughts. And that necessarily requires calling out what those predispositions are -- so you're going to end up with statements like "Reddit is biased against Conservative/Libertarian philosophies overall, on average." And if somebody jumps out and says, "Reddit is biased against everyone, it's itself biased for you to call out that specific bias" -- then there's no response to that other than that's just not how any of this works. If you go into a conversation about data bias saying that it's inappropriate for us to identify specific biases, then you are effectively arguing against reducing bias, regardless of what else you say. And even more to the point, even if that wasn't true -- fixing any systemic problem starts with triaging and identifying specific areas you want to target first. It is so weird to hear someone say, "I've never claimed we shouldn't fix the problem, but pointing out specific groups that the problem applies to is wrong." No, it's identifying problem areas that are high impact: a normal, good thing for researchers to do. To the extent that it's "biased", it's the kind of bias that we want in pretty much every field and on every subject -- we want researchers and debuggers to bias themselves towards focusing on high-impact areas first when they're fixing problems. And we want them to call out specific areas that they think deserve attention. That's what triaging is.
- throwaway322112 4y ago
  [flagged]
  - danShumway 4y ago
    > Ok, let's go with your definition. "My" definition: https://www.merriam-webster.com/dictionary/bias https://www.merriam-webster.com/dictionary/bias > an inclination of temperament or outlook > : systematic error introduced into sampling or testing by selecting or encouraging one outcome or answer over others I don't understand what you're trying to get at here, what I'm telling you is just the definition of bias that everybody uses. This is not a debatable thing, this is not some weird interpretation of bias that I made up :) Bias is a directional skew. It just is, that is what everybody means by the word. ---- > From these tests I conclude that the systemic bias is against men, white people, and Republicans, and in favor of women, black people, and Democrats. A fair conclusion? And we get to the heart of it, same as always. It turns out that when you prod these "philosophical" discussions what people actually mean is: "I disagree that those minority groups are oppressed, actually I'm oppressed." It's never actually about the philosophy, you just disagree about which groups Mozilla is trying to help. It's not about the "bias" it's about which groups Mozilla thinks that LLMs demonstrate bias against. It's not about about the process or the theory, it's about who the process and the theory are being applied to. Which, whatever, you disagree with Mozilla's perspective on how the data is biased and you think that actually the bias is against you. You could save us all a lot of time by starting with that claim instead of dressing it up as some kind of nonsensical take about methodology in correcting data skew. ---- Anyway, to your nonsense gotcha questions: 1. Sex discrimination is illegal, it would be wildly inappropriate for a police department to rely on an AI that dismissed a suspect because they were a woman. 2. LLMs don't get used to choose basketball players? But if they were, yeah, it would be a problem if an LLM dismissed a resume (again, not really how basketball works) because someone had a white-sounding name. 3. I literally brought up the example of Reddit. That's not a gotcha, I brought up that if you built an LLM on Reddit data it would be biased towards calling Republicans racist. Now if you don't think that's an unfair bias and you think Republicans are actually more likely to be racist, then that's your words, not mine. My words were that if you trained an LLM on a primarily Liberal forum, it would be biased against Conservatives and there would likely be alignment training you'd need to do on that LLM. ---- Now, are any of those larger issues than systemic racism? I would argue no. I would argue that Mozilla is still absolutely completely correct in triaging these issues and pointing out the most harmful effects of AI today. We don't have a lot of examples of LLMs systematically harming specifically Republicans. And I'm going to go out on a limb and say that's really the biggest thing you have issue with here -- you have issue with Mozilla's prioritization of harms and prioritization of biases to focus on, because you don't think the biases Mozilla has pointed out are actually a big deal. You brought up those gotcha questions to try and say, "look, bias against white male Conservatives is where the most harm actually occurs". And that's actually the position that we disagree on. All of the "philosophy" about bias in models is just distraction from that disagreement.

danShumway 4y ago

> I don't know how to be more clear: nowhere in this thread have I argued for using raw models without any de-biasing. Okay, great. But then we are arguing about an implementation, aren't we? We agree that de-biasing is necessary. It's just that you seem to think that calling out specific biases is the wrong way to approach that. But I'm curious how you're supposed to de-bias data without drawing any attention to or acknowledging the groups that the data is biased against? > The idea that "Internet data biases against certain groups" seems itself a biased statement. If a data set is biased, it's biased in a direction. By definition, that's what bias is. > I am pretty sure that Internet data is biased against all groups. I bet you can get a raw LLM to say offensive things about any group if you give it the right prompt. That's not what bias is. Bias is not "can you get it to say something offensive" -- bias is a systematic predisposition in a direction. Bias is not stuff like, "it made a dirty joke", bias is stuff like, "this hiring model is more likely to hire you if your name sounds white." And the goal here (believe it or not when I say this) is not to make models that never make offensive jokes. The goal is to avoid making models that reproduce and reinforce systemic issues in society. It's entirely appropriate when looking at the problem of "what social inequities are being reproduced by a model" to identify specific social inequities and inequalities that exist in society. ---- Part of the reason why I've been assuming that you're against de-biasing in general is that you keep saying stuff like "the idea that the Internet biases against certain groups is itself a biased statement." That is nonsensical, it seems like you're suggesting that companies should be correcting against bias without determining what the direction of that bias is or who it affects. How exactly should they do that? What does it mean to correct a systemic skew without identifying what the skew is? Again, look at the Reddit example. Yes, you can find offensive stuff on Reddit for everyone: atheists, vegans, Conservatives, Liberals, whoever. But de-biasing is not about whether or not something is offensive, it's about identifying and correcting a predisposition or tendency towards certain attitudes and thoughts. And that necessarily requires calling out what those predispositions are -- so you're going to end up with statements like "Reddit is biased against Conservative/Libertarian philosophies overall, on average." And if somebody jumps out and says, "Reddit is biased against everyone, it's itself biased for you to call out that specific bias" -- then there's no response to that other than that's just not how any of this works. If you go into a conversation about data bias saying that it's inappropriate for us to identify specific biases, then you are effectively arguing against reducing bias, regardless of what else you say. And even more to the point, even if that wasn't true -- fixing any systemic problem starts with triaging and identifying specific areas you want to target first. It is so weird to hear someone say, "I've never claimed we shouldn't fix the problem, but pointing out specific groups that the problem applies to is wrong." No, it's identifying problem areas that are high impact: a normal, good thing for researchers to do. To the extent that it's "biased", it's the kind of bias that we want in pretty much every field and on every subject -- we want researchers and debuggers to bias themselves towards focusing on high-impact areas first when they're fixing problems. And we want them to call out specific areas that they think deserve attention. That's what triaging is.

throwaway322112 4y ago
[flagged]
- danShumway 4y ago
  > Ok, let's go with your definition. "My" definition: https://www.merriam-webster.com/dictionary/bias https://www.merriam-webster.com/dictionary/bias > an inclination of temperament or outlook > : systematic error introduced into sampling or testing by selecting or encouraging one outcome or answer over others I don't understand what you're trying to get at here, what I'm telling you is just the definition of bias that everybody uses. This is not a debatable thing, this is not some weird interpretation of bias that I made up :) Bias is a directional skew. It just is, that is what everybody means by the word. ---- > From these tests I conclude that the systemic bias is against men, white people, and Republicans, and in favor of women, black people, and Democrats. A fair conclusion? And we get to the heart of it, same as always. It turns out that when you prod these "philosophical" discussions what people actually mean is: "I disagree that those minority groups are oppressed, actually I'm oppressed." It's never actually about the philosophy, you just disagree about which groups Mozilla is trying to help. It's not about the "bias" it's about which groups Mozilla thinks that LLMs demonstrate bias against. It's not about about the process or the theory, it's about who the process and the theory are being applied to. Which, whatever, you disagree with Mozilla's perspective on how the data is biased and you think that actually the bias is against you. You could save us all a lot of time by starting with that claim instead of dressing it up as some kind of nonsensical take about methodology in correcting data skew. ---- Anyway, to your nonsense gotcha questions: 1. Sex discrimination is illegal, it would be wildly inappropriate for a police department to rely on an AI that dismissed a suspect because they were a woman. 2. LLMs don't get used to choose basketball players? But if they were, yeah, it would be a problem if an LLM dismissed a resume (again, not really how basketball works) because someone had a white-sounding name. 3. I literally brought up the example of Reddit. That's not a gotcha, I brought up that if you built an LLM on Reddit data it would be biased towards calling Republicans racist. Now if you don't think that's an unfair bias and you think Republicans are actually more likely to be racist, then that's your words, not mine. My words were that if you trained an LLM on a primarily Liberal forum, it would be biased against Conservatives and there would likely be alignment training you'd need to do on that LLM. ---- Now, are any of those larger issues than systemic racism? I would argue no. I would argue that Mozilla is still absolutely completely correct in triaging these issues and pointing out the most harmful effects of AI today. We don't have a lot of examples of LLMs systematically harming specifically Republicans. And I'm going to go out on a limb and say that's really the biggest thing you have issue with here -- you have issue with Mozilla's prioritization of harms and prioritization of biases to focus on, because you don't think the biases Mozilla has pointed out are actually a big deal. You brought up those gotcha questions to try and say, "look, bias against white male Conservatives is where the most harm actually occurs". And that's actually the position that we disagree on. All of the "philosophy" about bias in models is just distraction from that disagreement.