5 ms·

> the idea that you can just throw data sources at an AI and expect the outcome to be fair is kind of silly. You are arguing against a straw man. We are discu

> the idea that you can just throw data sources at an AI and expect the outcome to be fair is kind of silly. You are arguing against a straw man. We are discussing a set of AI principles, not an implementation of those principles. I claim that any anti-bias principle should be applied equally to all, both in spirit and in practice. That is not the same as assuming the input is neutral, or assuming that de-biasing is unnecessary. By your own account, the researchers who do the de-biasing have a great amount of discretion about "how to fix that and who to focus on." I don't think it's unfair to say that it is their job to put their thumbs on the scale, according to the beliefs and priorities of themselves and their organizations. So how do we know that the model authors aren't just introducing their own bias, either by over-correcting one set of biases from the input, or under-correcting others? It seems likely that "de-biasing" can become "re-biasing." I would already suspect that this is a major risk with AI, but when an organization like Mozilla openly states in their guiding principles that certain groups are a special priority, re-biasing seems all but certain. Of course everyone will bring their own biases to the table when performing a job, that is inevitable. But an AI provider that wants to be trustworthy to a wide group of people should be extremely vigilant about correcting for their own personal biases, and be clear in their messaging that this is a core commitment. OpenAI, to their credit, seem to be taking this seriously: https://openai.com/blog/how-should-ai-systems-behave#addressing-biases https://openai.com/blog/how-should-ai-systems-behave#address...

danShumway 4y ago
> We are discussing a set of AI principles, not an implementation of those principles. If the position is that any implementation of those principles is inherently problematic, then that is nonsense. Quite frankly, I do think we're talking about implementation. De-biasing models is necessary. Anyone who claims otherwise... I just don't think that's a defensible position to take if you've ever worked with large datasets. So if you have a criticism of Mozilla's implementation of its de-biasing, then argue against that implementation. But arguing that there shouldn't be an implementation is ridiculous. ---- > So how do we know that the model authors aren't just introducing their own bias, either by over-correcting one set of biases from the input, or under-correcting others? You heckin don't know. But the alternative to that is not neutral models, it's still biased models -- because the data itself that you train an LLM on is inherently biased. Any large textual database you get off the Internet is going to have bias in it. Choosing to ignore that fact does not give you neutrality. You're very worried about the risk that de-biasing can become re-biasing, and on one hand, you're right, that's a very real risk. Censorship and alignment risks are very real concerns in AI. But you don't seem to be worried about the fact that models that are not "de-biased" are subject to the exact same concerns. Choosing not to de-bias is just saying that you'll have a biased model with all of the same concerns you raise above, except with no effort to correct that bias at all. You're scared of Mozilla pushing people to bias their models in a certain direction, you don't seem to have internalized that the models are already biased in a certain direction. And all of the risks of model bias apply to models that have been uncritically fed raw data. Again, disagree with specific implementations if you want, but it is extremely reasonable for AI researches to say "Internet data biases against certain groups and we would like to correct for that bias when training models." And there's no "neutral" way to do that -- the way you correct for bias is you identify the areas where bias exists and you push back on those trends in the data. Again, I'd bring up Reddit here. If you're trying to build an LLM on Reddit data and you want it to be "fair" to Conservatives/Christians, that means identifying that a platform like Reddit has a very clear Liberal/Athiest bias and pushing against that bias in the training data. And if someone comes up to you and says, "well, that's not neutral, you're privileging Christians, what about anti-atheist bias", then I just feel like that person doesn't really understand how correcting for data-skew works. ---- > but when an organization like Mozilla openly states in their guiding principles that certain groups are a special priority It's OK to triage issues and attack them in a specific order. Again, if you have issue with which groups Mozilla chose, then fine -- but you seem to be arguing that when de-biasing a model, it's not OK to try and pick out the most impactful biases and triage them and focus on the most important issues first -- and that's just a really silly thing to argue. Of course any attempt to correct for data bias starts with identifying specific areas where you want to correct.
- danShumway 4y ago
  To be fair to your concerns, how do we deal with the problem of propaganda in AI? How do you correct for institutional bias in de-biasing results? How do you guard against censorship that can itself push minorities out of being able to use those models? Well, you correct for that by having diverse models trained by diverse people that aren't localized to a singular company or insulated within a single industry. Which is... also kind of a big thing that Mozilla is arguing about here. User agency over models and training, privacy and local models, having a diverse set of companies and organizations building models rather than a single monopoly that controls access and training data, transparency and inspectability of AI that allows us to examine why it's producing certain output -- those are the actual ways to protect against de-biasing efforts turning into propaganda, and Mozilla's points above are decent ways to get closer to that goal. Those are the important steps you should be focusing on rather than policing companies for acknowledging and trying to reduce harm against minorities.
- throwaway322112 4y ago
  > De-biasing models is necessary. Anyone who claims otherwise... I just don't think that's a defensible position to take if you've ever worked with large datasets. I don't know how to be more clear: nowhere in this thread have I argued for using raw models without any de-biasing. Twice now you have ascribed this position to me. Are you reading what I have actually written? > but it is extremely reasonable for AI researches to say "Internet data biases against certain groups and we would like to correct for that bias when training models." The idea that "Internet data biases against certain groups" seems itself a biased statement. I am pretty sure that Internet data is biased against all groups. I bet you can get a raw LLM to say offensive things about any group if you give it the right prompt.
  - danShumway 4y ago
    > I don't know how to be more clear: nowhere in this thread have I argued for using raw models without any de-biasing. Okay, great. But then we are arguing about an implementation, aren't we? We agree that de-biasing is necessary. It's just that you seem to think that calling out specific biases is the wrong way to approach that. But I'm curious how you're supposed to de-bias data without drawing any attention to or acknowledging the groups that the data is biased against? > The idea that "Internet data biases against certain groups" seems itself a biased statement. If a data set is biased, it's biased in a direction. By definition, that's what bias is. > I am pretty sure that Internet data is biased against all groups. I bet you can get a raw LLM to say offensive things about any group if you give it the right prompt. That's not what bias is. Bias is not "can you get it to say something offensive" -- bias is a systematic predisposition in a direction. Bias is not stuff like, "it made a dirty joke", bias is stuff like, "this hiring model is more likely to hire you if your name sounds white." And the goal here (believe it or not when I say this) is not to make models that never make offensive jokes. The goal is to avoid making models that reproduce and reinforce systemic issues in society. It's entirely appropriate when looking at the problem of "what social inequities are being reproduced by a model" to identify specific social inequities and inequalities that exist in society. ---- Part of the reason why I've been assuming that you're against de-biasing in general is that you keep saying stuff like "the idea that the Internet biases against certain groups is itself a biased statement." That is nonsensical, it seems like you're suggesting that companies should be correcting against bias without determining what the direction of that bias is or who it affects. How exactly should they do that? What does it mean to correct a systemic skew without identifying what the skew is? Again, look at the Reddit example. Yes, you can find offensive stuff on Reddit for everyone: atheists, vegans, Conservatives, Liberals, whoever. But de-biasing is not about whether or not something is offensive, it's about identifying and correcting a predisposition or tendency towards certain attitudes and thoughts. And that necessarily requires calling out what those predispositions are -- so you're going to end up with statements like "Reddit is biased against Conservative/Libertarian philosophies overall, on average." And if somebody jumps out and says, "Reddit is biased against everyone, it's itself biased for you to call out that specific bias" -- then there's no response to that other than that's just not how any of this works. If you go into a conversation about data bias saying that it's inappropriate for us to identify specific biases, then you are effectively arguing against reducing bias, regardless of what else you say. And even more to the point, even if that wasn't true -- fixing any systemic problem starts with triaging and identifying specific areas you want to target first. It is so weird to hear someone say, "I've never claimed we shouldn't fix the problem, but pointing out specific groups that the problem applies to is wrong." No, it's identifying problem areas that are high impact: a normal, good thing for researchers to do. To the extent that it's "biased", it's the kind of bias that we want in pretty much every field and on every subject -- we want researchers and debuggers to bias themselves towards focusing on high-impact areas first when they're fixing problems. And we want them to call out specific areas that they think deserve attention. That's what triaging is.

danShumway 4y ago

> We are discussing a set of AI principles, not an implementation of those principles. If the position is that any implementation of those principles is inherently problematic, then that is nonsense. Quite frankly, I do think we're talking about implementation. De-biasing models is necessary. Anyone who claims otherwise... I just don't think that's a defensible position to take if you've ever worked with large datasets. So if you have a criticism of Mozilla's implementation of its de-biasing, then argue against that implementation. But arguing that there shouldn't be an implementation is ridiculous. ---- > So how do we know that the model authors aren't just introducing their own bias, either by over-correcting one set of biases from the input, or under-correcting others? You heckin don't know. But the alternative to that is not neutral models, it's still biased models -- because the data itself that you train an LLM on is inherently biased. Any large textual database you get off the Internet is going to have bias in it. Choosing to ignore that fact does not give you neutrality. You're very worried about the risk that de-biasing can become re-biasing, and on one hand, you're right, that's a very real risk. Censorship and alignment risks are very real concerns in AI. But you don't seem to be worried about the fact that models that are not "de-biased" are subject to the exact same concerns. Choosing not to de-bias is just saying that you'll have a biased model with all of the same concerns you raise above, except with no effort to correct that bias at all. You're scared of Mozilla pushing people to bias their models in a certain direction, you don't seem to have internalized that the models are already biased in a certain direction. And all of the risks of model bias apply to models that have been uncritically fed raw data. Again, disagree with specific implementations if you want, but it is extremely reasonable for AI researches to say "Internet data biases against certain groups and we would like to correct for that bias when training models." And there's no "neutral" way to do that -- the way you correct for bias is you identify the areas where bias exists and you push back on those trends in the data. Again, I'd bring up Reddit here. If you're trying to build an LLM on Reddit data and you want it to be "fair" to Conservatives/Christians, that means identifying that a platform like Reddit has a very clear Liberal/Athiest bias and pushing against that bias in the training data. And if someone comes up to you and says, "well, that's not neutral, you're privileging Christians, what about anti-atheist bias", then I just feel like that person doesn't really understand how correcting for data-skew works. ---- > but when an organization like Mozilla openly states in their guiding principles that certain groups are a special priority It's OK to triage issues and attack them in a specific order. Again, if you have issue with which groups Mozilla chose, then fine -- but you seem to be arguing that when de-biasing a model, it's not OK to try and pick out the most impactful biases and triage them and focus on the most important issues first -- and that's just a really silly thing to argue. Of course any attempt to correct for data bias starts with identifying specific areas where you want to correct.

danShumway 4y ago
To be fair to your concerns, how do we deal with the problem of propaganda in AI? How do you correct for institutional bias in de-biasing results? How do you guard against censorship that can itself push minorities out of being able to use those models? Well, you correct for that by having diverse models trained by diverse people that aren't localized to a singular company or insulated within a single industry. Which is... also kind of a big thing that Mozilla is arguing about here. User agency over models and training, privacy and local models, having a diverse set of companies and organizations building models rather than a single monopoly that controls access and training data, transparency and inspectability of AI that allows us to examine why it's producing certain output -- those are the actual ways to protect against de-biasing efforts turning into propaganda, and Mozilla's points above are decent ways to get closer to that goal. Those are the important steps you should be focusing on rather than policing companies for acknowledging and trying to reduce harm against minorities.
throwaway322112 4y ago
> De-biasing models is necessary. Anyone who claims otherwise... I just don't think that's a defensible position to take if you've ever worked with large datasets. I don't know how to be more clear: nowhere in this thread have I argued for using raw models without any de-biasing. Twice now you have ascribed this position to me. Are you reading what I have actually written? > but it is extremely reasonable for AI researches to say "Internet data biases against certain groups and we would like to correct for that bias when training models." The idea that "Internet data biases against certain groups" seems itself a biased statement. I am pretty sure that Internet data is biased against all groups. I bet you can get a raw LLM to say offensive things about any group if you give it the right prompt.
- danShumway 4y ago
  > I don't know how to be more clear: nowhere in this thread have I argued for using raw models without any de-biasing. Okay, great. But then we are arguing about an implementation, aren't we? We agree that de-biasing is necessary. It's just that you seem to think that calling out specific biases is the wrong way to approach that. But I'm curious how you're supposed to de-bias data without drawing any attention to or acknowledging the groups that the data is biased against? > The idea that "Internet data biases against certain groups" seems itself a biased statement. If a data set is biased, it's biased in a direction. By definition, that's what bias is. > I am pretty sure that Internet data is biased against all groups. I bet you can get a raw LLM to say offensive things about any group if you give it the right prompt. That's not what bias is. Bias is not "can you get it to say something offensive" -- bias is a systematic predisposition in a direction. Bias is not stuff like, "it made a dirty joke", bias is stuff like, "this hiring model is more likely to hire you if your name sounds white." And the goal here (believe it or not when I say this) is not to make models that never make offensive jokes. The goal is to avoid making models that reproduce and reinforce systemic issues in society. It's entirely appropriate when looking at the problem of "what social inequities are being reproduced by a model" to identify specific social inequities and inequalities that exist in society. ---- Part of the reason why I've been assuming that you're against de-biasing in general is that you keep saying stuff like "the idea that the Internet biases against certain groups is itself a biased statement." That is nonsensical, it seems like you're suggesting that companies should be correcting against bias without determining what the direction of that bias is or who it affects. How exactly should they do that? What does it mean to correct a systemic skew without identifying what the skew is? Again, look at the Reddit example. Yes, you can find offensive stuff on Reddit for everyone: atheists, vegans, Conservatives, Liberals, whoever. But de-biasing is not about whether or not something is offensive, it's about identifying and correcting a predisposition or tendency towards certain attitudes and thoughts. And that necessarily requires calling out what those predispositions are -- so you're going to end up with statements like "Reddit is biased against Conservative/Libertarian philosophies overall, on average." And if somebody jumps out and says, "Reddit is biased against everyone, it's itself biased for you to call out that specific bias" -- then there's no response to that other than that's just not how any of this works. If you go into a conversation about data bias saying that it's inappropriate for us to identify specific biases, then you are effectively arguing against reducing bias, regardless of what else you say. And even more to the point, even if that wasn't true -- fixing any systemic problem starts with triaging and identifying specific areas you want to target first. It is so weird to hear someone say, "I've never claimed we shouldn't fix the problem, but pointing out specific groups that the problem applies to is wrong." No, it's identifying problem areas that are high impact: a normal, good thing for researchers to do. To the extent that it's "biased", it's the kind of bias that we want in pretty much every field and on every subject -- we want researchers and debuggers to bias themselves towards focusing on high-impact areas first when they're fixing problems. And we want them to call out specific areas that they think deserve attention. That's what triaging is.