4 ms·

> We're all fine with Mozilla having opinions about "good" and "bad" states of the world until it has opinions about treating minorities equitably, then it beco

> We're all fine with Mozilla having opinions about "good" and "bad" states of the world until it has opinions about treating minorities equitably, then it becomes pressingly important that we have a philosophy discussion. It's because that was the only thing on the list that is openly discriminatory. If the intent was truly to avoid unfair bias against people, the mention of marginalized communities would be unnecessary. By definition, avoiding bias should be a goal that does not require considering some people or groups differently than others. The fact one set of groups is called out as being the primary consideration for protection makes it clear that the overriding value here is not to avoid bias universally, but rather to consider bias against "marginalized communities" to be worse than bias against other people. Since the launch of ChatGPT, plenty of conservatives have made bias complaints about it. The framework outlined by Mozilla gives the strong impression that they would consider such complaints to be not as important, or maybe not even a problem at all.

danShumway 4y ago
> By definition, avoiding bias should be a goal that does not require considering some people or groups differently than others. This is something that sounds nice on paper, but if you've worked with polling data or AI in general, you should have figured out by now that weighting data and compensating for biases in training sets is an essential part of any training/measuring process. LLMs are not trained on "neutral" data. We can have long, difficult conversations about how to fix that and who to focus on, but the idea that you can just throw data sources at an AI and expect the outcome to be fair is kind of silly. Case in point, you bring up bias against Conservatives -- well if you train an LLM on Reddit, it will have a Liberal bias, period. And I suspect if a company tried to correct that bias or even just identify it, you wouldn't call that unfair. In fact, there would be no way to train an LLM on Reddit and to get rid of a Liberal bias in its answers without systematically trying to correct for that bias during the training process. Neutrality where you just throw the data in and trust the results and deliberately refuse to weight the outputs only works when you're pulling data from neutral sources. And in the real world, that's practically never the case. It's certainly never the case with an LLM.
- throwaway322112 4y ago
  > the idea that you can just throw data sources at an AI and expect the outcome to be fair is kind of silly. You are arguing against a straw man. We are discussing a set of AI principles, not an implementation of those principles. I claim that any anti-bias principle should be applied equally to all, both in spirit and in practice. That is not the same as assuming the input is neutral, or assuming that de-biasing is unnecessary. By your own account, the researchers who do the de-biasing have a great amount of discretion about "how to fix that and who to focus on." I don't think it's unfair to say that it is their job to put their thumbs on the scale, according to the beliefs and priorities of themselves and their organizations. So how do we know that the model authors aren't just introducing their own bias, either by over-correcting one set of biases from the input, or under-correcting others? It seems likely that "de-biasing" can become "re-biasing." I would already suspect that this is a major risk with AI, but when an organization like Mozilla openly states in their guiding principles that certain groups are a special priority, re-biasing seems all but certain. Of course everyone will bring their own biases to the table when performing a job, that is inevitable. But an AI provider that wants to be trustworthy to a wide group of people should be extremely vigilant about correcting for their own personal biases, and be clear in their messaging that this is a core commitment. OpenAI, to their credit, seem to be taking this seriously: https://openai.com/blog/how-should-ai-systems-behave#addressing-biases https://openai.com/blog/how-should-ai-systems-behave#address...
  - danShumway 4y ago
    > We are discussing a set of AI principles, not an implementation of those principles. If the position is that any implementation of those principles is inherently problematic, then that is nonsense. Quite frankly, I do think we're talking about implementation. De-biasing models is necessary. Anyone who claims otherwise... I just don't think that's a defensible position to take if you've ever worked with large datasets. So if you have a criticism of Mozilla's implementation of its de-biasing, then argue against that implementation. But arguing that there shouldn't be an implementation is ridiculous. ---- > So how do we know that the model authors aren't just introducing their own bias, either by over-correcting one set of biases from the input, or under-correcting others? You heckin don't know. But the alternative to that is not neutral models, it's still biased models -- because the data itself that you train an LLM on is inherently biased. Any large textual database you get off the Internet is going to have bias in it. Choosing to ignore that fact does not give you neutrality. You're very worried about the risk that de-biasing can become re-biasing, and on one hand, you're right, that's a very real risk. Censorship and alignment risks are very real concerns in AI. But you don't seem to be worried about the fact that models that are not "de-biased" are subject to the exact same concerns. Choosing not to de-bias is just saying that you'll have a biased model with all of the same concerns you raise above, except with no effort to correct that bias at all. You're scared of Mozilla pushing people to bias their models in a certain direction, you don't seem to have internalized that the models are already biased in a certain direction. And all of the risks of model bias apply to models that have been uncritically fed raw data. Again, disagree with specific implementations if you want, but it is extremely reasonable for AI researches to say "Internet data biases against certain groups and we would like to correct for that bias when training models." And there's no "neutral" way to do that -- the way you correct for bias is you identify the areas where bias exists and you push back on those trends in the data. Again, I'd bring up Reddit here. If you're trying to build an LLM on Reddit data and you want it to be "fair" to Conservatives/Christians, that means identifying that a platform like Reddit has a very clear Liberal/Athiest bias and pushing against that bias in the training data. And if someone comes up to you and says, "well, that's not neutral, you're privileging Christians, what about anti-atheist bias", then I just feel like that person doesn't really understand how correcting for data-skew works. ---- > but when an organization like Mozilla openly states in their guiding principles that certain groups are a special priority It's OK to triage issues and attack them in a specific order. Again, if you have issue with which groups Mozilla chose, then fine -- but you seem to be arguing that when de-biasing a model, it's not OK to try and pick out the most impactful biases and triage them and focus on the most important issues first -- and that's just a really silly thing to argue. Of course any attempt to correct for data bias starts with identifying specific areas where you want to correct.

danShumway 4y ago

> By definition, avoiding bias should be a goal that does not require considering some people or groups differently than others. This is something that sounds nice on paper, but if you've worked with polling data or AI in general, you should have figured out by now that weighting data and compensating for biases in training sets is an essential part of any training/measuring process. LLMs are not trained on "neutral" data. We can have long, difficult conversations about how to fix that and who to focus on, but the idea that you can just throw data sources at an AI and expect the outcome to be fair is kind of silly. Case in point, you bring up bias against Conservatives -- well if you train an LLM on Reddit, it will have a Liberal bias, period. And I suspect if a company tried to correct that bias or even just identify it, you wouldn't call that unfair. In fact, there would be no way to train an LLM on Reddit and to get rid of a Liberal bias in its answers without systematically trying to correct for that bias during the training process. Neutrality where you just throw the data in and trust the results and deliberately refuse to weight the outputs only works when you're pulling data from neutral sources. And in the real world, that's practically never the case. It's certainly never the case with an LLM.

throwaway322112 4y ago
> the idea that you can just throw data sources at an AI and expect the outcome to be fair is kind of silly. You are arguing against a straw man. We are discussing a set of AI principles, not an implementation of those principles. I claim that any anti-bias principle should be applied equally to all, both in spirit and in practice. That is not the same as assuming the input is neutral, or assuming that de-biasing is unnecessary. By your own account, the researchers who do the de-biasing have a great amount of discretion about "how to fix that and who to focus on." I don't think it's unfair to say that it is their job to put their thumbs on the scale, according to the beliefs and priorities of themselves and their organizations. So how do we know that the model authors aren't just introducing their own bias, either by over-correcting one set of biases from the input, or under-correcting others? It seems likely that "de-biasing" can become "re-biasing." I would already suspect that this is a major risk with AI, but when an organization like Mozilla openly states in their guiding principles that certain groups are a special priority, re-biasing seems all but certain. Of course everyone will bring their own biases to the table when performing a job, that is inevitable. But an AI provider that wants to be trustworthy to a wide group of people should be extremely vigilant about correcting for their own personal biases, and be clear in their messaging that this is a core commitment. OpenAI, to their credit, seem to be taking this seriously: https://openai.com/blog/how-should-ai-systems-behave#addressing-biases https://openai.com/blog/how-should-ai-systems-behave#address...
- danShumway 4y ago
  > We are discussing a set of AI principles, not an implementation of those principles. If the position is that any implementation of those principles is inherently problematic, then that is nonsense. Quite frankly, I do think we're talking about implementation. De-biasing models is necessary. Anyone who claims otherwise... I just don't think that's a defensible position to take if you've ever worked with large datasets. So if you have a criticism of Mozilla's implementation of its de-biasing, then argue against that implementation. But arguing that there shouldn't be an implementation is ridiculous. ---- > So how do we know that the model authors aren't just introducing their own bias, either by over-correcting one set of biases from the input, or under-correcting others? You heckin don't know. But the alternative to that is not neutral models, it's still biased models -- because the data itself that you train an LLM on is inherently biased. Any large textual database you get off the Internet is going to have bias in it. Choosing to ignore that fact does not give you neutrality. You're very worried about the risk that de-biasing can become re-biasing, and on one hand, you're right, that's a very real risk. Censorship and alignment risks are very real concerns in AI. But you don't seem to be worried about the fact that models that are not "de-biased" are subject to the exact same concerns. Choosing not to de-bias is just saying that you'll have a biased model with all of the same concerns you raise above, except with no effort to correct that bias at all. You're scared of Mozilla pushing people to bias their models in a certain direction, you don't seem to have internalized that the models are already biased in a certain direction. And all of the risks of model bias apply to models that have been uncritically fed raw data. Again, disagree with specific implementations if you want, but it is extremely reasonable for AI researches to say "Internet data biases against certain groups and we would like to correct for that bias when training models." And there's no "neutral" way to do that -- the way you correct for bias is you identify the areas where bias exists and you push back on those trends in the data. Again, I'd bring up Reddit here. If you're trying to build an LLM on Reddit data and you want it to be "fair" to Conservatives/Christians, that means identifying that a platform like Reddit has a very clear Liberal/Athiest bias and pushing against that bias in the training data. And if someone comes up to you and says, "well, that's not neutral, you're privileging Christians, what about anti-atheist bias", then I just feel like that person doesn't really understand how correcting for data-skew works. ---- > but when an organization like Mozilla openly states in their guiding principles that certain groups are a special priority It's OK to triage issues and attack them in a specific order. Again, if you have issue with which groups Mozilla chose, then fine -- but you seem to be arguing that when de-biasing a model, it's not OK to try and pick out the most impactful biases and triage them and focus on the most important issues first -- and that's just a really silly thing to argue. Of course any attempt to correct for data bias starts with identifying specific areas where you want to correct.