4 ms·

Gender bias in LLMs when asked about historical figures

1 points by yenniejun111 3y ago

GaggiX 3y ago
>Gender bias: Both models disproportionately predict male historical figures. It doesn't really surprise anyone that for much of history, societies were patriarchal, meaning that women had limited access to education, and traditional gender roles assigned women to domestic roles while men were the primary actors in public life. So, it shouldn't surprise anyone that the 'top 10 most important historical figures' are (mostly) men. GPT-4 and Claude cannot change history, and I think it's much more important to think about the present.
pk-protect-ai 3y ago
All the biases in LLMs are not in the NN but in the training data sets. So if you want the LLMs to be non biased, you need to curate the datasets which you are feeding into them. LLMs are biased in regard of religions for example too. They are biased in a way the majority of the internet audience is biased. This things have to be fought in the training data sets, not with the tuning afterwards ...
- brucethemoose2 3y ago
  I can't speak for LLMs, but Stable Diffusion goes above and beyond bias you would expect in a dataset. Corporate executive photos are an often cited example, but a good one. It almost exclusively generates stereotypical white guys, where the dataset can't be that homogenous. "Dark skin" pretty much exclusively generates African Americans, and its hard to steer away. Its also almost impossible to generate a dark skinned person with light eyes. What I am getting at is that reasonable dataset correlations can turn into hard assumptions in diffusion, and maybe other transformers models.
  - GaggiX 3y ago
    That's probably all to blame on CFG.
  - pk-protect-ai 3y ago
    You can't blame this on the math or transformers though. This is the thing about statistic, - the datasets have a tendency inherited from the human perception and biases. You can actually spend some time and money to analyze the LION 5B dataset to find out how homogeneous it is. People create photos and tag them, however small the bias in initial data is, it will be exposed by the NN, because it is more probable.

1 points by yenniejun111 3y ago

>Gender bias: Both models disproportionately predict male historical figures. It doesn't really surprise anyone that for much of history, societies were patriarchal, meaning that women had limited access to education, and traditional gender roles assigned women to domestic roles while men were the primary actors in public life. So, it shouldn't surprise anyone that the 'top 10 most important historical figures' are (mostly) men. GPT-4 and Claude cannot change history, and I think it's much more important to think about the present.

pk-protect-ai 3y ago

All the biases in LLMs are not in the NN but in the training data sets. So if you want the LLMs to be non biased, you need to curate the datasets which you are feeding into them. LLMs are biased in regard of religions for example too. They are biased in a way the majority of the internet audience is biased. This things have to be fought in the training data sets, not with the tuning afterwards ...

brucethemoose2 3y ago
I can't speak for LLMs, but Stable Diffusion goes above and beyond bias you would expect in a dataset. Corporate executive photos are an often cited example, but a good one. It almost exclusively generates stereotypical white guys, where the dataset can't be that homogenous. "Dark skin" pretty much exclusively generates African Americans, and its hard to steer away. Its also almost impossible to generate a dark skinned person with light eyes. What I am getting at is that reasonable dataset correlations can turn into hard assumptions in diffusion, and maybe other transformers models.
- GaggiX 3y ago
  That's probably all to blame on CFG.
- pk-protect-ai 3y ago
  You can't blame this on the math or transformers though. This is the thing about statistic, - the datasets have a tendency inherited from the human perception and biases. You can actually spend some time and money to analyze the LION 5B dataset to find out how homogeneous it is. People create photos and tag them, however small the bias in initial data is, it will be exposed by the NN, because it is more probable.