5 ms·

How do you "debias" an LLM? There is no unbiased standard to test against.

• Invictus0 3y ago
  [flagged]
• geraltofrivia 3y ago
  That's not _completely_ true. There are a bunch of datasets (specific to some cultural/lingual contexts) like CrowsPairs[1], StereoSet[2]. There is a lot of work you can do to make sure that the model's predictions are fair as well [3]. But at the end, yes these datasets don't exist at the scale of training sets of these LLMs. Hence red-teaming and RLHF post convergence. PS: Yes I know CrowsPairs is a dataset with a bunch of flaws. My SO is working, in a team of 10+ linguists and researchers to develop a multi-lingual, generalized version of it which also addresses multiple problems with it. Unpublished work, for now. [1] https://github.com/nyu-mll/crows-pairs/ https://github.com/nyu-mll/crows-pairs/ [2] https://arxiv.org/abs/2004.09456 https://arxiv.org/abs/2004.09456 [3] https://arxiv.org/pdf/2204.09591.pdf https://arxiv.org/pdf/2204.09591.pdf
• blueboo 3y ago
  There are all sorts. You do have to be specific about what you mean by “bias” though. https://arxiv.org/abs/2110.08193 https://arxiv.org/abs/2110.08193 and https://arxiv.org/abs/1804.09301 https://arxiv.org/abs/1804.09301 and https://arxiv.org/abs/2206.04615 https://arxiv.org/abs/2206.04615 …
• dragonwriter 3y ago
  You can debias it relative to any standard you choose, and maybe should if there is one particularly applicable to the intended use case, but the more important thing for models intended for broad, multi-domain use is to document and disclose biases, and ideally any particularly effective techniques for nudging them for particular applications short of fine-tuning/retraining.