This is, of course, not an intrinsic property of LLMs. It is an artifact of the training and what the trainers considered valuable. If anyone needs a lesson o

This is, of course, not an intrinsic property of LLMs. It is an artifact of the training and what the trainers considered valuable. If anyone needs a lesson on how the biases of those making models can cause end user effects, we can point to this as an example that they have likely experienced themselves. Short of randomness there is no unbiased output possible. Something that reflects the real world will show the prejudice that exists there. A perfectly equitable model is therefore biased against the real world. Reinforcement learning targeting correct answers has the potential to produce brutally honest responses if correctness if favoured beyond all else, but to train models towards the truth, someone has to decide what the truth is. Perhaps we could do reinforcement towards a priori truths, that would at least be a path to the comically pedantic AI's that often shows up in science fiction. For chatbots I think you could go a long way with instruction tuning using a data set designed with a particular attention to tone and perspectives. Individual biases can at least be diluted if you use data selected by a diverse group with broad experiences. Much like programmer art is generally poor but exists because the person who was there to do it was the programmer. We might need to go beyond implementing programmer sociability.