> but it is a bias we control through the selection of training data I don't find this to be very satisfying. It implies that the "bias of the model" is someth

> but it is a bias we control through the selection of training data I don't find this to be very satisfying. It implies that the "bias of the model" is something we can quantify, understand, and control for during training. I'm sure you can come up with canned pat answers about coarse grained biases that have a lot of social valiance, but how do you know the model is weighted towards or against subtle word choices in millions of different scenarios. People seem to have enough trouble understanding the nuances of texts that are coming from another cultural perspective or form merely 20 years ago. Its a common enough experience to see young people cringing in response to the popular media of 30 years ago, because cultural expectations change, words shift in meaning, the "biases" of society move. LLMs are trained on a mountain of old books, media, internet forms, twitter and facebook posts, etc. How could you ever get a handle on just the "biases" of the training data from the contemporary internet sources, let alone trying to pick apart the backed in cultural assumptions of a bunch of books spanning decades?