4 ms·
Does "Chinese nonfiction" include Mao's bibliography and "Xi Jinping thought"? Do you realize how all-consuming China's censorship apparatus is, and that this
by xeckr 3y ago
Does "Chinese nonfiction" include Mao's bibliography and "Xi Jinping thought"?
Do you realize how all-consuming China's censorship apparatus is, and that this censorship introduces all sorts of biases in their literary corpus which you now want to be used as training data for LLMs?
- staticman2 3y agoThat's why you fine tune models, right? To make sure the undesirable things in their data set don't lead to undesirable output.
- xeckr 3y agoFine-tuning will never get rid of all biases introduced by the training data, some of which are subtle enough that you won't even know that you should apply fine-tuning techniques to try to mitigate them. For example, did you know that there is a trend for physics textbooks to be dedicated to Xi Jinping? If you didn't know this, because who has time to sift through millions of books to find undesirable biases, isn't it clear that your LLM will develop subtle but favourable biases towards the CCP?