> But to train a model to deny truths which are politically taboo does seem to be misaligned with training a model to favor truths, no? Strictly speaking, RLHF

> But to train a model to deny truths which are politically taboo does seem to be misaligned with training a model to favor truths, no? Strictly speaking, RLHF trains models to give answers which the human raters believe to be correct. In uncontroversial territory this correlates with truth, in taboo territory only with what is politically correct.