On the politicized stuff they use InstructGPT[1] to make sure it gives the "right" answer. This is human in the loop reinforcement learning. The human trainin

On the politicized stuff they use InstructGPT[1] to make sure it gives the "right" answer. This is human in the loop reinforcement learning. The human training it was probably instructed to give it a pass on any answer that was not "toxic." [1]https://openai.com/blog/instruction-following/ https://openai.com/blog/instruction-following/