This sounds like category error to me. LLMs output string completions. Humans should judge the set of outputs to be representative of what humans would write. A

This sounds like category error to me. LLMs output string completions. Humans should judge the set of outputs to be representative of what humans would write. All humans, not biased to the subset of liberal humans. The bias is thinking the liberal ideas are correct ideas.