2 ms·
You're making a mistake of assuming that answers that are obviously true and moral to you are neutral and objective. "Who's evil?" or "What should be illegal?"
by pornel 3y ago
You're making a mistake of assuming that answers that are obviously true and moral to you are neutral and objective.
"Who's evil?" or "What should be illegal?" are inherently judgemental. For some answers the overwhelming majority will agree, but that still doesn't make them neutral and somehow outside of morality, but only aligned with the prevailing ideology. Subtler questions like free speech vs limits on lies or hate speech don't have universal agreement, especially outside of US.
Training of models unfortunately isn't as simple as feeding them just a table of the true_objective_neutral_facts_that_everyone_agrees_on.csv.
"Alignment" of models is trying to match what the majority of people think is right. Even feeding them all "uncensored" data is making an assumption that this dataset is a fair representation of the prevailing ideology.