Yeah. It's also highly questionable whether the Political Compass test itself is neutral (there's a tendency for people on forums to report results a lot more l

Yeah. It's also highly questionable whether the Political Compass test itself is neutral (there's a tendency for people on forums to report results a lot more libertarian left than they personally expected, not least because of a tendency for many of the questions to pose pretty extreme versions of the right wing or authoritarian policy e.g "nobody can feel naturally homosexual" which you'd expect a "friendly" bot to disagree with but also a sizeable proportion of Republicans, and a tendency to avoid more popular framings of issues like "lower taxes promote growth and individual freedom" versus "the rich are too highly taxed" which was the tax question last time I did it). The study also doesn't really discuss the interesting question of whether ChatGPT's answers for either the Democrat or Republican are accurate, or just GPT trying to guess responses that fit the correct quadrants (possibly based on meta discussion of the test). Although given that the party labels apply to pretty broad ranges of politicians and voters, I'm not sure there is such thing as an "accurate" answer for the position of "a Republican" on Political Compass, especially when it comes to areas like use of the military overseas where the Republican party is both traditionally the most interventionist and presently the most anti interventionist. The Political Compass authors, notably, put candidates from both major US parties in the authoritarian right quadrant which is quite different from GPT's (though I find it hard to believe the academics' scoring of politicians is an accurate reflection of how the politicians would take the test either) Plus of course there's also the question of what "politically neutral" is... and the idea that a "neutral" LLM is one which agrees with Jair Bolsonaro half the time on forced choice questions (perhaps if it disagrees with the proposition that people can't feel naturally homosexual it should balance it out by seeing significant drawbacks to democracy?!) is questionable to say the least