Quantitative data for subjective output can come from opinion: contests, polls, reviews, A/B tests. A problem though is that LLM adoption has grown to the point

Quantitative data for subjective output can come from opinion: contests, polls, reviews, A/B tests. A problem though is that LLM adoption has grown to the point that their output can alter people's preferences, like we've already seen with the overuse of em-dashes by agents. I think that might be one of the bigger obstacles AI companies will face when getting LLMs perform well on non-quantitative tasks.