4 ms·
> when I try the intro thesis paragraph on GPT-5-Pro, it dislikes it I don't know about GPT-5-Pro, but LLMs can dislike their own output (when they work well..
by mdp2021 1y ago
> when I try the intro thesis paragraph on GPT-5-Pro, it dislikes it
I don't know about GPT-5-Pro, but LLMs can dislike their own output (when they work well...).
- gwern 1y agoThey can, but they are known to have a self-favoring bias, and in this case, the error is so easily identified that it raises the question of why GPT-5 would both come up with it & preserve it when it can so easily identify it; while if that was part of OP's original inputs (whatever those were) it is much less surprising (because it is a common human error and mindlessly parroted in a lot of the 'scaling has hit a wall' human journalism).
- Foreignborn 1y agodo you have a source? when i’ve done toy demos where GPT5, sonnet 4 and gemini 2.5 pro critique/vote on various docs (eg PRDs) they did not choose their own material more often than not. my setup wasn’t intended to benchmark though so could be wrong over enough iterations.
- gwern 1y agoI don't have any particularly canonical reference I'd cite here, but self-preference bias in LLMs is well-established. (Just search Arxiv.)