That's a problem of mode collapse in post-trained models, which AI labs don't seem to be interested in, or at least their attempts to fix it are not visible, an

That's a problem of mode collapse in post-trained models, which AI labs don't seem to be interested in, or at least their attempts to fix it are not visible, and there's relatively little research around it. Base models are diverse and it's not an issue in them. It's actually way deeper than just the lack of superficial linguistic diversity or a technically collapsed output distribution. A mode-collapsed model learns the 1:1 (or at least really stiff) mapping between input ideas and output ideas. One consequence of that is if you give it conceptually short prompts, you can expect your outputs to be identical to those that other people got! The fewer distinct ideas your prompt expresses, the less room for variance there is.