The training corpus is only learned very approximately and poorly during pretraining. You can use inference-time compute to try and cope, but this can at best

The training corpus is only learned very approximately and poorly during pretraining. You can use inference-time compute to try and cope, but this can at best make you somewhat more self-consistent; it cannot recreate info that you didn't learn effectively to begin with!