> don’t see this as a risk for leaking that data in the later output of the model. Outputs often repeat the inputs (in fact, this is one of the behaviors that

> don’t see this as a risk for leaking that data in the later output of the model. Outputs often repeat the inputs (in fact, this is one of the behaviors that reinforcement appears to intentionally increase). So: You provide a secret. Raters prefer outputs with repeat your secret. Model becomes more likely to say the secret thing, even when its not in the prompt.