3 ms·
It's going to have similar stuff in it's training data, it's probably just being triggered to spit some of it out.
by version_five 3y ago
It's going to have similar stuff in it's training data, it's probably just being triggered to spit some of it out.
- LouisSayers 3y agoPlaying around with it a bit more, it does look like training data
- version_five 3y agoIt's definitely an interesting find if you can repeatably get it to spit out apparent training examples.
- brucethemoose2 3y agoYeah, exactly. Chat models ingest the entire conversation every time they are queried, so the context (and the training data) is the whole Q+A. Hence sometimes the llm thinks a particular answer is "done" without emitting the proper end token, so they keep trying to complete the context... Which is another question a user would ask from its training dataset. You can force this behavior in llama chat finetunes by forcing the model to generate infinitely. It will keep on generating questions + answers.