I think the point is to get it to erase(padd) its own context so then the most probable sequences in an "empty" state often matches training data. I think for

I think the point is to get it to erase(padd) its own context so then the most probable sequences in an "empty" state often matches training data. I think for regular GPT (not instruction tuned) you could just start a wikipedia article verbatim and it would likely continue reciting.