As Yann Lecun said, “in high dimension, there is no such thing as interpolation. In high dimension, everything is extrapolation.” It’s trivial to come up with

As Yann Lecun said, “in high dimension, there is no such thing as interpolation. In high dimension, everything is extrapolation.” It’s trivial to come up with a prompt that doesn’t exist in the dataset. To generalize, the model cannot memorize.