Most of what you think is extrapolation is interpolation. Models do NOT extrapolate unless they are given access to resources/data that is not somehow in their

Most of what you think is extrapolation is interpolation. Models do NOT extrapolate unless they are given access to resources/data that is not somehow in their training dataset. We have done so much encoding of knowledge that a majority of serious use-cases for LLMs are Interpolation, but the fact that we even need retrieval augmented generation shows that they do not extrapolate.