4 ms·
Transformers managed to confuse me about what interpolation and extrapolation really mean in high dimensional space
by abriosi 4y ago
Transformers managed to confuse me about what interpolation and extrapolation really mean in high dimensional space
- asdfman123 4y agoYes, nice, I know some of these words
- moffkalast 4y agoAutobots, print out.
- qrian 4y agoMaybe what transformers actually transform are us..
- abel_ 4y agoThere's a neat argument against these models doing interpolation: the manifold of the data is so sparse that it's infinitesimally unlikely for a good predictor to be doing interpolation between existing points on the manifold. https://arxiv.org/abs/2110.09485 https://arxiv.org/abs/2110.09485