3 ms·
Just shooting from the hip, LLMs operate out on a frontier where the curse of dimensionality removes a large chunk of the practical value from the concept of a
by bunderbunder 1y ago
Just shooting from the hip, LLMs operate out on a frontier where the curse of dimensionality removes a large chunk of the practical value from the concept of a convex hull. Especially in a case like this where the vector embedding process places hard limits on the range of possible magnitudes and directions for any single vector.
- drdeca 1y agoOutside of the context of a convex hull, I don’t know how to make a distinction between interpolation and extrapolation. This is the core of my question. What precisely is it that you mean when you say that it is interpolating rather than extrapolating? In the only definition that I know, the one based on convex hulls, I believe it would be extrapolating rather than interpolating. But people often say it is interpolating rather than extrapolating, and I don’t know what they mean.
- bunderbunder 1y agoI doubt they're really thinking about it in a mathematical sense when they say that. I'm guessing, for example, that "extrapolate" is meant in the more colloquial sense, which is maybe closer to "deduce" in practice.