3 ms·
You have to define what you mean by "interpolate". The mechanisms that LLM use are not mysterious, and they are not the same as used by humans.
by slashdave 5mo ago
You have to define what you mean by "interpolate". The mechanisms that LLM use are not mysterious, and they are not the same as used by humans.
- drdeca 5mo agoIf you interpret “interpolate” in the literal sense, and apply it to the mechanisms behind LLMs, then the claim that they only interpolate, is straightforwardly false. Taking it instead as a metaphorical claim may be more valid, but in that case it doesn’t depend on our understanding of how LLMs work.
- slashdave 5mo agoLLMs are statistical models by construction, so depending on how liberal you want to be with terminology, "interpolate" is not so bad. Might make a statistician upset.
- baq 5mo agoThey’re also capable of performing arbitrary computation when ran in a loop - so they can be made to quite literally interpolate whatever. Philosophers are quite upset, too.
- slashdave 5mo agoDefine "arbitrary". Without RAG, they screw up basic algebra.
- drdeca 5mo agoBut people aren’t giving a (less literal) definition of what they mean by “interpolate” that relies on the internal mechanisms of these models, just a vague metaphor, which, as this vague metaphor, there’s nothing it uses about LLMs that makes the question “do LLMs just interpolate” less of a type error than “do people just interpolate”. And I don’t think it’s a good metaphor.