Fwiw - I wrote a paper documenting LLM (and neural) regression a few years ago (Neurips 2018) https://arxiv.org/abs/1808.00508 https://arxiv.org/abs/1808.00508

Fwiw - I wrote a paper documenting LLM (and neural) regression a few years ago (Neurips 2018) https://arxiv.org/abs/1808.00508 https://arxiv.org/abs/1808.00508 They can interpolate but extrapolation is hard because of the constraints from nonlinearities