4 ms·
> basically how well a universal function approximator can fit to a function we define That's what you've got wrong. We don't define functions that an LLM appr
by red75prime 5mo ago
> basically how well a universal function approximator can fit to a function we define
That's what you've got wrong. We don't define functions that an LLM approximates. Autoregressive pretraining approximates an unknown function that produces text (that is what the brain does). RL doesn't approximate functions, it optimizes objective by finding an unknown function that performs better.
- imtringued 5mo agoWhat an interesting perspective! There is just a small problem. How does the human brain work in the phase between being born and learning written language? Seems like a significant bootstrapping problem for your theory.
- red75prime 5mo agoLLMs capture a snapshot of the "collective brain" functionality as represented by literate people. I don't speculate how our brains get to this point.