4 ms·
except that it's really not. It's about finding a function to fit the data. And "generalizing" is only when the question is jsut barely outside their training d
by vrighter 2mo ago
except that it's really not. It's about finding a function to fit the data. And "generalizing" is only when the question is jsut barely outside their training data. Like a truncated taylor series, is still somewhat accurate just outside the region it was designed for, But I couldn't say a truncated taylor series generalizes over the function it's approximating.
- versteegen 2mo agoI urge you to reconsider your beliefs. You are missing something important because you are thinking in terms of low-dimensional statistics. Deep learning doesn't just fit data, it finds features (abstractions) of the data.
- vrighter 2mo agothis is not a belief. it is a fact. Doing the same thing in a higher dimensional space is still doing the same thing.
- SgtBastard 2mo agoI asked the original commenter to confirm my understanding of what they were saying. I find it a fascinating alternative view to what is largely well understood (your counter-point). The thing that stood out is the comment that it’s an approximation of the original data generator (humanity). Early approximations were poor (GPT 2-3, to an extent GPT-4). I’m not so sure I can reject the hypothesis that such an approximation can be found.