4 ms·
I don't think it's obvious. First of all there's the "magic" and often surprisingly good results that leads many to think there's something else there. Then it'
by sottol 3y ago
I don't think it's obvious. First of all there's the "magic" and often surprisingly good results that leads many to think there's something else there.
Then it's also not entirely clear that nothing else is learnt beyond next word, there was talk of runtime reconfigurable neural nets being embedded within the transformer weights where the models can learn on the fly and such.
And I think some believed that there was some higher-level encoding of human reasoning happening necessary to predict the next word well in some contexts beyond just memorization.
I think research like this is necessary to put some "obvious beliefs" onto solid ground.