4 ms·
Did anyone predict the unreasonable effectiveness of LLMs? We seemed to have stumbled upon something quite big as humans…
by mensetmanusman 4y ago
Did anyone predict the unreasonable effectiveness of LLMs?
We seemed to have stumbled upon something quite big as humans…
- tough 4y agoOr maybe language just isn't that complex at all? Same with pixels? Dunno
- GeorgeTirebiter 4y agoPerhaps Monica Anderson understood early? https://experimental-epistemology.ai/the-red-pill-of-machine-learning/ https://experimental-epistemology.ai/the-red-pill-of-machine...
- loveparade 4y agoI worked in one of the big labs when the first large models came out and I can pretty confidently say that nobody in the field predicted this. Sure, there were always people who said "let's make models bigger because why not, we have the infra and it'll be a good paper" but nobody expected them to become this good just by being bigger and using more data. The consensus was that they'd hit a ceiling of what they can do much sooner.
- tomr75 4y agoaren't humans trained on massive data sets, why not expect the same for digital intelligence?
- soraki_soladead 4y agoOnly some model architectures continue to get better as you pump in more data. Transformers and their variants have this property more so than prior architectures.
- mensetmanusman 4y agoLarge yes, but millions of orders of magnitude less.
- corbulo 4y agoWe stumbled upon a mirror of our own cognitive bias. Another chapter in the era of Narcissus and Echo.