4 ms·
"displays surprising emergent properties" Section 3 states "includes managing intricate algorithmic tasks." I take this as meaning it can write new code (not r
by waynecochran 3y ago
"displays surprising emergent properties"
Section 3 states "includes managing intricate algorithmic tasks." I take this as meaning it can write new code (not regurgitation). It is impressive that is can write code as described in the prompt, but I didn't think this was considered "emergent" but "generalization." What is the difference?
- mjburgess 3y agoThere is no difference, "emergent" is another hype term used in place of "discontinuous generalisation". ie., with 1MB data you dont get generalisation, with 1TB you do. So they call that emergence. I'm almost at the point where I can let these things go; but each new bit of mystifying hype, another stone falls in my shoe.
- irthomasthomas 3y agoYeah, the way I read "emergent" is as "surprising" to our intuition, but technically expected. So I read this as surprisingly surprising performance.
- kirab 3y agoThe English explanations you find first for emergent are not really fitting here. Interestingly the Germany explanation for the same word seems to be much better. I'm pretty sure they mean emergent as in emergent gameplay. Here's a good explanation for that: > Emergent gameplay refers to complex situations in video games, board games, or table top role-playing games that emerge from the interaction of relatively simple game mechanics. So emergent is not inherently surprising. Games like Dwarf Fortress or Rimworld have a lot of emergent complexity but most of it was not surprising. So in the context of LLM it means that complex reasoning is emergent based on simple underlying mechanics. And since we thought humans’ complex reasoning skills were very unique.. We are surprised by the emergence of it in LLMs. PS: The explanation for "emergence" is much better: > In philosophy, systems theory, science, and art, emergence occurs when a complex entity has properties or behaviors that its parts do not have on their own, and emerge only when they interact in a wider whole.
- irthomasthomas 3y agoSorry, I meant the way I read peoples colloquial use of it toward these llms. They aren't actually displaying emergence, as this level of performance was predicted.
- mjburgess 3y agoYes, so this is the meaning of the word I'm rejecting. NN models do not acquire emergent properties. Human reasoning (, game play, etc.) is emergent in the sense that it is irreducible to its parts. "Generalisation in high-parameter training regiemes" is fully reducible. It is not emergence in any relevant sense of the term.
- EnergyAmy 3y agoI think you're just inventing your own definition of the word "emergent", then getting on your high horse when people, quite obviously, don't use the same definition as the one you just made up.
- waynecochran 3y agoWhat is the "mirage" then that folks are claiming in LLM's as described in the Stanford research? https://hai.stanford.edu/news/ais-ostensible-emergent-abilities-are-mirage https://hai.stanford.edu/news/ais-ostensible-emergent-abilit...
- mjburgess 3y agoStanford is the centre of the AI hype. They are a profoundly mendacious VC-seeking institution, in this matter. None involved know what emergence means; they're engineers chasing and creating hype.
- waynecochran 3y agoIn this case they are squashing the hype by saying there is no real emergent behavior.