6 ms·
When we talk about "emergence" in machine learning, we talk about those metrics with a sudden jump, as explained in the paper that introduced the term: https://
by n2d4 3y ago
When we talk about "emergence" in machine learning, we talk about those metrics with a sudden jump, as explained in the paper that introduced the term: https://arxiv.org/abs/2206.07682 https://arxiv.org/abs/2206.07682
- exe34 3y agoJust because somebody overloaded a word doesn't mean we can't correct people who use it wrong. The correct phrase should be "sudden/unexpected emergence". Otherwise the phrase "gradual emergence" would be an oxymoron.
- n2d4 3y agoAs another commenter explained, different domains use words differently. The ML paper I linked has been influential enough to use the word "emergent" in machine learning to describe this behaviour, so you're right that, within the context of ML and AI, "gradual emergence" is an oxymoron. Again, I understand this might not align with what you think "emergence" means in the English language, in the same way in which "metal" in physics has a completely different meaning from "metal" in chemistry, "metal" in astrophysics, and "metal" in the English language.
- exe34 3y agoTo be fair, I don't think anybody in chemistry or astrophysics is very proud of this kind of overloading. It was done by some Philistines and the rest of us have to live with their defacement of the dictionary.
- IshKebab 3y agoThey didn't introduce the term. It's a standard English word: https://en.wiktionary.org/wiki/emergent#English https://en.wiktionary.org/wiki/emergent#English > (philosophy, sciences) Having properties as a whole that are more complex than the properties contributed by each of the components individually. If they've said that it requires a sudden jump then they're trying to redefine the word to mean something it doesn't. But I haven't heard anyone else use that definition. The emergent properties of AI are like "how can it write and entire Python program when we've only ever asked it to predict single words". It doesn't require any sudden jumps.
- bpshaver 3y agoIndeed, I was surprised by the article's assumption that "emergence" is about a sudden jump in ability. I work in AI and ML but my academic background is in the philosophy of mind. When people talk about "emergence" w/r/t AI I assume they're speculating about how consciousness might be an emergent property of a neural network, in the same way some people speculate that our minds are emergent properties of our brains. I think that speculation is foolish, but that is beside the point. It has nothing to do with a sudden jump in abilities, per se.
- nyrikki 3y agoPolysemous words, which sound the same and spell the same but have two or more than two different meanings are common. Each subfield defines rigourous terms and some may be overloaded. It is impossible to have any universal definition of words that are universal, exact definitions have to be domain specific.
- IshKebab 3y agoProfound, but the AI field has not redefined this term, even if one paper tried to.
- bpshaver 3y agoPolysemous words exist of course, but that doesn't mean they're always unproblematic. The best examples, like bank (where the money is) versus bank (sloped piece of land) are unlikely to be confused because there is no domain where they're often used in conjunction. In this context, its charitable to note that some polysemy is at work within the domain (because the application of terms is so new) and so its useful to have these kind of distinction-making conversations. Many confusions arise when people have different senses of a term but insist there is no polysemy and they're all talking about the same thing.