10 ms·
I've always understood it more to mean, "phenomena that happen due to the interactions of a system's parts without being explicitly encoded into their individua
by bunderbunder 1y ago
I've always understood it more to mean, "phenomena that happen due to the interactions of a system's parts without being explicitly encoded into their individual behavior." Fractal patterns in nature are a great example of emergent phenomena. A single water molecule contains no explicit plan for how to get together with its buddies and make spiky hexagon shapes when they get cold.
And I've always understood talking about emergence as if it were some sort of quasi-magical and unprecedented new feature of LLMs to mean, "I don't have a deep understanding of how machine learning works." Emergent behavior is the entire point of artificial neural networks, from the latest SOTA foundation model all the way back to the very first tiny little multilayer perceptron.
- andy99 1y agoEmergence in the context of LLMs is really just us learning that "hey, you don't actually need intelligence to do <task>, turns out it can be done using a good enough next token predictor. We're basically learning what intelligence isn't as we see some of the things these models can do. I always understood this to be the initial framing, e.g. in the Language Models are Few Shot Learners paper but then it got flipped around.
- dmd 1y agoOr maybe you need intelligence to be a good enough next token predictor. Maybe the thing that “just” predicts the next token can be called “intelligence”.
- bunderbunder 1y agoMaybe? Mostly I just think that "Intelligence" and "AI" go together like "life, the universe and everything" and "42".
- HPsquared 1y agoOr more generally "fitting a model to data".
- Workaccount2 1y agoThe challenge there would be showing that humans have this thing called intelligence. You yourself are just outputting ephemeral actions that rise out of your subconscious. We have no idea what that system feeding our output looks like (except it's some kind of organic neural net) and hence there isn't really a basis for discriminating what is and isn't intelligent besides "if it solves problems, it has some degree of intelligence"
- PaulDavisThe1st 1y agoTo return an old but still good analogy ... If you want to understand how birds fly, the fact that planes also fly is near useless. While a few common aerodynamic principles apply, both types of flight are so different from each other that you do not learn very much about one from the other. On the other hand, if your goal is just "humans moving through the air for extended distances", it doesn't matter at all that airplanes do not fly the way birds do. And then, on the generated third hand, if you need the kind of tight quarters maneuverability that birds can do in forests and other tangled spaces, then the way our current airplanes fly is of little to no use at all, and you're going to need a very different sort of technology than the one used in current aircraft. And on the accidentally generated fourth hand, if your goal is "moving very large mass over very long distance", the the mechanisms of bird flight are likely to be of little utility. The fact that two different systems can be described in a similar way (e.g. "flying") doesn't by itself tell you that they are working in remotely the same way or capable of the same sorts of things.
- Workaccount2 1y agoI think that many birds gets too sensitive when discussing what "flight" means, heh
- andy99 1y agoA better bird analogy would be if we didn't understand at all how flight worked, and then started throwing rocks and had pseudo-intellectuals saying "how do we know that isn't all that flight is, we've clearly invented artificial flight".
- prats226 1y agoIf we use some metric as proxy for intelligence, emergence simply means a non-linear sudden change in that metric?