3 ms·
To me, it seems like a good definition. If scaling the size of the model leads to a linear increase in capability, then nothing is emergent. If scaling the si
by bravura 4y ago
To me, it seems like a good definition.
If scaling the size of the model leads to a linear increase in capability, then nothing is emergent.
If scaling the size of the model accretes almost no gains, and then there is a huge unpredicted step in capabilities, a capability has "emerged".
- chasd00 4y agoI thought the thing that is scaring people is that there isn’t a defined relationship between model scale and capabilities. You may triple the model size and get no increase but then quadruple it and get 10,000% increase in capability. So no one knows where these massive models are headed and people want to pause until they understand the relationship.
- layer8 4y agoEmergence doesn’t need to depend on scale. For example, gliders in the Game of Life are emergent behavior, and they don’t depend on scale. The way proteins fold is emergent from the physical forces, but is not a function of scale, just of particular configurations. Emergence is about the level on which an explanation or description works. You only “see” the glider move at a higher level of description than the cellular rules.
- Havoc 4y agohmm...yes that's perhaps better. Defining it as a bigger improvement than would be expected from the size increase. Don't think the baseline expectation would be linear per se though