3 ms·

> The absence of one desired emergent property isn’t evidence against a different, observed property. Sure, but why one and not the other? You'd think there w

> The absence of one desired emergent property isn’t evidence against a different, observed property. Sure, but why one and not the other? You'd think there would be more examples of simple logical relationships in almost any training set than there would be examples of arithmetic. I don't see an obvious reason why rules for general arithmetic would just fall right out and a few basic rules of logical inference are much harder to get to.

dragonwriter 3y ago
> Sure, but why one and not the other? If we could describe the interaction of factors that determined it, we wouldn't call it an emergent property. > You'd think there would be more examples of simple logical relationships in almost any training set than there would be examples of arithmetic. I'd think the examples of arithmetic relationships would be more consistent (and often symbolic), while given the ambiguity of the language relevant to the kind of logical relationships at issue, it would be more murky to even infer the broad rule, and less clear from the language of particular examples whether the broad rule, even if it could be inferred, applies. > I don't see an obvious reason why rules for general arithmetic would just fall right out Again, if anyone could explain an obvious (or even non-obvious) reason for that, it wouldn’t be described as an emergent property. It would be described as a function of the specific things that contribute to it.
- photonthug 3y ago
  Idk, one obvious reason could be that emergence is a mirage ( https://arxiv.org/abs/2304.15004 https://arxiv.org/abs/2304.15004 ). Given two concepts of roughly equivalent complexity and representation in training, where one is successfully generalized and another is not, that would seem to support (but not prove) the mirage hypothesis. I take your point about representational differences though. Thinking more about this, the OP is basically talking about whether models understand that equality commutes, and I suppose even an LLM that can add may not have worked out that addition commutes.

dragonwriter 3y ago

> Sure, but why one and not the other? If we could describe the interaction of factors that determined it, we wouldn't call it an emergent property. > You'd think there would be more examples of simple logical relationships in almost any training set than there would be examples of arithmetic. I'd think the examples of arithmetic relationships would be more consistent (and often symbolic), while given the ambiguity of the language relevant to the kind of logical relationships at issue, it would be more murky to even infer the broad rule, and less clear from the language of particular examples whether the broad rule, even if it could be inferred, applies. > I don't see an obvious reason why rules for general arithmetic would just fall right out Again, if anyone could explain an obvious (or even non-obvious) reason for that, it wouldn’t be described as an emergent property. It would be described as a function of the specific things that contribute to it.

photonthug 3y ago
Idk, one obvious reason could be that emergence is a mirage ( https://arxiv.org/abs/2304.15004 https://arxiv.org/abs/2304.15004 ). Given two concepts of roughly equivalent complexity and representation in training, where one is successfully generalized and another is not, that would seem to support (but not prove) the mirage hypothesis. I take your point about representational differences though. Thinking more about this, the OP is basically talking about whether models understand that equality commutes, and I suppose even an LLM that can add may not have worked out that addition commutes.