3 ms·

Haven't I seen lots of stuff showing LLMs do arithmetic for examples they haven't seen? If there is an issue really grasping basic logic though, doesn't that pu

Haven't I seen lots of stuff showing LLMs do arithmetic for examples they haven't seen? If there is an issue really grasping basic logic though, doesn't that put a damper on the spooky "emergent properties" explanation for stuff like addition?

viraptor 3y ago
Maybe? The list is talking about one side of the issue I think - currently trained LLMs don't automatically remember the inverse of this relation. But the other side is: if you provide enough training on relations similar to this one, will the LLM start applying it to other examples as well?
dragonwriter 3y ago
> Haven’t I seen lots of stuff showing LLMs do arithmetic for examples they haven’t seen? If there is an issue really grasping basic logic though, doesn’t that put a damper on the spooky “emergent properties” explanation for stuff like addition? No. The absence of one desired emergent property isn’t evidence against a different, observed property. (And “emergent properties” isn’t an explanation as much as a statement that we don’t understand the mechanism by which the training data encodes the knowledge and did not plan for it to be encoded.)
- photonthug 3y ago
  > The absence of one desired emergent property isn’t evidence against a different, observed property. Sure, but why one and not the other? You'd think there would be more examples of simple logical relationships in almost any training set than there would be examples of arithmetic. I don't see an obvious reason why rules for general arithmetic would just fall right out and a few basic rules of logical inference are much harder to get to.
  - dragonwriter 3y ago
    > Sure, but why one and not the other? If we could describe the interaction of factors that determined it, we wouldn't call it an emergent property. > You'd think there would be more examples of simple logical relationships in almost any training set than there would be examples of arithmetic. I'd think the examples of arithmetic relationships would be more consistent (and often symbolic), while given the ambiguity of the language relevant to the kind of logical relationships at issue, it would be more murky to even infer the broad rule, and less clear from the language of particular examples whether the broad rule, even if it could be inferred, applies. > I don't see an obvious reason why rules for general arithmetic would just fall right out Again, if anyone could explain an obvious (or even non-obvious) reason for that, it wouldn’t be described as an emergent property. It would be described as a function of the specific things that contribute to it.
    - photonthug 3y ago
      Idk, one obvious reason could be that emergence is a mirage ( https://arxiv.org/abs/2304.15004 https://arxiv.org/abs/2304.15004 ). Given two concepts of roughly equivalent complexity and representation in training, where one is successfully generalized and another is not, that would seem to support (but not prove) the mirage hypothesis. I take your point about representational differences though. Thinking more about this, the OP is basically talking about whether models understand that equality commutes, and I suppose even an LLM that can add may not have worked out that addition commutes.
emmender1 3y ago
spooky AI taking over the world was FUD spread by those with strong vested interests. and to be fair, a many credulous tech-types and their VC handlers were goaded on by this misson to "tame the AI beast". For the VCs it is always a search for the next big bandwagon, and the W3-crypto bust has left some of them licking their wounds. the reality is that the emperor has no clothes (cant compute). LLM is a great assistant but is not showing any sparks of AGI. Those seeing sparks of AGI are in denial due to vested interests.

viraptor 3y ago

Maybe? The list is talking about one side of the issue I think - currently trained LLMs don't automatically remember the inverse of this relation. But the other side is: if you provide enough training on relations similar to this one, will the LLM start applying it to other examples as well?

dragonwriter 3y ago

> Haven’t I seen lots of stuff showing LLMs do arithmetic for examples they haven’t seen? If there is an issue really grasping basic logic though, doesn’t that put a damper on the spooky “emergent properties” explanation for stuff like addition? No. The absence of one desired emergent property isn’t evidence against a different, observed property. (And “emergent properties” isn’t an explanation as much as a statement that we don’t understand the mechanism by which the training data encodes the knowledge and did not plan for it to be encoded.)

photonthug 3y ago
> The absence of one desired emergent property isn’t evidence against a different, observed property. Sure, but why one and not the other? You'd think there would be more examples of simple logical relationships in almost any training set than there would be examples of arithmetic. I don't see an obvious reason why rules for general arithmetic would just fall right out and a few basic rules of logical inference are much harder to get to.
- dragonwriter 3y ago
  > Sure, but why one and not the other? If we could describe the interaction of factors that determined it, we wouldn't call it an emergent property. > You'd think there would be more examples of simple logical relationships in almost any training set than there would be examples of arithmetic. I'd think the examples of arithmetic relationships would be more consistent (and often symbolic), while given the ambiguity of the language relevant to the kind of logical relationships at issue, it would be more murky to even infer the broad rule, and less clear from the language of particular examples whether the broad rule, even if it could be inferred, applies. > I don't see an obvious reason why rules for general arithmetic would just fall right out Again, if anyone could explain an obvious (or even non-obvious) reason for that, it wouldn’t be described as an emergent property. It would be described as a function of the specific things that contribute to it.
  - photonthug 3y ago
    Idk, one obvious reason could be that emergence is a mirage ( https://arxiv.org/abs/2304.15004 https://arxiv.org/abs/2304.15004 ). Given two concepts of roughly equivalent complexity and representation in training, where one is successfully generalized and another is not, that would seem to support (but not prove) the mirage hypothesis. I take your point about representational differences though. Thinking more about this, the OP is basically talking about whether models understand that equality commutes, and I suppose even an LLM that can add may not have worked out that addition commutes.

emmender1 3y ago

spooky AI taking over the world was FUD spread by those with strong vested interests. and to be fair, a many credulous tech-types and their VC handlers were goaded on by this misson to "tame the AI beast". For the VCs it is always a search for the next big bandwagon, and the W3-crypto bust has left some of them licking their wounds. the reality is that the emperor has no clothes (cant compute). LLM is a great assistant but is not showing any sparks of AGI. Those seeing sparks of AGI are in denial due to vested interests.