4 ms·

This is a weakness of the training data, not the architecture. Training is not only about learning information, but also learning how to handle it. It will onl

This is a weakness of the training data, not the architecture. Training is not only about learning information, but also learning how to handle it. It will only learn to reason "if A-> B then B->A" if the data contains situations where it must do this. In the paper, their training process contained no examples where this reasoning was necessary - only A->B relations. It actually got worse than the base model, because GPT-3's training data did contain some examples of B->A relations.

dragonwriter 3y ago
> "if A-> B then B->A” This would be invalid logic. The issue is “if A=B then B=A” not “if A->B then B->A” But, more, the issue is recognizing when “X is Y” is describing X being a member of a broader set (“George Washington is a former President of the United States” does not imply that “George Washington” and “a former president” are equivalent) vs. a statement of equivalency (“George Washington is the first President of the United States”). Now, in many cases, the use of a definite article (“the”) vs. an indefinite article (“a/an”) after “is” is determinative, but there are cases where no article is used that can go either way, and there’s probably cases where the use of articles is confusing (the definite article can often apply in a limited rather than general context, for instance.) I agree that this is a training data not model issue, but its also a more complex training issue that it might naively seem. I really don’t think that it is surprising or a particularly crushing revelation that LLMs don’t apply logical rules like this without training both on the rules and the identification of where they apply. A lot of what we have with modern LLMs is throwing a lot of data at them without focus on what they are supposed to learn outside of a very narrow set of tasks, and then discovering what they did and didn’t learn outside of that, and then if something turns out to be important and not learned in one generation from the general data thrown at it, doing more focused training on a later generation targeting that concept.

dragonwriter 3y ago

> "if A-> B then B->A” This would be invalid logic. The issue is “if A=B then B=A” not “if A->B then B->A” But, more, the issue is recognizing when “X is Y” is describing X being a member of a broader set (“George Washington is a former President of the United States” does not imply that “George Washington” and “a former president” are equivalent) vs. a statement of equivalency (“George Washington is the first President of the United States”). Now, in many cases, the use of a definite article (“the”) vs. an indefinite article (“a/an”) after “is” is determinative, but there are cases where no article is used that can go either way, and there’s probably cases where the use of articles is confusing (the definite article can often apply in a limited rather than general context, for instance.) I agree that this is a training data not model issue, but its also a more complex training issue that it might naively seem. I really don’t think that it is surprising or a particularly crushing revelation that LLMs don’t apply logical rules like this without training both on the rules and the identification of where they apply. A lot of what we have with modern LLMs is throwing a lot of data at them without focus on what they are supposed to learn outside of a very narrow set of tasks, and then discovering what they did and didn’t learn outside of that, and then if something turns out to be important and not learned in one generation from the general data thrown at it, doing more focused training on a later generation targeting that concept.