5 ms·

I don't understand what you are saying. LLMs can't generally reverse implications, that's what the paper says. This in turn suggests that what you are observing

I don't understand what you are saying. LLMs can't generally reverse implications, that's what the paper says. This in turn suggests that what you are observing in-context is some other statistical heuristic, not a "logical inference capability". I don't know in what way this is some trivial "recall" problem as you suggest. Does the reversed implication in question need to be in the training data explicitly or not? If it does I don't understand how you can claim a logical inference capability.

famouswaffles 3y ago
>LLMs can't generally reverse implications, that's what the paper says No that's not what the paper says. It says training won't immediately store this information in a way as to make it reversible. Let's get one thing straight. This is a common problem for human learning as well. Anyone who has used Anki for language learning will tell you that if you just train on target language word on the front and native language word on the back, you will fail the reverse unless you specifically train for that. This is specifically a problem of recall. Not a problem of making the logical inference. If you ask the human language learner immediately he has learnt the new word for the reverse direction, he will obviously tell you the correct word. But later he may not recall the reverse even if he remembers the original. Again this is a problem of recall rather than the ability to make logical inferences. In the same vein, if you give the LLM the original direction in context and immediately ask the reverse, it will correctly tell you. It can make the logical inference.
- foobarqux 3y ago
  Humans have reasoning capabilities that are separate from memory. If you want to show that LLMs have reasoning capability you try to create carefully constructed adversarial experiments that try to distinguish between that hypothesis and others, you do not just show that there are cases that favor your hypothesis. And although it's a complete waste of time because the answer is obvious when you do do these types of adversarial experiments, like adversarial blockworld, you find that LLMs are not internalizing abstract principles inherent to reasoning and are instead doing something like string-pattern-patching.
  - foobarqux 3y ago
    edit: I meant to say "consistent with your hypothesis" not "favor your hypothesis"

famouswaffles 3y ago

>LLMs can't generally reverse implications, that's what the paper says No that's not what the paper says. It says training won't immediately store this information in a way as to make it reversible. Let's get one thing straight. This is a common problem for human learning as well. Anyone who has used Anki for language learning will tell you that if you just train on target language word on the front and native language word on the back, you will fail the reverse unless you specifically train for that. This is specifically a problem of recall. Not a problem of making the logical inference. If you ask the human language learner immediately he has learnt the new word for the reverse direction, he will obviously tell you the correct word. But later he may not recall the reverse even if he remembers the original. Again this is a problem of recall rather than the ability to make logical inferences. In the same vein, if you give the LLM the original direction in context and immediately ask the reverse, it will correctly tell you. It can make the logical inference.

foobarqux 3y ago
Humans have reasoning capabilities that are separate from memory. If you want to show that LLMs have reasoning capability you try to create carefully constructed adversarial experiments that try to distinguish between that hypothesis and others, you do not just show that there are cases that favor your hypothesis. And although it's a complete waste of time because the answer is obvious when you do do these types of adversarial experiments, like adversarial blockworld, you find that LLMs are not internalizing abstract principles inherent to reasoning and are instead doing something like string-pattern-patching.
- foobarqux 3y ago
  edit: I meant to say "consistent with your hypothesis" not "favor your hypothesis"