3 ms·
There are lots of examples, the recent paper on LLMs failing to invert logic (A is B therefore B is A) is one. I don't know how you can use these systems for an
by foobarqux 3y ago
There are lots of examples, the recent paper on LLMs failing to invert logic (A is B therefore B is A) is one. I don't know how you can use these systems for any significant length of time and not run into logical errors.
- famouswaffles 3y ago>the recent paper on LLMs failing to invert logic (A is B therefore B is A) is one. That's not what that paper says
- foobarqux 3y agoYes, we discussed this before. You are wrong. If the system can produce the statement A is the mother of B but not B is son/daughter of A then it fails to do something humans can trivially do.
- famouswaffles 3y agoThe system can produce both and the paper makes this clear for those who bothered to read it.
- foobarqux 3y agoNo, you yourself said it can only do so in-context.
- BoppreH 3y agoBut at different rates from humans? I ran into the exact same issue myself today, while studying German flash cards. I could remember e.g., "ruhig -> quiet", but not "quiet -> ruhig".
- foobarqux 3y agoYou are confusing memory with logical inference. "A is the mother of B" is basically the same logical concept as "B is the child of A", you wouldn't know one and not the other. Because LLMs are basically next-word-prediction systems they need to be trained on both pieces of text. In fact you could train them on logically inconsistent text like "A is the mother of B" and "B is not the child of A" and they would happily spit out nonsense.
- Jensson 3y ago> In fact you could train them on logically inconsistent text like "A is the mother of B" and "B is not the child of A" and they would happily spit out nonsense. That is where LLMs are today, and why we can't solve the hallucination problem.