3 ms·
What about LLM reasoning ability? Faith and Fate: Limits of Transformers on Compositionality https://arxiv.org/abs/2305.18654 https://arxiv.org/abs/2305.18654
by MAXPOOL 3y ago
What about LLM reasoning ability?
Faith and Fate: Limits of Transformers on Compositionality https://arxiv.org/abs/2305.18654 https://arxiv.org/abs/2305.18654
Transformers solve compositional reasoning tasks by reducing multi-step compositional reasoning into linearized subgraph matching without problem-solving skills. They can solve problems when they have reasoning graphs in the memory.
- esjeon 3y agoLLMs do logic by mimicking logical structures on the text level (and that's why they often need be ordered to do step-by-step for correct answers), so this one may also have the same ability as long as memories are properly utilized.
- BSEdlMMldESB 3y agoI think this boils down to the capacity to match together parenthesis in a logical-syntax way however, the "parenthesis" can be any symbol. even grammatical clauses are one sort of "parenthesis" in the way I'm thinking about them
- abc_lisper 3y agoFunny, I write Clojure for my day job and fun, so I have tried to use ChatGPT to generate code. If anything, it sucks at paren matching. It reminded me of stable diffusion's "six finger problem".
- BSEdlMMldESB 3y agoas I said, it's not exactly "parenthesis" with the strictness that real programing needs. in fact, my whole idea has got me on a deep dive into the nature of the decimal point (up to which extent is the decimal point representation of numbers and instance of a "fixed point"? I don't know! I cannot understand a fix point just yet; and for me to say I get decimal notation actually means I understand something about p-adic representation; which I'm still working on figuring out) I thought these models got more 'logical' after training with computer code