3 ms·
> People are being told it can do far more than it really is. Meanwhile these HN comments are split between: * Lots of people confirming what the paper itself
by kalkin 2y ago
> People are being told it can do far more than it really is.
Meanwhile these HN comments are split between:
* Lots of people confirming what the paper itself notes (but doesn't highlight), that the most advanced models actually can solve this problem at least a significant portion of the time. (A proportion which one can pretty easily project is only likely to increase with future models.)
* Lots of people saying "this confirms LLMs can't do reasoning".
Questions I'd ask you to consider:
* Is "LLMs can't do reasoning" actually more accurate than the typical hype?
* Is a "critical understanding of how [LLMs] work" that would predict they simply cannot solve this problem actually a good understanding?
- mjburgess 2y agoThe critical understanding doesnt predict that LLMs cannot solve problems. It predicts how they will solve them. There is no information, a priori, what the LLM has been trained on. You have to prompt, then see the answer. Once the answer arrives, the critical understanding provides a route to repairing the answer when not accurate or useful. LLMs do not reason. They appear to reason by repeating the structure of reasoning in their training data. This is indistinguishable in many cases.
- travis86 2y agoI am very curious how you would define 'reasoning'.