4 ms·

The "GPT-4 Can't Reason" paper [1] [2] is excellent and proves without a shadow of a doubt that there is no reasoning taking place. It also raises strong doubts

The "GPT-4 Can't Reason" paper [1] [2] is excellent and proves without a shadow of a doubt that there is no reasoning taking place. It also raises strong doubts about the efficacy and reliability of multi-agent approaches that rely on LLM-based planning (such as Cradle [3], which is currently on the front page). When it was previously discussed here, it received a torrent of low-quality comments that can only be described as confirmation bias: commenters tried the examples verbatim (instead of introducing random perturbations to get around model updates) in order to disprove the thesis. It's sad that there are "engineers" out there so blinded by their own (wishful thinking, vested interests) that can not accept the obvious. [1] https://arxiv.org/abs/2308.03762 https://arxiv.org/abs/2308.03762 [2] https://medium.com/@konstantine_45825/gpt-4-cant-reason-addendum-ed79d8452d44 https://medium.com/@konstantine_45825/gpt-4-cant-reason-adde... [3] https://baai-agents.github.io/Cradle/ https://baai-agents.github.io/Cradle/

krageon 2y ago
I think most people are a little bit tired of hearing that things that are manifestly possible, are instead impossible. If we'd take the time to recognise that the truth is somewhere in the middle (lots of tasks can be done, lots of them cannot), the conversation could at the very least be interesting. As it stands every publication on this subject is filled with a lot of people that feel they need a polarised stance or they'll immediately be shot: Either they say LLMs are fundamentally broken and can't do shit (this is ridiculous) or they say that LLMs are self-aware and we should give them every single task ever (also ridiculous). It is really tedious to read the same dumb shit every single time.
- mewpmewp2 2y ago
  If any LLM can solve a novel exercise that requires some form of reasoning, wouldn't that be evidence that LLMs are capable of at least some level of reasoning? Not necessarily AGI or average human level reasoning, but just any form.
deleted 2y ago
[deleted]

krageon 2y ago

I think most people are a little bit tired of hearing that things that are manifestly possible, are instead impossible. If we'd take the time to recognise that the truth is somewhere in the middle (lots of tasks can be done, lots of them cannot), the conversation could at the very least be interesting. As it stands every publication on this subject is filled with a lot of people that feel they need a polarised stance or they'll immediately be shot: Either they say LLMs are fundamentally broken and can't do shit (this is ridiculous) or they say that LLMs are self-aware and we should give them every single task ever (also ridiculous). It is really tedious to read the same dumb shit every single time.

mewpmewp2 2y ago
If any LLM can solve a novel exercise that requires some form of reasoning, wouldn't that be evidence that LLMs are capable of at least some level of reasoning? Not necessarily AGI or average human level reasoning, but just any form.

deleted 2y ago

[deleted]