4 ms·

Here is another opinion on the original paper - Comment on The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Le

Here is another opinion on the original paper - Comment on The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity https://arxiv.org/abs/2506.09250 https://arxiv.org/abs/2506.09250

tkgally 1y ago
The points made in the abstract of that paper look like pretty serious challenges to the Apple paper. What might be the counterobjections?
- jmsdnns 1y ago
  Worth looking at how many papers the authors have published before assuming they challenge Apple.
  - frozenseven 1y ago
    If they make a solid case against the Apple paper, why does it matter? It's not like they're the first ones to criticize (read: easily debunk) Apple on such grounds.
    - jmsdnns 1y ago
      Apple's paper said a bunch of folks with weird predictions are wrong and so obviously lots of bad information will emerge in response. Who says what matters more now than it used to.
- imiric 1y ago
  Personal experience? I don't care about benchmarks, nor LLMs' capability to solve puzzles. This is the first thing AI companies optimize their training for, which is misleading and practically false advertising. I care about how good LLMs are for helping me with specific tasks. Do they generate code that on the surface appears correct, yet on closer inspection has security and performance issues, is unnecessarily complex, often doesn't even compile, which takes me more time to troubleshoot and fix than if I were to write it myself? Do they explain concepts confidently while being wrong, which I have no way of knowing unless I'm a domain expert? Do they repeat all these issues even after careful re-prompting and with all the contextual information they would need? Does all this waste my time more than it helps? The answer is yes to all of the above. So while we can argue whether LLMs can think and reason, my personal experience tells me that they absolutely cannot, and that any similarity to what humans can do is nothing but an illusion.

tkgally 1y ago

The points made in the abstract of that paper look like pretty serious challenges to the Apple paper. What might be the counterobjections?

jmsdnns 1y ago
Worth looking at how many papers the authors have published before assuming they challenge Apple.
- frozenseven 1y ago
  If they make a solid case against the Apple paper, why does it matter? It's not like they're the first ones to criticize (read: easily debunk) Apple on such grounds.
  - jmsdnns 1y ago
    Apple's paper said a bunch of folks with weird predictions are wrong and so obviously lots of bad information will emerge in response. Who says what matters more now than it used to.
imiric 1y ago
Personal experience? I don't care about benchmarks, nor LLMs' capability to solve puzzles. This is the first thing AI companies optimize their training for, which is misleading and practically false advertising. I care about how good LLMs are for helping me with specific tasks. Do they generate code that on the surface appears correct, yet on closer inspection has security and performance issues, is unnecessarily complex, often doesn't even compile, which takes me more time to troubleshoot and fix than if I were to write it myself? Do they explain concepts confidently while being wrong, which I have no way of knowing unless I'm a domain expert? Do they repeat all these issues even after careful re-prompting and with all the contextual information they would need? Does all this waste my time more than it helps? The answer is yes to all of the above. So while we can argue whether LLMs can think and reason, my personal experience tells me that they absolutely cannot, and that any similarity to what humans can do is nothing but an illusion.