3 ms·

"We found that LRMs have limitations in exact computation: they fail to use explicit algorithms and reason inconsistently across puzzles." That's the real conc

"We found that LRMs have limitations in exact computation: they fail to use explicit algorithms and reason inconsistently across puzzles." That's the real conclusion of the Apple paper. It's correct. LLMs are terrible at arithmetic, or even counting. We knew that. So, now what? It would be interesting to ask the "AI system" to write a program to solve such puzzle problems. Most of the puzzles given have an algorithmic solution. This may be a strategy problem. LLMs may need to internalize Polya's How to Solve It?[2] Read the linked Wikipedia article. Most of those are steps an LLM can do, but a strategy controller is needed to apply them in a useful order and back off when stuck. The "Illusion of thinking" article is far less useful than the the Apple paper. (Did anybody proofread the Apple paper? [1] There's a misplaced partial sentence in the middle of page 2. Or a botched TeX macro.) [1] https://ml-site.cdn-apple.com/papers/the-illusion-of-thinking.pdf https://ml-site.cdn-apple.com/papers/the-illusion-of-thinkin... [2] https://en.wikipedia.org/wiki/How_to_Solve_It https://en.wikipedia.org/wiki/How_to_Solve_It

withinboredom 1y ago
Eventually (many of us have hit it, and if you haven’t, you will), your program will hit a certain size/complexity where the AI also falls flat. These tests basically try to measure that limit, not just solve the puzzle.
- scotty79 1y ago
  So far they've shown that it fails early if the answer would exceed context size anyways. At least that was the case for the Hanoi problem from what I heard. They somehow failed to notice that though.
tsurba 1y ago
Is it a puzzle if there is no algorithm? But testing via coding algos to known puzzles is problematic as the code may be in the training set. Hence you need new puzzles, which is kinda what ARC was meant to do, right? Too bad OpenAI lost credibility for that set by having access to it, but ”verbally promising” (lol) not to train on it, etc.
pram 1y ago
LLMs are awful and untrustworthy at math on their own for sure, but I found out you can tell Claude to use bc and suddenly they're not so bad when they have a calculator. It's as smart as me in that respect ;P
- imiric 1y ago
  You underestimate your abilities, and overestimate the LLM's. Which version of `bc` was the LLM trained on? Perhaps this is not so important for a stable tool that doesn't change much, but it's critical for many programs and programming libraries. I lost count of the number of times when an LLM generated code that doesn't work for the current version of the library I'm using. In some cases you can tell it to use a specific version, or even feed it the documentation for that version, but that often fails to correct the issue. And this is without considering the elephant in the room: hallucination. LLMs will often mix up or invent APIs that don't exist. Apologists will say that agents are the solution, and that feeding the error back to the LLM will fix it, but that is often not the case either. That might even make things worse by breaking something else, especially in large-context sessions.
  - pram 1y ago
    Oh yeah it’s still going to hallucinate eventually, 100%. My point is mostly you can take it from “absolutely useless” to “situationally useful” if you’re honest about its limitations and abilities. Nothing has convinced me that LLMs will ever be good at math on their own.

withinboredom 1y ago

Eventually (many of us have hit it, and if you haven’t, you will), your program will hit a certain size/complexity where the AI also falls flat. These tests basically try to measure that limit, not just solve the puzzle.

scotty79 1y ago
So far they've shown that it fails early if the answer would exceed context size anyways. At least that was the case for the Hanoi problem from what I heard. They somehow failed to notice that though.

tsurba 1y ago

Is it a puzzle if there is no algorithm? But testing via coding algos to known puzzles is problematic as the code may be in the training set. Hence you need new puzzles, which is kinda what ARC was meant to do, right? Too bad OpenAI lost credibility for that set by having access to it, but ”verbally promising” (lol) not to train on it, etc.

pram 1y ago

LLMs are awful and untrustworthy at math on their own for sure, but I found out you can tell Claude to use bc and suddenly they're not so bad when they have a calculator. It's as smart as me in that respect ;P

imiric 1y ago
You underestimate your abilities, and overestimate the LLM's. Which version of `bc` was the LLM trained on? Perhaps this is not so important for a stable tool that doesn't change much, but it's critical for many programs and programming libraries. I lost count of the number of times when an LLM generated code that doesn't work for the current version of the library I'm using. In some cases you can tell it to use a specific version, or even feed it the documentation for that version, but that often fails to correct the issue. And this is without considering the elephant in the room: hallucination. LLMs will often mix up or invent APIs that don't exist. Apologists will say that agents are the solution, and that feeding the error back to the LLM will fix it, but that is often not the case either. That might even make things worse by breaking something else, especially in large-context sessions.
- pram 1y ago
  Oh yeah it’s still going to hallucinate eventually, 100%. My point is mostly you can take it from “absolutely useless” to “situationally useful” if you’re honest about its limitations and abilities. Nothing has convinced me that LLMs will ever be good at math on their own.