6 ms·

How could it fail to solve some maths problems if it has a method for reasoning through things?

chairhairair 2y ago
Simple questions like this are not welcomed by LLM hype sellers. The word "reasoning" is being used heavily in this announcement, but with an intentional corruption of the normal meaning. The models are amazing but they are fundamentally not "reasoning" in a way we'd expect a normal human to. This is not a "distinction without a difference". You still CANNOT rely on the outputs of these models in the same way you can rely on the outputs of simple reasoning.
- exe34 2y ago
  it depends who's doing the simple reasoning. Richard Feynman? yes. Donald Trump? no.
logicchains 2y ago
I have a method for reasoning through things but I'm pretty sure I'd fail some of those tough math problems too.
HarHarVeryFunny 2y ago
It's using tree search (tree of thoughts), driven by some RL-derived heuristics controlling what parts of the practically infinite set of potential responses to explore. How good the responses are will depend on how good these heuristics are.
- RandomLensman 2y ago
  That doesn't sound like a method for reasoning.
  - HarHarVeryFunny 2y ago
    It's hard to judge how similar the process is to human reasoning (which is effectively also a tree search), but apparently the result is the same in many cases. They are only vaguely describing the process: "Similar to how a human may think for a long time before responding to a difficult question, o1 uses a chain of thought when attempting to solve a problem. Through reinforcement learning, o1 learns to hone its chain of thought and refine the strategies it uses. It learns to recognize and correct its mistakes. It learns to break down tricky steps into simpler ones. It learns to try a different approach when the current one isn’t working. This process dramatically improves the model’s ability to reason."
    - RandomLensman 2y ago
      Not sure the way to superior "reasoning machines" would be through emulating humans.
      - HarHarVeryFunny 2y ago
        True, although it's not clear exactly what this is really doing. The RL was presumably trained on human input, but the overall agentic flow (it seems this is an agent), sounds to me like a neuro-symbolic hybrid, potentially brute force iterating to great depth, so maybe more computer than brain inspired. It seems easy to imagine this type of approach being super human on narrow tasks that play to it's strengths such as pure reasoning tasks (math/science), but it's certainly not AGI as for example there is no curiosity to explore the unknown, no ability to learn from exploration, etc. It'll take a while to become apparent exactly what types of real world application this is useful for, both in terms of capability and cost.
        RandomLensman 2y ago
        Even on narrow tasks: could you imagine such a system proving (or disproving) the Riemann hypothesis? Feels more like for narrow tasks with a kind of well defined approach, perhaps?
        2y ago
hatthew 2y ago
Because some steps in its reasoning were wrong
- RandomLensman 2y ago
  I would demand more from machine reasoning, just like we demand an extremely low error rate from machine calculations.

Simple questions like this are not welcomed by LLM hype sellers. The word "reasoning" is being used heavily in this announcement, but with an intentional corruption of the normal meaning. The models are amazing but they are fundamentally not "reasoning" in a way we'd expect a normal human to. This is not a "distinction without a difference". You still CANNOT rely on the outputs of these models in the same way you can rely on the outputs of simple reasoning.

exe34 2y ago
it depends who's doing the simple reasoning. Richard Feynman? yes. Donald Trump? no.

logicchains 2y ago

I have a method for reasoning through things but I'm pretty sure I'd fail some of those tough math problems too.

HarHarVeryFunny 2y ago

It's using tree search (tree of thoughts), driven by some RL-derived heuristics controlling what parts of the practically infinite set of potential responses to explore. How good the responses are will depend on how good these heuristics are.

RandomLensman 2y ago
That doesn't sound like a method for reasoning.
- HarHarVeryFunny 2y ago
  It's hard to judge how similar the process is to human reasoning (which is effectively also a tree search), but apparently the result is the same in many cases. They are only vaguely describing the process: "Similar to how a human may think for a long time before responding to a difficult question, o1 uses a chain of thought when attempting to solve a problem. Through reinforcement learning, o1 learns to hone its chain of thought and refine the strategies it uses. It learns to recognize and correct its mistakes. It learns to break down tricky steps into simpler ones. It learns to try a different approach when the current one isn’t working. This process dramatically improves the model’s ability to reason."
  - RandomLensman 2y ago
    Not sure the way to superior "reasoning machines" would be through emulating humans.
    - HarHarVeryFunny 2y ago
      True, although it's not clear exactly what this is really doing. The RL was presumably trained on human input, but the overall agentic flow (it seems this is an agent), sounds to me like a neuro-symbolic hybrid, potentially brute force iterating to great depth, so maybe more computer than brain inspired. It seems easy to imagine this type of approach being super human on narrow tasks that play to it's strengths such as pure reasoning tasks (math/science), but it's certainly not AGI as for example there is no curiosity to explore the unknown, no ability to learn from exploration, etc. It'll take a while to become apparent exactly what types of real world application this is useful for, both in terms of capability and cost.
      - RandomLensman 2y ago
        Even on narrow tasks: could you imagine such a system proving (or disproving) the Riemann hypothesis? Feels more like for narrow tasks with a kind of well defined approach, perhaps?
        2y ago

hatthew 2y ago

Because some steps in its reasoning were wrong

RandomLensman 2y ago
I would demand more from machine reasoning, just like we demand an extremely low error rate from machine calculations.