3 ms·

LLMs work by generating the most likely continuation to a prompt. But they can also generate multiple likely continuations. This create multiple branches which

by bjourne 4mo ago

LLMs work by generating the most likely continuation to a prompt. But they can also generate multiple likely continuations. This create multiple branches which in turn can generate even more branches. The LLM can then evaluate the branches, prune the unpromising ones, and merge the best ones. More branches means more tokens, means more effort.

simianwords 4mo ago
this has nothing to do with the thinking effort however
- bjourne 4mo ago
  Yes, it does. Breadth of search is exactly what the effort setting controls.
  - pyentropy 4mo ago
    LLM-judge/parallel branching ≠ multi-token prediction ≠ reasoning effort. See https://developers.openai.com/cookbook/articles/openai-harmony#example-system-message https://developers.openai.com/cookbook/articles/openai-harmo... and src/openai/types/shared/reasoning_effort.py
    - bjourne 4mo ago
      [flagged]
      - simianwords 4mo ago
        No it doesn't and lets not call people names. You can verify this using ChatGPT or anything else. You are mistaken and there are no "branches" happening.
        bjourne 4mo ago
        [flagged]
        tomhow 4mo ago
        Hey, name-calling like this is not cool on HN. You've been here long enough to know this, and we've asked you repeatedly to observe the guidelines. If you keep this up we'll have to assume you have no intention of using the site as intended and ban the account.
  - FergusArgyll 4mo ago
    I think you may be confusing the openai "pro" series models with thinking. Thos are rumored to be multi "branched"