2 ms·

I have the opposite problem. I'm not well-calibrated on when I'd want lower reasoning than what's available to me (and how to compare that to lower-tier models)

I have the opposite problem. I'm not well-calibrated on when I'd want lower reasoning than what's available to me (and how to compare that to lower-tier models). OpenAI now has Luna, Terra and Sol, each at Low, Medium, High and Xhigh, with Pro/Ultra depending on harness and plan. That's ~15 possible combinations of model and reasoning level, and there isn't a satisfactory explanation of which one you want for any particular task. I feel that work is basically split into two tiers, hard (which requires a good model and lots of reasoning by definition) and relatively easy (which won't consume much of my limits despite a great model and reasoning, so I may just as well keep it on Sol High).

redox99 3mo ago
Yeah, it's hard, there's so many permutations. But most are bad so it narrows it down. Luna: Can be useful if price sensitive, always use with AT LEAST high effort. But codex plans are very generous, so just ignore it. Terra: Forget it exists Sol: Just use this. Medium can work for specific edits. Otherwise just use high or xhigh. OpenAI basically agrees with this, and the slider gives you those options. TL;DR: Just use sol medium/high/xhigh
- drivebyhooting 3mo ago
  Why not pro/ultra?
  - redox99 3mo ago
    Pro: It's web chat only, I haven't tried it so idk. I think it's useful for Math and that kind of stuff, but I only do programming. Ultra: Might be useful, the one time I tried it, it was extremely wasteful in terms of tokens. Definitely not a daily driver. There's also max effort, I think it can be useful but the gains compared to xhigh are quite small. I think max and ultra are only worth it when the others fail.
    - drivebyhooting 3mo ago
      It’s very confusing to me. I’ve occasionally used pro to do high-level research and design. Then I ask it to create a prompt for Codex ultra. Ultra can do a lot of genetic benchmarking and testing to elucidate and resolve quandaries. I don’t know if this is a good workflow.
majormajor 3mo ago
> (which won't consume much of my limits despite a great model and reasoning, so I may just as well keep it on Sol High) This equation significantly changes if you're paying API prices vs on a subscription. (Such as if you're integrating it into a different product, where it becomes worth it to figure out what the cheapest you can leverage is.)
andhuman 2mo ago
I don’t see why they just don’t allow a smaller model to answer the question while letting the bigger one vet it. The vetting can be asynchronous and can be delivered after a few seconds (if it’s an easy query). If it’s a hard query, the UI can show the answer is currently being vetted or something.
- Davidzheng 2mo ago
  if the bigger model can vet it fast it can also answer it fast.

redox99 3mo ago

Yeah, it's hard, there's so many permutations. But most are bad so it narrows it down. Luna: Can be useful if price sensitive, always use with AT LEAST high effort. But codex plans are very generous, so just ignore it. Terra: Forget it exists Sol: Just use this. Medium can work for specific edits. Otherwise just use high or xhigh. OpenAI basically agrees with this, and the slider gives you those options. TL;DR: Just use sol medium/high/xhigh

drivebyhooting 3mo ago
Why not pro/ultra?
- redox99 3mo ago
  Pro: It's web chat only, I haven't tried it so idk. I think it's useful for Math and that kind of stuff, but I only do programming. Ultra: Might be useful, the one time I tried it, it was extremely wasteful in terms of tokens. Definitely not a daily driver. There's also max effort, I think it can be useful but the gains compared to xhigh are quite small. I think max and ultra are only worth it when the others fail.
  - drivebyhooting 3mo ago
    It’s very confusing to me. I’ve occasionally used pro to do high-level research and design. Then I ask it to create a prompt for Codex ultra. Ultra can do a lot of genetic benchmarking and testing to elucidate and resolve quandaries. I don’t know if this is a good workflow.

majormajor 3mo ago

> (which won't consume much of my limits despite a great model and reasoning, so I may just as well keep it on Sol High) This equation significantly changes if you're paying API prices vs on a subscription. (Such as if you're integrating it into a different product, where it becomes worth it to figure out what the cheapest you can leverage is.)

andhuman 2mo ago

I don’t see why they just don’t allow a smaller model to answer the question while letting the bigger one vet it. The vetting can be asynchronous and can be delivered after a few seconds (if it’s an easy query). If it’s a hard query, the UI can show the answer is currently being vetted or something.

Davidzheng 2mo ago
if the bigger model can vet it fast it can also answer it fast.