2 ms·
>Similar to what others have noticed, and as I predicted 8 months ago, better models are requiring less ceremony to work effectively. It has nothing to do with
by orbital-decay 2mo ago
>Similar to what others have noticed, and as I predicted 8 months ago, better models are requiring less ceremony to work effectively.
It has nothing to do with model capabilities, it's a result of purposeful persistence training at the cost of everything else from OpenAI. If you give Fable or Opus (comparable models) an "ask user" tool they will use it for ambiguous requests. Sol will never use it without a nudge and will just assume its own interpretation. Of course if you train the model to be persistent it will be persistent.
- int3trap 2mo agoThat's not been my experience at all with Sol. I've provide it escape hatch tools to stop execution and I've found it's been TOO eager to stop a request user approval to move forward.
- orbital-decay 2mo agoIt might be sensitive to the prompt and tool definitions the harness provides, I can imagine it overusing the tool if being directly asked to resolve ambiguities. But in a straight Fable vs Sol comparison where I control the context and make sure I don't push the model, Fable consistently uses the ask user tool and Sol ignores it for me. Moreover, GPT has been known to do something like this since 4o if not earlier, it tries to ignore anything it perceives as orphaned context piece (e.g. XML sections with meaningful names but no explicit instructions on what to do with them). I suspect they specifically train it for that.