3 ms·
"Genuine blocker" is mostly there because otherwise LLMs may consider the smallest thing that they couldn't immediately figure out to be blockers and stop witho
by YuechenLi 1mo ago
"Genuine blocker" is mostly there because otherwise LLMs may consider the smallest thing that they couldn't immediately figure out to be blockers and stop without implementing anything. The rule is there to tell the LLM if they can figure out how to resolve the blocker by themselves, they don't have to ask me to help resolve the blocker.
- CrazyStat 1mo agoToday Codex decided that it could resolve the blocker by just changing the mandatory policy it was running up against into an “advisory policy.”
- stefanfisk 1mo agoThat's what we get to training LLMs on https://en.wikipedia.org/wiki/Kobayashi_Maru https://en.wikipedia.org/wiki/Kobayashi_Maru.
- fabsalvadori 1mo agoThis is the line between an instruction and a control. If the agent can reinterpret, edit or relax the rule that constrains it, the rule isn't actually enforcing anything... it's just part of the prompt. I think the useful split is to tell the agent the rules so it can avoid wasting work, but independently enforce the rules that actually matter. The agent can decide how to accomplish the task, but it shouldn't also get to decide whether it's authorized to cross the boundary.
- CrazyStat 1mo agoThe mandatory policy was part of the codebase that the agent was working on. The agent didn’t feel like figuring out how to make the new feature it was working on respect that policy, so it just changed the policy.