4 ms·

To be fair to the agent... I think there is some behind the scenes prompting from claude code (or open code, whichever is being used here) for plan vs build mo

by sgillen 7mo ago

To be fair to the agent... I think there is some behind the scenes prompting from claude code (or open code, whichever is being used here) for plan vs build mode, you can even see the agent reference that in its thought trace. Basically I think the system is saying "if in plan mode, continue planning and asking questions, when in build mode, start implementing the plan" and it looks to me(?) like the user switched from plan to build mode and then sent "no". From our perspective it's very funny, from the agents perspective maybe it's confusing. To me this seems more like a harness problem than a model problem.

christoff12 7mo ago
Asking a yes/no question implies the ability to handle either choice.
- not_kurt_godel 7mo ago
  This is a perfect example of why I'm not in any rush to do things agentically. Double-checking LLM-generated code is fraught enough one step at a time, but it's usually close enough that it can be course-corrected with light supervision. That calculus changes entirely when the automated version of the supervision fails catastrophically a non-trivial percent of the time.
- wongarsu 7mo ago
  It's meant as a "yes"/"instead, do ..." question. When it presents you with the multiple choice UI at that point it should be the version where you either confirm (with/without auto edit, with/without context clear) or you give feedback on the plan. Just telling it no doesn't give the model anything actionable to do
  - keerthiko 7mo ago
    It can terminate the current plan where it's at until given a new prompt, or move to the next item on its todo list /shrug
- Lerc 7mo ago
  But I think if you sit down and really consider the implications of it and what yes or not actually means in reality, or even a overabundance of caution causing extraneous information to confuse the issue enough that you don't realise that this sentence is completely irrelevant to the problem at hand and could be inserted by a third party, yet the AI is the only one to see it. I agree.
- efitz 7mo ago
  To an LLM, answering “no” and changing the mode of the chat window are discrete events that are not necessarily related. Many coding agents interpret mode changes as expressions of intent; Cline, for example, does not even ask, the only approval workflow is changing from plan mode to execute mode. So while this is definitely both humorous and annoying, and potentially hazardous based on your workflow, I don’t completely blame the agent because from its point of view, the user gave it mixed signals.
  - hananova 7mo ago
    Yeah but why should I care? That’s not how consent works. A million yesses and a single no still evaluates to a hard no.
    - thepasch 7mo ago
      The point is that if the harness’ workflow gives contradictory and confusing instructions to the model, it’s a harness issue, not necessarily a model issue.
      - daveguy 7mo ago
        First it was a model issue, then it was a prompting issue, then it was a context issue, then it was an agent issue, now it's a harness issue. AI advocates keep accusing AI skeptics of moving goalposts. But it seems like every 3-6 months another goalpost is added.
        thepasch 7mo ago
        Your comment doesn’t make as strong of a point as you think it does; it might make the opposite point. Because, yes, first, it was a model issue, and then more advanced models started appearing and prompting them correctly became more important. Then models learned through RLHF to deal with vague prompting better, and context management became more important. Then models became better (though not great) at inherent context recollection and attention distribution, so now, you need to be careful what instructions a model receives and at what points because it’s literally better at following them. It’s not so much that the goalposts are being moved, it’s that they’re literally being, like, *cleared*. This isn’t a tech that’s already fully explored and we just need to make it good now, it’s effectively an entirely new field of computing. When ChatGPT came out years ago no one would have DREAMT of an LLM ever autonomously using CLI tools to write entire projects worth of code off of a single text prompt. We’d only just figured out how to turn them into proper chatbots. The point is that we have no idea where the ceiling is right now, so demanding well-defined goalposts is like saying we need to have a full geological map of Mars before we can set foot on it, when part of the point of going to Mars is to find out about that. As a side point, the agent is the harness; or, rather, an agent is a model called on a loop, and the harness is where that loop lives (and where it can be influenced/stopped). So what I can say about most - not all, but most, including you, seemingly - AI skeptics is that they tend to not actually be particularly up-to-date and/or engaged with how these systems actually work and how capable they actually are at this point. Which is not supposed to be a dig or shade, because I’m pretty sure we’ve never had any tech move this fast before. But the general public is so woefully underinformed about this. I’ve recently had someone tell me in awe about how ChatGPT was able to read their handwritten note and solve a few math equations.
- Joker_vD 7mo ago
  Not when you're talking with humans, not really. Which is one of the reasons I got into computing in the first place, dangit!
reconnecting 7mo ago
There is the link to the full session below. https://news.ycombinator.com/item?id=47357042#47357656 https://news.ycombinator.com/item?id=47357042#47357656
- bensyverson 7mo ago
  Do we know if thinking was on high effort? I've found it sometimes overthinks on high, so I tend to run on medium.
  - breton 7mo ago
    it was on "max"
BosunoB 7mo ago
The whole idea of just sending "no" to an LLM without additional context is kind of silly. It's smart enough to know that if you just didn't want it to proceed, you would just not respond to it. The fact that you responded to it tells it that it should do something, and so it looks for additional context (for the build mode change) to decide what to do.
- ForHackernews 7mo ago
  > It's smart enough to know that if you just didn't want it to proceed, you would just not respond to it. No it absolutely is not. It doesn't "know" anything when it's not responding to a prompt. It's not consciously sitting there waiting for you to reply.
  - BosunoB 7mo ago
    I didn't mean to imply that it was. But when you reply to it, if you just say "no" then it's aware that you could've just not responded, and that normally you would never respond to it unless you were asking for something more. It just doesn't make any sense to respond no in this situation, and so it confuses the LLM and so it looks for more context.
    - alpaca128 7mo ago
      > it's aware that you could've just not responded It's not aware of anything and doesn't know that a world outside the context window exists.
      - BosunoB 7mo ago
        No, it has knowledge of what it is and how it is used. I'm guessing you and the other guy are taking issue with the words "aware of" when I'm just saying it has knowledge of these things. Awareness doesn't have to imply a continual conscious state.
        7mo ago
stefan_ 7mo ago
This is probably just OpenCode nonsense. After prompting in "plan mode", the models will frequently ask you if you want to implement that, then if you don't switch into "build mode", it will waste five minutes trying but failing to "build" with equally nonsense behavior. Honestly OpenCode is such a disappointment. Like their bewildering choice to enable random formatters by default; you couldn't come up with a better plan to sabotage models and send them into "I need to figure out what my change is to commit" brainrot loops.
Waterluvian 7mo ago
If we’re in a shoot first and ask questions later kind of mood and we’re just mowing down zombies (the slow kind) and for whatever reason you point to one and ask if you should shoot it… and I say no… you don’t shoot it!
clbrmbr 7mo ago
This. The models struggle with differentiating tool responses from user messages. The trouble is these are language models with only a veneer of RL that gives them awareness of the user turn. They have very little pretraining on this idea of being in the head of a computer with different people and systems talking to you at once. —- there’s more that needs to go on than eliciting a pre-learned persona.
adyavanapalli 7mo ago
It definitely _could be_ an agent harness issue. For example, this is the logic opencode uses: 1. Agent is "plan" -> inject PROMPT_PLAN 2. Agent is "build" AND a previous assistant message was from "plan" -> inject BUILD_SWITCH 3. Otherwise -> nothing injected And these are the prompts used for the above. PROMPT_PLAN: https://github.com/anomalyco/opencode/blob/dev/packages/opencode/src/session/prompt/plan.txt https://github.com/anomalyco/opencode/blob/dev/packages/open... BUILD_SWITCH: https://github.com/anomalyco/opencode/blob/dev/packages/opencode/src/session/prompt/build-switch.txt https://github.com/anomalyco/opencode/blob/dev/packages/open... Specifically, it has the following lines: > You are permitted to make file changes, run shell commands, and utilize your arsenal of tools as needed. I feel like that's probably enough to cause an LLM to change it's behavior.

by sgillen 7mo ago

christoff12 7mo ago

Asking a yes/no question implies the ability to handle either choice.

not_kurt_godel 7mo ago
This is a perfect example of why I'm not in any rush to do things agentically. Double-checking LLM-generated code is fraught enough one step at a time, but it's usually close enough that it can be course-corrected with light supervision. That calculus changes entirely when the automated version of the supervision fails catastrophically a non-trivial percent of the time.
wongarsu 7mo ago
It's meant as a "yes"/"instead, do ..." question. When it presents you with the multiple choice UI at that point it should be the version where you either confirm (with/without auto edit, with/without context clear) or you give feedback on the plan. Just telling it no doesn't give the model anything actionable to do
- keerthiko 7mo ago
  It can terminate the current plan where it's at until given a new prompt, or move to the next item on its todo list /shrug
Lerc 7mo ago
But I think if you sit down and really consider the implications of it and what yes or not actually means in reality, or even a overabundance of caution causing extraneous information to confuse the issue enough that you don't realise that this sentence is completely irrelevant to the problem at hand and could be inserted by a third party, yet the AI is the only one to see it. I agree.
efitz 7mo ago
To an LLM, answering “no” and changing the mode of the chat window are discrete events that are not necessarily related. Many coding agents interpret mode changes as expressions of intent; Cline, for example, does not even ask, the only approval workflow is changing from plan mode to execute mode. So while this is definitely both humorous and annoying, and potentially hazardous based on your workflow, I don’t completely blame the agent because from its point of view, the user gave it mixed signals.
- hananova 7mo ago
  Yeah but why should I care? That’s not how consent works. A million yesses and a single no still evaluates to a hard no.
  - thepasch 7mo ago
    The point is that if the harness’ workflow gives contradictory and confusing instructions to the model, it’s a harness issue, not necessarily a model issue.
    - daveguy 7mo ago
      First it was a model issue, then it was a prompting issue, then it was a context issue, then it was an agent issue, now it's a harness issue. AI advocates keep accusing AI skeptics of moving goalposts. But it seems like every 3-6 months another goalpost is added.
      - thepasch 7mo ago
        Your comment doesn’t make as strong of a point as you think it does; it might make the opposite point. Because, yes, first, it was a model issue, and then more advanced models started appearing and prompting them correctly became more important. Then models learned through RLHF to deal with vague prompting better, and context management became more important. Then models became better (though not great) at inherent context recollection and attention distribution, so now, you need to be careful what instructions a model receives and at what points because it’s literally better at following them. It’s not so much that the goalposts are being moved, it’s that they’re literally being, like, *cleared*. This isn’t a tech that’s already fully explored and we just need to make it good now, it’s effectively an entirely new field of computing. When ChatGPT came out years ago no one would have DREAMT of an LLM ever autonomously using CLI tools to write entire projects worth of code off of a single text prompt. We’d only just figured out how to turn them into proper chatbots. The point is that we have no idea where the ceiling is right now, so demanding well-defined goalposts is like saying we need to have a full geological map of Mars before we can set foot on it, when part of the point of going to Mars is to find out about that. As a side point, the agent is the harness; or, rather, an agent is a model called on a loop, and the harness is where that loop lives (and where it can be influenced/stopped). So what I can say about most - not all, but most, including you, seemingly - AI skeptics is that they tend to not actually be particularly up-to-date and/or engaged with how these systems actually work and how capable they actually are at this point. Which is not supposed to be a dig or shade, because I’m pretty sure we’ve never had any tech move this fast before. But the general public is so woefully underinformed about this. I’ve recently had someone tell me in awe about how ChatGPT was able to read their handwritten note and solve a few math equations.
Joker_vD 7mo ago
Not when you're talking with humans, not really. Which is one of the reasons I got into computing in the first place, dangit!

reconnecting 7mo ago

There is the link to the full session below. https://news.ycombinator.com/item?id=47357042#47357656 https://news.ycombinator.com/item?id=47357042#47357656

bensyverson 7mo ago
Do we know if thinking was on high effort? I've found it sometimes overthinks on high, so I tend to run on medium.
- breton 7mo ago
  it was on "max"

BosunoB 7mo ago

The whole idea of just sending "no" to an LLM without additional context is kind of silly. It's smart enough to know that if you just didn't want it to proceed, you would just not respond to it. The fact that you responded to it tells it that it should do something, and so it looks for additional context (for the build mode change) to decide what to do.

ForHackernews 7mo ago
> It's smart enough to know that if you just didn't want it to proceed, you would just not respond to it. No it absolutely is not. It doesn't "know" anything when it's not responding to a prompt. It's not consciously sitting there waiting for you to reply.
- BosunoB 7mo ago
  I didn't mean to imply that it was. But when you reply to it, if you just say "no" then it's aware that you could've just not responded, and that normally you would never respond to it unless you were asking for something more. It just doesn't make any sense to respond no in this situation, and so it confuses the LLM and so it looks for more context.
  - alpaca128 7mo ago
    > it's aware that you could've just not responded It's not aware of anything and doesn't know that a world outside the context window exists.
    - BosunoB 7mo ago
      No, it has knowledge of what it is and how it is used. I'm guessing you and the other guy are taking issue with the words "aware of" when I'm just saying it has knowledge of these things. Awareness doesn't have to imply a continual conscious state.
      - 7mo ago

stefan_ 7mo ago

This is probably just OpenCode nonsense. After prompting in "plan mode", the models will frequently ask you if you want to implement that, then if you don't switch into "build mode", it will waste five minutes trying but failing to "build" with equally nonsense behavior. Honestly OpenCode is such a disappointment. Like their bewildering choice to enable random formatters by default; you couldn't come up with a better plan to sabotage models and send them into "I need to figure out what my change is to commit" brainrot loops.

Waterluvian 7mo ago

If we’re in a shoot first and ask questions later kind of mood and we’re just mowing down zombies (the slow kind) and for whatever reason you point to one and ask if you should shoot it… and I say no… you don’t shoot it!

clbrmbr 7mo ago

This. The models struggle with differentiating tool responses from user messages. The trouble is these are language models with only a veneer of RL that gives them awareness of the user turn. They have very little pretraining on this idea of being in the head of a computer with different people and systems talking to you at once. —- there’s more that needs to go on than eliciting a pre-learned persona.

adyavanapalli 7mo ago

It definitely _could be_ an agent harness issue. For example, this is the logic opencode uses: 1. Agent is "plan" -> inject PROMPT_PLAN 2. Agent is "build" AND a previous assistant message was from "plan" -> inject BUILD_SWITCH 3. Otherwise -> nothing injected And these are the prompts used for the above. PROMPT_PLAN: https://github.com/anomalyco/opencode/blob/dev/packages/opencode/src/session/prompt/plan.txt https://github.com/anomalyco/opencode/blob/dev/packages/open... BUILD_SWITCH: https://github.com/anomalyco/opencode/blob/dev/packages/opencode/src/session/prompt/build-switch.txt https://github.com/anomalyco/opencode/blob/dev/packages/open... Specifically, it has the following lines: > You are permitted to make file changes, run shell commands, and utilize your arsenal of tools as needed. I feel like that's probably enough to cause an LLM to change it's behavior.