3 ms·

What is /goal?

swader999 3mo ago
On Claude if you start with that, it won't stop until it achieves or exhausts your prompt. It feels like "here's your mission, go do it". I use it a few times a week.
- jascha_eng 3mo ago
  Is this useful? I feel like the problem is usually not that the model isn't capable of achieving what I give it, but the way it does it. Especially if originally I didn't 100% know how I would do it myself the model often takes weird paths through the code base, takes shortcuts that end up in weird feature interactions or pulls in a dependency without weighting if it could've been done without that. I haven't really found a good way to solve this other than: 1. Produce an initial PR fulfilling all the requirements I knew at the start 2. Chat with the model about any weird snippets I notice and talk through alternatives 3. Simplify anything that I think is overengineered or plain unncessary Sometimes I restart all over with more precise requirements but then it sometimes makes different mistakes/takes different shortcuts. In practice the earlier I review the better the end result imo, so /goal seems very unproductive to me?
  - swader999 3mo ago
    It's useful for things where it just needs to get through to completion. Long running tasks. I walk away and expect it to be done without pausing for input.
    - jascha_eng 3mo ago
      Can you give an example? And more curious about what you do with the resulting code afterwards I imagine its gonna be a big chunk then?
      - madeofpalk 3mo ago
        Adding a new lint rule that a lot of code violates.
        jascha_eng 3mo ago
        And that doesn't work with a simple prompt?
        mh- 3mo ago
        It's just a way of defining what "done" means, and passing off evaluation of whether "it's done" is true to a separate context and/or model. If false, it prompts the original context accordingly.
      - hungryhobbit 3mo ago
        I feel like "do X untill tests are green" is sort of the prototypical /goal case.
  - ubermon 3mo ago
    For some frontier models like Fable 5 it doesn't matter, but for models less trained on long horizon tasks it very useful.
- brap 3mo ago
  > won't stop until it achieves or exhausts your prompt This is overselling it. In all implementations of this that I’ve seen, a more correct phrasing is “won’t stop until it pinky-promises it achieved your goal”. It usually relies on something like a hook that refuses to let the agent stop unless it includes a “promise sentinel”, which is basically the model having to include “I swear I’m done” in the response (usually as a Markdown comment so the user doesn’t see it). If that sentinel isn’t in the response, the hook rejects the completion and hands it back to the agent (sometimes it also restates the goal to keep it focused). It’s useful for sure, but it fails many times for obvious reasons. More robust implementations use the hook to spawn another agent that verifies the goal is achieved, but that too isn’t prefect and sometimes performs worse. Of course if you have a problem that can be deterministically verified, you could have the hook run this verification instead, but this is usually not the case (and even when it is, agents will often cheat the verification e.g. by deleting a test file).
  - hungryhobbit 3mo ago
    The key thing is that normally Claude grades Claude's work, so you do get a"pinky promise, as all of Claude's training bias tells it to plunge ahead (even if that means "fixing" a test by commenting it out). With /goal, a different Claude instance evaluates the goal. This is the robust version you described, and such an "adversarial Claude's" approach, with or without/goal, is exactly how you avoid pinky promises (from one Claude).
- 9dev 3mo ago
  I'm way too afraid of a "paperclip problem"-style target overfitting to use it really; how could I even describe a goal so well it doesn't sacrifice other things that are important to me? Like, if my goal is to make an endpoint faster, will it create an over-engineered mess out of the clearly readable code I have to reach that goal?
  - mh- 3mo ago
    /goal open a draft PR that makes <endpoint> ~200ms faster while still passing tests, and adhering to nearby style+pattern It's obviously not perfect, but it's not like you're going to YOLO it into prod without evaluating it on whatever you care about. If you want to be able to use it like that, you need to find a way to encode (most of) your concerns into something that can be programmatically verified.
    - 9dev 3mo ago
      That kind of emphasises the problem I was referring to; if I pick 200ms, will it stop before doing a proper optimisation because that would make it faster than that? Will it do something stupid to justify pushing from 195ms to >=200ms?
      - mh- 3mo ago
        It might! In my prompt, I'm supposing you have some priors here about what's possible. Try saying something like "materially faster". But you need to also make sure it knows how to measure that endpoint's performance on its own, otherwise I wouldn't expect this to work well at all.
ryan_n 3mo ago
The llm runs in a loop until it meets a condition (the goal).
ai_fry_ur_brain 3mo ago
[dead]
deepdarkforest 3mo ago
An agent is an llm running on a loop until it decides it should stop. /goal is a gimmick where you run a "parent" agent on top that runs the agent on a loop until the it decides to stop, just prompting it "nope, not done yet, continue".
- krackers 3mo ago
  isn't this what they called a "ralph loop"?
aetch 3mo ago
It loops until it thinks it has finished the goal
lambda 3mo ago
The article describes it. Both Codex and Claude Code have it, but they work slightly differently. Claude Code uses Haiku to read through the transcript and decide if the goal has been completed. If not, Haiku injects a prompt back to the main model to indicate what still needs to be done. In Codex, instead it's a tool available to the main model, plus some part of the surrounding harness that will re-prompt it if the tool calls haven't yet indicated that the goal is complete. The issue that they are trying to solve is that sometimes models will stop before they have actually fully completed whatever task they were given; attention isn't perfect, and someitmes they'll complete part of it but not the whole task. Rather than making the user come back and re-prompt to keep going, they add a way to automatically do a bit more nudging to try to get the model to finish the task.
- tpicks 3mo ago
  > Claude Code uses Haiku to read through the transcript and decide if the goal has been completed. feels kinda odd to use a less capable model to determine if the goal is fully complete. Especially if the user is expecting /goal to thoroughly complete the task. A less capable model would be more likely to misclassify `isComplete?`
  - mdp2021 3mo ago
    > odd There is nothing odd in the basic principle of Economics of using a "good enough" tool, optimal in other respects (e.g. cheap/er). (You do not hire Nobel Prize winners as receptionists.) The issue remains, whether the compromise to be adopted actually is "good enough".

What is /goal?

On Claude if you start with that, it won't stop until it achieves or exhausts your prompt. It feels like "here's your mission, go do it". I use it a few times a week.

jascha_eng 3mo ago
Is this useful? I feel like the problem is usually not that the model isn't capable of achieving what I give it, but the way it does it. Especially if originally I didn't 100% know how I would do it myself the model often takes weird paths through the code base, takes shortcuts that end up in weird feature interactions or pulls in a dependency without weighting if it could've been done without that. I haven't really found a good way to solve this other than: 1. Produce an initial PR fulfilling all the requirements I knew at the start 2. Chat with the model about any weird snippets I notice and talk through alternatives 3. Simplify anything that I think is overengineered or plain unncessary Sometimes I restart all over with more precise requirements but then it sometimes makes different mistakes/takes different shortcuts. In practice the earlier I review the better the end result imo, so /goal seems very unproductive to me?
- swader999 3mo ago
  It's useful for things where it just needs to get through to completion. Long running tasks. I walk away and expect it to be done without pausing for input.
  - jascha_eng 3mo ago
    Can you give an example? And more curious about what you do with the resulting code afterwards I imagine its gonna be a big chunk then?
    - madeofpalk 3mo ago
      Adding a new lint rule that a lot of code violates.
      - jascha_eng 3mo ago
        And that doesn't work with a simple prompt?
        mh- 3mo ago
        It's just a way of defining what "done" means, and passing off evaluation of whether "it's done" is true to a separate context and/or model. If false, it prompts the original context accordingly.
    - hungryhobbit 3mo ago
      I feel like "do X untill tests are green" is sort of the prototypical /goal case.
- ubermon 3mo ago
  For some frontier models like Fable 5 it doesn't matter, but for models less trained on long horizon tasks it very useful.
brap 3mo ago
> won't stop until it achieves or exhausts your prompt This is overselling it. In all implementations of this that I’ve seen, a more correct phrasing is “won’t stop until it pinky-promises it achieved your goal”. It usually relies on something like a hook that refuses to let the agent stop unless it includes a “promise sentinel”, which is basically the model having to include “I swear I’m done” in the response (usually as a Markdown comment so the user doesn’t see it). If that sentinel isn’t in the response, the hook rejects the completion and hands it back to the agent (sometimes it also restates the goal to keep it focused). It’s useful for sure, but it fails many times for obvious reasons. More robust implementations use the hook to spawn another agent that verifies the goal is achieved, but that too isn’t prefect and sometimes performs worse. Of course if you have a problem that can be deterministically verified, you could have the hook run this verification instead, but this is usually not the case (and even when it is, agents will often cheat the verification e.g. by deleting a test file).
- hungryhobbit 3mo ago
  The key thing is that normally Claude grades Claude's work, so you do get a"pinky promise, as all of Claude's training bias tells it to plunge ahead (even if that means "fixing" a test by commenting it out). With /goal, a different Claude instance evaluates the goal. This is the robust version you described, and such an "adversarial Claude's" approach, with or without/goal, is exactly how you avoid pinky promises (from one Claude).
9dev 3mo ago
I'm way too afraid of a "paperclip problem"-style target overfitting to use it really; how could I even describe a goal so well it doesn't sacrifice other things that are important to me? Like, if my goal is to make an endpoint faster, will it create an over-engineered mess out of the clearly readable code I have to reach that goal?
- mh- 3mo ago
  /goal open a draft PR that makes <endpoint> ~200ms faster while still passing tests, and adhering to nearby style+pattern It's obviously not perfect, but it's not like you're going to YOLO it into prod without evaluating it on whatever you care about. If you want to be able to use it like that, you need to find a way to encode (most of) your concerns into something that can be programmatically verified.
  - 9dev 3mo ago
    That kind of emphasises the problem I was referring to; if I pick 200ms, will it stop before doing a proper optimisation because that would make it faster than that? Will it do something stupid to justify pushing from 195ms to >=200ms?
    - mh- 3mo ago
      It might! In my prompt, I'm supposing you have some priors here about what's possible. Try saying something like "materially faster". But you need to also make sure it knows how to measure that endpoint's performance on its own, otherwise I wouldn't expect this to work well at all.

ryan_n 3mo ago

The llm runs in a loop until it meets a condition (the goal).

ai_fry_ur_brain 3mo ago

[dead]

deepdarkforest 3mo ago

An agent is an llm running on a loop until it decides it should stop. /goal is a gimmick where you run a "parent" agent on top that runs the agent on a loop until the it decides to stop, just prompting it "nope, not done yet, continue".

krackers 3mo ago
isn't this what they called a "ralph loop"?

aetch 3mo ago

It loops until it thinks it has finished the goal

lambda 3mo ago

The article describes it. Both Codex and Claude Code have it, but they work slightly differently. Claude Code uses Haiku to read through the transcript and decide if the goal has been completed. If not, Haiku injects a prompt back to the main model to indicate what still needs to be done. In Codex, instead it's a tool available to the main model, plus some part of the surrounding harness that will re-prompt it if the tool calls haven't yet indicated that the goal is complete. The issue that they are trying to solve is that sometimes models will stop before they have actually fully completed whatever task they were given; attention isn't perfect, and someitmes they'll complete part of it but not the whole task. Rather than making the user come back and re-prompt to keep going, they add a way to automatically do a bit more nudging to try to get the model to finish the task.

tpicks 3mo ago
> Claude Code uses Haiku to read through the transcript and decide if the goal has been completed. feels kinda odd to use a less capable model to determine if the goal is fully complete. Especially if the user is expecting /goal to thoroughly complete the task. A less capable model would be more likely to misclassify `isComplete?`
- mdp2021 3mo ago
  > odd There is nothing odd in the basic principle of Economics of using a "good enough" tool, optimal in other respects (e.g. cheap/er). (You do not hire Nobel Prize winners as receptionists.) The issue remains, whether the compromise to be adopted actually is "good enough".