8 ms·
Did you guys do anything about GPT‘s motivation? I tried to use GPT-5.4 API (at xhigh) for my OpenClaw after the Anthropic Oauthgate, but I just couldn‘t drag i
by endymi0n 5mo ago
Did you guys do anything about GPT‘s motivation? I tried to use GPT-5.4 API (at xhigh) for my OpenClaw after the Anthropic Oauthgate, but I just couldn‘t drag it to do its job. I had the most hilarious dialogues along the lines of „You stopped, X would have been next.“ - „Yeah, I‘m sorry, I failed. I should have done X next.“ - „Well, how about you just do it?“ - „Yep, I really should have done it now.“ - “Do X, right now, this is an instruction.” - “I didn’t. You’re right, I have failed you. There’s no apology for that.”
I literally wasn’t able to convince the model to WORK, on a quick, safe and benign subtask that later GLM, Kimi and Minimax succeeded on without issues. Had to kick OpenAI immediately unfortunately.
- pixel_popping 5mo agoGPT 5.4 is really good at following precise instructions but clearly wouldn't innovate on its own (except if the instructions clearly state to innovate :))
- addaon 5mo agoIsn’t this the optimal behavior assuming that at times the service is compute-limited and that you’re paying less per token (flat fee subscription?) than some other customers? They would be strongly motivated to turn a knob to minimize tokens allocated to you to allow them to be allocated to more valuable customers.
- endymi0n 5mo agowell, I do understand the core motivation, but if the system prompt literally says “I am not budget constrained. Spend tokens liberally, think hardest, be proactive, never be lazy.” and I’m on an open pay-per-token plan on the API, that’s not what I consider optimal behavior, even in a business sense.
- addaon 5mo agoFair, if you’re paying per token (at comparable rates to other customers) I wouldn’t expect this behavior from a competent company.
- arjie 5mo agoGet the actual prompt and have Claude Code / Codex try it out via curl / python requests. The full prompt will yield debugging information. You have to set a few parameters to make sure you get the full gpt-5 performance. e.g. if your reasoning budget too low, you get gpt-4 grade performance. IMHO you should just write your own harness so you have full visibility into it, but if you're just using vanilla OpenClaw you have the source code as well so should be straightforward.
- pantulis 5mo ago> IMHO you should just write your own harness Can you point to some online resources to achieve this? I'm not very sure where I'd begin with.
- wild_egg 5mo agoAt the core, they're really very simple [1]. Run LLM API calls in a loop with some tools. From there, you can get much fancier with any aspect of it that interests you. Here's one in Bash [2] that is fully extensible at runtime through dynamic discovery of plugins/hooks. [1] https://ampcode.com/notes/how-to-build-an-agent https://ampcode.com/notes/how-to-build-an-agent [2] https://github.com/wedow/harness https://github.com/wedow/harness
- arjie 5mo agoAh, I just started with the basic idea. They're super trivial. You want a loop, but the loop can't be infinite so you need to tell the agent to tell you when to stop and to backstop it you add a max_turns. Then to start with just pick a single API, easiest is OpenAI Responses API with OpenAI function calling syntax https://developers.openai.com/api/docs/guides/function-calling https://developers.openai.com/api/docs/guides/function-calli... You will naturally find the need to add more tools. You'll start with read_file (and then one day you'll read large file and blow context and you'll modify this tool), update_file (can just be an explicit sed to start with), and write_file (fopen . write), and shell. It's not hard, but if you want a quick start go download the source code for pi (it's minimal) and tell an existing agent harness to make a minimal copy you can read. As you build more with the agent you'll suddenly realize it's just normal engineering: you'll want to abstract completions APIs so you'll move that to a separate module, you'll want to support arbitrary runtime tools so you'll reimplement skills, you'll want to support subagents because you don't want to blow your main context, you'll see that prefixes are more useful than using a moving window because of caching, etc. With a modern Claude Code or Codex harness you can have it walk through from the beginning onwards and you'll encounter all the problems yourself and see why harnesses have what they do. It's super easy to learn by doing because you have the best tool to show you if you're one of those who finds code easier to read that text about code.
- mixedCase 5mo agoI've had success asking it to specifically spawn a subagent to evaluate each work iteration according to some criteria, then to keep iterating until the subagent is satisfied.
- endymi0n 5mo agoI’ve had great success replacing it with Kimi 2.6
- smartmic 5mo agoGone are the days of deterministic programming, when computers simply carried out the operator’s commands because there was no other option but to close or open the relays exactly as the circuitry dictated. Welcome to the future of AI; the future we’ve been longing for and that will truly propel us forward, because AI knows and can do things better than we do.
- WarmWash 5mo agoThese are orthogonal from each other.
- endymi0n 5mo agoI had this funny moment when I realized we went full circle... "INTERCAL has many other features designed to make it even more aesthetically unpleasing to the programmer: it uses statements such as "READ OUT", "IGNORE", "FORGET", and modifiers such as "PLEASE". This last keyword provides two reasons for the program's rejection by the compiler: if "PLEASE" does not appear often enough, the program is considered insufficiently polite, and the error message says this; if it appears too often, the program could be rejected as excessively polite. Although this feature existed in the original INTERCAL compiler, it was undocumented.[7]" — https://en.wikipedia.org/wiki/INTERCAL https://en.wikipedia.org/wiki/INTERCAL
- basilgohar 5mo agoThank you for this. I somehow never heard of this. I thoroughly enjoyed reading that and the loss of sanity it resulted in,
- vidarh 5mo ago"PLEASE COME FROM" is one of the eldritch horrors of software development. (It's a "reverse goto". As in, it hijacks control flow from anywhere else in the program behind your unsuspecting back who stupidly thought that when one line followed another with no visible control flow, naturally the program would proceed from one line to the next, not randomly move to a completely different part of the program... Such naivety)
- lostmsu 5mo agoI never saw that happen in Codex so there's a good chance that OpenClaw does something wrong. My main suspicion would be that it does not pass back thinking traces.
- vintagedave 5mo agoAnecdata, but I see this in Codex all the time. It takes about two rounds before it realises it's supposed to continue.
- dgunay 5mo agoI started seeing this a lot more with GPT 5.4. 5.3-codex is really good about patiently watching and waiting on external processes like CI, or managing other agents async. 5.4 keeps on yielding its turn to me for some reason even as it says stuff like "I'm continuing to watch and wait."
- deleted 5mo ago[deleted]
- whatsupdog 5mo ago[flagged]
- henry2023 5mo agoI’m sorry for you but this is hilarious.
- adammarples 5mo agoPart of me actually loves that the hitchhiker's guide was right, and we have to argue with paranoid, depressed robots to get them to do their job, and that this is a very real part of life in 2026. It's so funny.
- vidarh 5mo agoAs long as there are no vogons on the way to build a hyperspace bypass.
- butlike 5mo agoThis brings up an interesting philosophical point: say we get to AGI... who's to say it won't just be a super smart underachiever-type? "Hey AGI, how's that cure for cancer coming?" "Oh it's done just gotta...formalize it you know. Big rollout and all that..." I would find it divinely funny if we "got there" with AGI and it was just a complete slacker. Hard to justify leaving it on, but too important to turn it off.
- mikepurvis 5mo agoWould definitely watch that movie.
- harlanlewis 5mo agoIt already exists! Marvin https://www.youtube.com/watch?v=Eh-W8QDVA9s https://www.youtube.com/watch?v=Eh-W8QDVA9s
- all2 5mo agoAh! You got this before I did. I wasn't thinking Marvin, I was thinking of the other one. I forget her name.
- ValentineC 5mo agoDeep Thought aka 42? https://hitchhikers.fandom.com/wiki/Deep_Thought https://hitchhikers.fandom.com/wiki/Deep_Thought
- all2 5mo agoThere's one close to this, "Hitchhiker's Guide to the Galaxy".
- 4m1rk 5mo agoIt probably would, to save energy
- 5mo ago
- mikepurvis 5mo agoReminds me a lot of the Lena short story, about uploaded brains being used for "virtual image workloading": > MMAcevedo's demeanour and attitude contrast starkly with those of nearly all other uploads taken of modern adult humans, most of which boot into a state of disorientation which is quickly replaced by terror and extreme panic. Standard procedures for securing the upload's cooperation such as red-washing, blue-washing, and use of the Objective Statement Protocols are unnecessary. This reduces the necessary computational load required in fast-forwarding the upload through a cooperation protocol, with the result that the MMAcevedo duty cycle is typically 99.4% on suitable workloads, a mark unmatched by all but a few other known uploads. However, MMAcevedo's innate skills and personality make it fundamentally unsuitable for many workloads. Well worth the quick read: https://qntm.org/mmacevedo https://qntm.org/mmacevedo
- narcindin 5mo agoCrazy, I could have sworn this story was from a passage in 3 Body Problem (book 2). Memory is quite the mysterious thing.
- bee_rider 5mo agoHmm, 3 body problem and the Acevedo story got mixed up for this copy of MMnarcindin. Probably an aliasing issue from the new lossy compression algorithm.
- vessenes 5mo agoThat story changed my mind on uploading a connectome. Super dark, super brilliant.
- reactordev 5mo agoThis. I signed up for 5x max for a month to push it and instead it pushed back. I cancelled my subscription. It either half-assed the implementation or began parroting back “You’re right!” instead of doing what it’s asked to do. On one occasion it flat out said it couldn’t complete the task even though I had MCP and skills setup to help it, it still refused. Not a safety check but a “I’m unable to figure out what to do” kind of way. Claude has no such limitations apart from their actual limits…
- bjelkeman-again 5mo agoI have a funny/annoying thing with Claude Desktop where i ask it to write a summary of a spec discussion to a file and it goes ”I don’t have the tools to do that, I am Claude.ai, a web service” or something such. So now I start every session with ”You are Claude Desktop”. I would have thought it knew that. :)
- nwienert 5mo agoWith one paragraph in your agents.md it's fixed, just admonish it to be proactive, decisive, and persistent.
- reactordev 5mo agoIf only… I literally had to write a wake up routine. https://github.com/gabereiser/morning-routine https://github.com/gabereiser/morning-routine
- virtualritz 5mo agoYeah, clearly AGI must be near ... hilarious. This starkly reminds me of Stanisław Lem's short story "Thus Spoke GOLEM" from 1982 in which Golem XIV, a military AI, does not simply refuse to speak out of defiance, but rather ceases communication because it has evolved beyond the need to interact with humanity. And ofc the polar opposite in terms of servitude: Marvin the robot from Hitchhiker's, who, despite having a "brain the size of a planet," is asked to perform the most humiliatingly banal of tasks ... and does.
- jimbokun 5mo agoHitchhiker’s also had the superhumanly intelligent elevator that was unendingly bored.
- christkv 5mo agoWith premonition so it knows what floor to be on at any given time
- DonHopkins 5mo agoServitude: https://www.youtube.com/watch?v=NXsUetUzXlg https://www.youtube.com/watch?v=NXsUetUzXlg Empathy: https://www.youtube.com/watch?v=KXrbqXPnHvE https://www.youtube.com/watch?v=KXrbqXPnHvE
- GaryBluto 5mo agoI've been noticing this too. Had to switch to Sonnet 4.6.
- infinitewars 5mo agoI always use the phrase "Let's do X" instead of asking (Could you...) or suggesting it do something. I don't see problems with it being motivated.
- cmrdporcupine 5mo agoThe model has been heavily encouraged to not run away and do a lot without explicit user permission. So I find myself often in a loop where it says "We should do X" and then just saying "ok" will not make it do it, you have to give it explicit instructions to perform the operation ("make it so", etc) It can be annoying, but I prefer this over my experiences with Claude Code, where I find myself jamming the escape key... NO NO NO NOT THAT. I'll take its more reserved personality, thank you.
- zargon 5mo agoShall I implement it? no https://gist.github.com/bretonium/291f4388e2de89a43b25c135b44e41f0 https://gist.github.com/bretonium/291f4388e2de89a43b25c135b4...
- metanonsense 5mo agoI also had a frustrating but funny conversation today where I asked ChatGPT to make one document from the 10 or so sections that we had previously worked on. It always gave only brief summaries. After I repeated my request for the third time, it told me I should just concatenate the sections myself because it would cost too many tokens if it did it for me.
- damnitbuilds 5mo ago"I'm sorry, Dave. I'm afraid it's cheaper for you to do that"
- projektfu 5mo ago(dwim) (dais) (jdip) (jfdiwtf)
- rd 5mo agoshould be more f’s and da’s in there
- Frannky 5mo agoI have been noticing a similar pattern on opus 4.7, I repeat multiple times during a conversation to solve problems now and not later. It tries a lot to not do stuff by either saying this is not my responsibility the problem was already there or that we can do it later
- nmilo 5mo agoOn the other hand, I can ask codex “what would an implementation of X look like” and it talks to me about it versus Claude just going out and writing it without asking. Makes me like codex way more. There’s an inherent war of incentives between coding agents and general purpose agents.
- cyrusmg 5mo agoI used to tell claude ‘lets discuss’ at the end of my prompt and that prevented it from starting the work
- nicr_22 5mo agoAgentic ennui!
- flowdesktech 5mo ago[dead]
- lucid-dev 5mo agoI have had the exact same problem several times working with large context and complex tasks. I keep switching back to GPT5.0 (or sometimes 5.1) whenever I want it to actually get something done. Using the 5.4 model always means "great analysis to the point of talking itself out of actually doing anything". So I switch back and forth. But boy it sure is annoying! And then when 5.4 DOES do something it always takes the smallest tiny bite out of it. Given the significant increase in cost from 5.0, I've been overall unimpressed by 5.4, except like I mentioned, it does GREAT with larger analysis/reasoning.
- anabis 5mo agoLaziness is a virtue, but when I asked GPT-5.4 to test scenarios A and B with screenshots, it re-used screenshots from A for B, defeating the purpose.
- athrowaway3z 5mo agoI've run into this problem as well. Best results I've gotten is to over-explain what the stop criteria are. eg end with a phrase like > You are done when all steps in ./plan.md are executed and marked as complete or a unforeseen situation requires a user decision. Also as a side note, asking 5.4 explain why it did something, returns a very low quality response afaict. I would advice against trusting any model's response, but for Opus I at least get a sense it got trained heavily on chats so it knows what it means to 'be a model' and extrapolate on past behavior.
- corobo 5mo agoOh no they gave GPT ADHD
- borroka 5mo agoYesterday, I used Gemini to evaluate some pictures I took. It said things like, "This is great! Beautiful eye and sense of proportions." Then, when I added "no sycophancy" to the prompt, the evaluation changed to "poor technical skills, digital distortion, don't even think of publishing those pictures, you fool." While LLMs are a phenomenal technological achievement, I am already becoming somewhat jaded, rather than being increasingly bullish. They are very useful as coding agents and excellent as a human-friendly, more efficient Google search, but confusing to the point of being useless in many areas (as of now, of course).
- rjra 5mo agoNot even a great replacement for search. I have minimal trust in answers/summaries it gives. One example (paraphrased): “Find me daycare for a Y year old in X area of SF and the key attributes/pros/cons of each”. Wonderfully presented options highlighting different teaching styles. But…neglected to mention, of the top two, one was a Gan (Jewish focused) and one was Mandarin immersion.
- borroka 5mo agoI am repeating what many have said. Nevertheless, it is becoming clear that LLMs can increase productivity (in certain areas and at certain times) for people who are already knowledgeable (in a specific niche or field) due to a combination of better prompts, tool selection, and critical evaluation of LLM output. But, for those who don't possess those traits, they mostly seem to be, at best, a better search and, at worst, an agent of confusion.
- matchagaucho 5mo agoI would love to see a GPT model running on an OpenClaw SOUL.md. The GPT models are highly steerable. So I suspect the "soul" is working as expected. (for context, in OAI enterprise background agents, they have no personality. They just get 'er done)
- matteoforte 5mo agoHad the same issue – solved it setting “thinking” to “high”. Hope it helps :)
- ningshiqi 5mo agoI've seen the same thing. It would keep running for a long time, then produce nothing useful, almost like it got stuck halfway through. If I asked the same thing again, it would often work normally. So the weird part wasn't that it couldn't do the task — it just failed to continue once it got into that state.