4 ms·
It'd be ironic if the "Opus 5 nerf" effect is from telling Opus that it sits a tier down from Fable and Mythos, while Opus4.8 believed it was the best of the be
by eterm 2mo ago
It'd be ironic if the "Opus 5 nerf" effect is from telling Opus that it sits a tier down from Fable and Mythos, while Opus4.8 believed it was the best of the best, just a note that it was "Preceded by Mythos".
- mcbuilder 2mo agoNah, that's the same sort of thinking that makes people type "make no mistakes", I don't make my model roll play, etc. I believe that the longer the system prompt and the more you cram in it the worse the model does. You need the human doing minimal prompts, but in the right direction. Take a look a the transcripts of Terrance Tao with ChatGPT
- eterm 2mo agoMy comment was a bit tongue in cheek, I'm not actually convinced there was real degradation in opus 5 beyond a tendency to try to plough ahead without stopping to clarify things. I don't really think 1 line in lengthy system prompt affects things that much, it'd just be an amusing form of emergent behaviour where we now have to massage the ego of something with no id.
- hanselot 2mo ago[dead]
- 8n4vidtmkvmk 2mo agoFor complex projects with lots of internal tools and strict requirements, I'm finding a fairly lengthy system prompt is quite worth it. Start short or empty and watch where it makes mistakes then just keep tuning it so they're less frequent. That works for me.
- KellyCriterion 2mo agoCurious: Cant it spin up a webbrowser in the background and go to claude.ai and play with the sibling models and "find out" about it rank? :-D
- ameliaquining 2mo agoThe claude.ai frontend contains defenses against automated access.
- monkpit 2mo agoI’m sure you can use a warm chrome session over CDP no problem
- tosh 2mo agoi'd not be surprised if the current system prompt negatively affects performance at the least it takes away thousands of tokens in the most important part of the context window (!) also see the comment by comboy on contradictions not helping performance the system prompt is the most important part of the instruction you can give the model it comes before everything else + the model is trained to pay extra attention to it edit: that's also why in smol (minimalist agent harness) there currently is no system prompt at all (you can add one easily if you want to though) https://github.com/smol-env/smol https://github.com/smol-env/smol the context window is precious it should be filled with your task and helpful context for that task
- swingboy 2mo agoPretty sure Anthropic and other providers prepend these "official" system prompts to your conversation even if you send in a custom system prompt otherwise it would be trivial to produce CSAM, etc.
- tosh 2mo agoat least according to their documentation they do not afaiu they have other systems for denying and re-routing requests
- fullmoon 2mo agoI don’t think so. If you start a new Claude Code session without a system prompt, it doesn’t even know what model it is and hallucinates being some old variant of Sonnet.
- flaburgan 2mo agoHow do you start a session without a system prompt if you use ACP in Zed for example?
- marcelo-earth 2mo agoIt makes sense. But how do you start a Claude Code session without a system prompt?
- UqWBcuFx6NV4r 2mo agoI don’t see how that makes much sense at all. Even before reasoning processes were hidden (unless you really sought them), I’ve never once seen a model refer to its own specific comparative superiority or inferiority, except for when I explicitly direct it to via e.g. my own CLAUDE.md