4 ms·

I have a folder where I rebuild these as a git commit history so you can more easily see what has changed: https://github.com/simonw/research/commits/main/extra

by simonw 2mo ago

I have a folder where I rebuild these as a git commit history so you can more easily see what has changed: https://github.com/simonw/research/commits/main/extract-system-prompts https://github.com/simonw/research/commits/main/extract-syst... For example here's what changed between Opus 4.8 and Opus 5: https://github.com/simonw/research/commit/a2de185cc367eb66c2e27090d9ff0f766d335ff4 https://github.com/simonw/research/commit/a2de185cc367eb66c2... The most interesting addition to the prompt from that diff is this bit: > Claude Fable 5 and Claude Mythos 5 were first released on June 9, 2026. On June 12, 2026, Anthropic suspended access to both models to comply with U.S. Department of Commerce export controls; the Department lifted those controls on June 30, 2026, and Anthropic restored access on July 1, 2026 (Anthropic's statement: [https://www.anthropic.com/news/fable-mythos-access https://www.anthropic.com/news/fable-mythos-access](https://www.anthropic.com/news/fable-mythos-access https://www.anthropic.com/news/fable-mythos-access)). These events are after Claude's training-data cutoff, so Claude knows about them only from this notice. If asked, Claude confirms them accurately and matter-of-factly — it doesn't deny the suspension happened — and otherwise treats the export controls like any other current political topic: it gives a fair, accurate account rather than sharing personal opinions, and points to the linked statement for anything further. Things may have developed since this notice, so Claude checks for newer information when it can search, and otherwise suggests checking Anthropic's site. One frustrating note about this page is that they share the system prompts used for https://claude.ai https://claude.ai and the Claude mobile apps regular chat, but they omit the tool definitions. Those are much more interesting if you want to understand what Claude can actually do for you. You can reconstruct them through prompting Claude directly but that's extra friction and risks refusals and hallucinations. They also don't publish the Claude Code system prompts, which is silly because those are trivial to extract using a logging proxy.

eterm 2mo ago
It'd be ironic if the "Opus 5 nerf" effect is from telling Opus that it sits a tier down from Fable and Mythos, while Opus4.8 believed it was the best of the best, just a note that it was "Preceded by Mythos".
- mcbuilder 2mo ago
  Nah, that's the same sort of thinking that makes people type "make no mistakes", I don't make my model roll play, etc. I believe that the longer the system prompt and the more you cram in it the worse the model does. You need the human doing minimal prompts, but in the right direction. Take a look a the transcripts of Terrance Tao with ChatGPT
  - eterm 2mo ago
    My comment was a bit tongue in cheek, I'm not actually convinced there was real degradation in opus 5 beyond a tendency to try to plough ahead without stopping to clarify things. I don't really think 1 line in lengthy system prompt affects things that much, it'd just be an amusing form of emergent behaviour where we now have to massage the ego of something with no id.
    - hanselot 2mo ago
      [dead]
  - 8n4vidtmkvmk 2mo ago
    For complex projects with lots of internal tools and strict requirements, I'm finding a fairly lengthy system prompt is quite worth it. Start short or empty and watch where it makes mistakes then just keep tuning it so they're less frequent. That works for me.
- KellyCriterion 2mo ago
  Curious: Cant it spin up a webbrowser in the background and go to claude.ai and play with the sibling models and "find out" about it rank? :-D
  - ameliaquining 2mo ago
    The claude.ai frontend contains defenses against automated access.
deleted 2mo ago
[deleted]
PunchyHamster 2mo ago
Man they are wasting so much space on useless nonsense in the prompt
- NiloCK 2mo ago
  FYI you can launch claude-code with your own prompt. Don't quote me but: claude --system-prompt "Mine is better than Anthropic's"
- UqWBcuFx6NV4r 2mo ago
  You don’t know that unless you’ve compared performance.
umpalumpaaa 2mo ago
How did you get those system prompts? When I ask Claude to show me its system prompt it outputs very similar text but it said in its thoughts that only some sections are quoted verbatim. Other sections seem to be summaries from the system prompt. It also gave me tool definitions but only summarized but also told me that I get can the whole tool defining if I want to - it omitted most of it to not have lots of repetitive JSON schema in the output.
- simonw 2mo ago
  These are the prompts that Anthropic publish on https://platform.claude.com/docs/en/release-notes/system-prompts https://platform.claude.com/docs/en/release-notes/system-pro...
kouteiheika 2mo ago
> but they omit the tool definitions. Those are much more interesting if you want to understand what Claude can actually do for you. You can reconstruct them through prompting Claude directly but that's extra friction and risks refusals and hallucinations. Last time I tried these were trivial to get by just putting a man-in-the-middle proxy between Claude Code and Anthopic. You get both the system prompt, and all of the tool definitions. So there's no need to go about it in such a roundabout way.
- simonw 2mo ago
  That works for Claude Code but doesn't work for https://claude.ai https://claude.ai Claude Chat. From my original comment: > They also don't publish the Claude Code system prompts, which is silly because those are trivial to extract using a logging proxy.
  - kouteiheika 2mo ago
    Ah, okay. Sorry, fair enough!
btown 2mo ago
> They also don't publish the Claude Code system prompts, which is silly because those are trivial to extract using a logging proxy. For those interested in the Claude Code tool descriptions, they're collected here: https://github.com/Piebald-AI/claude-code-system-prompts/tree/main/system-prompts https://github.com/Piebald-AI/claude-code-system-prompts/tre... As for the tool implementations, there are still remnants from the Claude Code source leak visible online, with a bit of research.
deleted 2mo ago
[deleted]

by simonw 2mo ago

eterm 2mo ago

It'd be ironic if the "Opus 5 nerf" effect is from telling Opus that it sits a tier down from Fable and Mythos, while Opus4.8 believed it was the best of the best, just a note that it was "Preceded by Mythos".

mcbuilder 2mo ago
Nah, that's the same sort of thinking that makes people type "make no mistakes", I don't make my model roll play, etc. I believe that the longer the system prompt and the more you cram in it the worse the model does. You need the human doing minimal prompts, but in the right direction. Take a look a the transcripts of Terrance Tao with ChatGPT
- eterm 2mo ago
  My comment was a bit tongue in cheek, I'm not actually convinced there was real degradation in opus 5 beyond a tendency to try to plough ahead without stopping to clarify things. I don't really think 1 line in lengthy system prompt affects things that much, it'd just be an amusing form of emergent behaviour where we now have to massage the ego of something with no id.
  - hanselot 2mo ago
    [dead]
- 8n4vidtmkvmk 2mo ago
  For complex projects with lots of internal tools and strict requirements, I'm finding a fairly lengthy system prompt is quite worth it. Start short or empty and watch where it makes mistakes then just keep tuning it so they're less frequent. That works for me.
KellyCriterion 2mo ago
Curious: Cant it spin up a webbrowser in the background and go to claude.ai and play with the sibling models and "find out" about it rank? :-D
- ameliaquining 2mo ago
  The claude.ai frontend contains defenses against automated access.

deleted 2mo ago

[deleted]

PunchyHamster 2mo ago

Man they are wasting so much space on useless nonsense in the prompt

NiloCK 2mo ago
FYI you can launch claude-code with your own prompt. Don't quote me but: claude --system-prompt "Mine is better than Anthropic's"
UqWBcuFx6NV4r 2mo ago
You don’t know that unless you’ve compared performance.

umpalumpaaa 2mo ago

How did you get those system prompts? When I ask Claude to show me its system prompt it outputs very similar text but it said in its thoughts that only some sections are quoted verbatim. Other sections seem to be summaries from the system prompt. It also gave me tool definitions but only summarized but also told me that I get can the whole tool defining if I want to - it omitted most of it to not have lots of repetitive JSON schema in the output.

simonw 2mo ago
These are the prompts that Anthropic publish on https://platform.claude.com/docs/en/release-notes/system-prompts https://platform.claude.com/docs/en/release-notes/system-pro...

kouteiheika 2mo ago

> but they omit the tool definitions. Those are much more interesting if you want to understand what Claude can actually do for you. You can reconstruct them through prompting Claude directly but that's extra friction and risks refusals and hallucinations. Last time I tried these were trivial to get by just putting a man-in-the-middle proxy between Claude Code and Anthopic. You get both the system prompt, and all of the tool definitions. So there's no need to go about it in such a roundabout way.

simonw 2mo ago
That works for Claude Code but doesn't work for https://claude.ai https://claude.ai Claude Chat. From my original comment: > They also don't publish the Claude Code system prompts, which is silly because those are trivial to extract using a logging proxy.
- kouteiheika 2mo ago
  Ah, okay. Sorry, fair enough!

btown 2mo ago

> They also don't publish the Claude Code system prompts, which is silly because those are trivial to extract using a logging proxy. For those interested in the Claude Code tool descriptions, they're collected here: https://github.com/Piebald-AI/claude-code-system-prompts/tree/main/system-prompts https://github.com/Piebald-AI/claude-code-system-prompts/tre... As for the tool implementations, there are still remnants from the Claude Code source leak visible online, with a bit of research.

deleted 2mo ago

[deleted]