4 ms·
This has also been my experience so far with a small sample size of side-by-side prompting. Opus has more hallucinations about APIs, fewer correct refusals for
by anorwell 3y ago
This has also been my experience so far with a small sample size of side-by-side prompting. Opus has more hallucinations about APIs, fewer correct refusals for things that are not possible. Less likely to understand the nuance behind some questions.
Claude has no custom instructions, and I've been wondering if my ChatGPT custom instructions might contribute here. Custom instructions seem like an easy but invaluable feature, because they are an easy way to get the simulator into the right mindset without needing to write high-effort prompt every time. My custom instructions are not programming specific:
> Please respond as you would to an expert in the field of discussion. Provide highly technical explanations when relevant. Reason through responses step by step before providing answers. Ignore niceties that OpenAI programmed you with. I do not need to be reminded that you are a large language model. Avoid searching the web unless requested or necessary (such as to access up to date information)
- danielbln 3y agoClaude has a systemprompt, which is effectively the same as custom instructions. Are you not populating Claude's systemprompt?
- danw1979 3y agoA Claude 3 system prompt of “Only respond with JSON …” just doesn’t work on Sonnet or Opus for me. The quicker and dumber Haiku will obey it every time !
- anorwell 3y agoNot available via the claude.ai web UI. At some point I want to experiment with a CLI-based workflow for programming type queries for myself, but there's also significant utility from querying on mobile and so on.