3 ms·

A few miscellaneous observations about Claude's new beta function calling / structured data support (https://docs.anthropic.com/claude/docs/tool-use https://doc

A few miscellaneous observations about Claude's new beta function calling / structured data support (https://docs.anthropic.com/claude/docs/tool-use https://docs.anthropic.com/claude/docs/tool-use) that I encountered while making this notebook: 1. It is not nearly as good as ChatGPT's function calling conformance, hence why the system prompt engineering is more aggressive than usual. 2. Claude doesn't seem to handle nesting schema well: otherwise I would have allowed "generating X cards in a cycle" as a feature. The documentation does state that it can't handle "deeply nested" data but a single list is not deeply nested. 3. The documentation mentioned that Claude can do a chain-of-thoughts with tools enabled: in my testing this reduces quality drastically. In Opus, which does it by default, it's a waste of extremely expensive output tokens and it has a tendency to ignore fields. 4. Haiku and Sonnet have different vibes but similar subjective quality, in this case Sonnet being more "correct" and Haiku being more "fun", which is surprising.

BoorishBears 3y ago
You should look at the consumer app system prompt: https://twitter.com/amandaaskell/status/1765207842993434880 https://twitter.com/amandaaskell/status/1765207842993434880 I've had way more success taking it and forming it to my tasks than stuffing in a bunch of tokens about how to do things. In one case I was able to go from ~6k in input tokens to ~3k because I no longer had to provide a mountain of examples and instructions for corner cases
- namanyayg 3y ago
  Can you elaborate on how does the claude system prompt help in creating prompts for different tasks?
  - BoorishBears 3y ago
    Anytime I see a prompt from one of these companies, I assume it matches the style of instructions the model encountered during pre-training. And the kinds of instruction formats that were encountered during pre-training end up informing what style of instruction the model is best at following. An extreme example would be prompt templates that a "raw" instruct-tuned LLM follows: the model will technically work with a suboptimal format, but you get much better performance if you follow the prompt template the model was trained/fine-tuned on. - It's not a guarantee a given prompting style was involved during pre-training of course, but at the very least it's going to a provide a jumping off point that the creators of the model co-signed on.

BoorishBears 3y ago

You should look at the consumer app system prompt: https://twitter.com/amandaaskell/status/1765207842993434880 https://twitter.com/amandaaskell/status/1765207842993434880 I've had way more success taking it and forming it to my tasks than stuffing in a bunch of tokens about how to do things. In one case I was able to go from ~6k in input tokens to ~3k because I no longer had to provide a mountain of examples and instructions for corner cases

namanyayg 3y ago
Can you elaborate on how does the claude system prompt help in creating prompts for different tasks?
- BoorishBears 3y ago
  Anytime I see a prompt from one of these companies, I assume it matches the style of instructions the model encountered during pre-training. And the kinds of instruction formats that were encountered during pre-training end up informing what style of instruction the model is best at following. An extreme example would be prompt templates that a "raw" instruct-tuned LLM follows: the model will technically work with a suboptimal format, but you get much better performance if you follow the prompt template the model was trained/fine-tuned on. - It's not a guarantee a given prompting style was involved during pre-training of course, but at the very least it's going to a provide a jumping off point that the creators of the model co-signed on.