2 ms·
> I have seen people actually not get: that LLMs wrap tool calls in special tokens This isn't necessarily true. I'm working on a local harness that doesn't do
by dvt 20d ago
> I have seen people actually not get: that LLMs wrap tool calls in special tokens
This isn't necessarily true. I'm working on a local harness that doesn't do this and instead coerces everything to YAML (including tool calls) for better bucketing. Some models are indeed trained on the `<|tool_call>...<tool_call|>` token schema (or something similar—e.g. jinja), but it's vendor-specific and often times inconsistent (so you're constantly fixing calls or going back to the LLM).
- stephenblum 20d agonice! like HTML tags. I remember seeing this back with meta Llama 2. the <|start_header_id|>assistant<|end_header_id|> style. And [TOOL_CALLS] ... [/TOOL_CALLS] style with Mistral