3 ms·

> The reason why they are bad at tool calling is they aren't trained on it. Yes they are.* > OpenAI just talked about it recently as being the answer to auton

> The reason why they are bad at tool calling is they aren't trained on it. Yes they are.* > OpenAI just talked about it recently as being the answer to autonomous agents You nailed it. It's key to autonomous agents, OpenAI says so out loud, and yet, we're 6 months into reasoning models and performance is regressing, which OpenAI also says out loud but in the fine print of a model card. I know this is a flaming hot take because current thing is reasoning. But it's completely mistaken that reasoning models help with tool-use, both in theory and practice, which puts them in quite a situation. I'm sure they'll figure it out, but I'm also sure on a long enough timeline the LLM is a computer. * I have a bad habit of dismissing without evidence that which is asserted without evidence, and 'not even wrong', in the Pauli sense. It's a cheap way to avoid confrontation. But it makes me look petulant. We can observe, inter alia, every release post September 2024 (i.e. o1-mini and o1-preview) can call tools. (i.e. o1, o3-mini)

mountainriver 2y ago
Please cite the source for o1 being trained heavily on tool usage. Reasoning will obviously improve agentic workflows, reasoning is the main problem we see today with autonomous agents. It seems to be likely the _last_ issue we have with them
- refulgentis 2y ago
  Hmm... - It's going to be difficult to find a source that says heavily on tool use. They don't talk about training at all anymore! :( And saying you trained heavily on tool use would imply your model isn't trained well for other stuff, which they'd want to avoid. - Nothing implies they didn't do the exact same tool training they have been. Both model cards mention tool use. - Both o1 and o3-mini have bog-standard tool calling via API. I'm not sure what gives us the sense that they weren't trained on them - The claim shifted significantly: From: "The reason why they are bad at tool calling is they aren't trained on it" To: "[I don't think you can find a source that says] o1 [was] trained heavily on tool usage."

mountainriver 2y ago

Please cite the source for o1 being trained heavily on tool usage. Reasoning will obviously improve agentic workflows, reasoning is the main problem we see today with autonomous agents. It seems to be likely the _last_ issue we have with them

refulgentis 2y ago
Hmm... - It's going to be difficult to find a source that says heavily on tool use. They don't talk about training at all anymore! :( And saying you trained heavily on tool use would imply your model isn't trained well for other stuff, which they'd want to avoid. - Nothing implies they didn't do the exact same tool training they have been. Both model cards mention tool use. - Both o1 and o3-mini have bog-standard tool calling via API. I'm not sure what gives us the sense that they weren't trained on them - The claim shifted significantly: From: "The reason why they are bad at tool calling is they aren't trained on it" To: "[I don't think you can find a source that says] o1 [was] trained heavily on tool usage."