4 ms·

How is this different from function calling?

binarymax 2y ago
Function calling uses JSON mode. While it has been mostly correct, I do get an incorrectly formatted response sometimes (maybe 1 in 10k requests?). So it sounds like this fixes that bug.
tedsanders 2y ago
Under the hood, it's quite similar to function calling. A few differences: - Structured Outputs is a bit more straightforward. e.g., you don't have to pretend you're writing a function where the second arg could be a two-page report to the user, and then pretend the "function" was called successfully by returning {"success": true} - Having two interfaces lets us teach the model different default behaviors and styles, depending on which you use - Another difference is that our current implementation of function calling can return both a text reply plus a function call (e.g., "Let me look up that flight for you"), whereas Structured Outputs will only return the JSON (I worked on this feature at OpenAI.)
- technics256 2y ago
  How can we enable the text reply with a function call? Usually the message returned is a tool call only when it calls a tool?
  - tedsanders 2y ago
    There's no special interface, but you can write an instruction in a system message in the first position. E.g., "Before each function call, explain to the user what you're about to do." It's not super reliable, but the model can do it. Few-shot prompting might help as well.
nsonha 2y ago
I guess practically this will be the equivalent to function calling with a single tool. The AI splits out some payload that you presumably feed into whatever you do next. There are only 2 possible outcomes: can/cannot produce payload, which is equivalent to call/not call the next functionality. With function/tool calling, you give it a list of tools with payload and the AI will select between them, or refuse, so there are more than 2 possible outcomes. (I've never used any of these APIs, get that from just reading the docs)

Function calling uses JSON mode. While it has been mostly correct, I do get an incorrectly formatted response sometimes (maybe 1 in 10k requests?). So it sounds like this fixes that bug.

tedsanders 2y ago

Under the hood, it's quite similar to function calling. A few differences: - Structured Outputs is a bit more straightforward. e.g., you don't have to pretend you're writing a function where the second arg could be a two-page report to the user, and then pretend the "function" was called successfully by returning {"success": true} - Having two interfaces lets us teach the model different default behaviors and styles, depending on which you use - Another difference is that our current implementation of function calling can return both a text reply plus a function call (e.g., "Let me look up that flight for you"), whereas Structured Outputs will only return the JSON (I worked on this feature at OpenAI.)

technics256 2y ago
How can we enable the text reply with a function call? Usually the message returned is a tool call only when it calls a tool?
- tedsanders 2y ago
  There's no special interface, but you can write an instruction in a system message in the first position. E.g., "Before each function call, explain to the user what you're about to do." It's not super reliable, but the model can do it. Few-shot prompting might help as well.

nsonha 2y ago

I guess practically this will be the equivalent to function calling with a single tool. The AI splits out some payload that you presumably feed into whatever you do next. There are only 2 possible outcomes: can/cannot produce payload, which is equivalent to call/not call the next functionality. With function/tool calling, you give it a list of tools with payload and the AI will select between them, or refuse, so there are more than 2 possible outcomes. (I've never used any of these APIs, get that from just reading the docs)