3 ms·

Oh, that's really nice. I'd also like to see what syntax the LLM uses to _trigger_ these calls, and what prompt is sent to the LLM to tell it how to do that. I

• medbrane 2y ago
  That's dependent on the particular LLM one uses. But it can be a json with the tool name and the payload for the tool.
• laichzeit0 2y ago
  The LLM returns a message called ToolMessage which then describes which function to call and the parameters (you register these functions/tools as part of the initialisation step, like when you pass it temperature, or whatever other options your LLM allows you to set). So think of it as instead of steaming back text, it’s streaming back text to tell you “please call this function with these args” and you can do with that what you want. Ideally you’d call that function and then give the output back to the LLM. Nothing magic really.