4 ms·

Are tools fine tuned in both or just prompt based? For a fair comparison

• Joschkabraun 3y ago
  All models got the same prompt fed which was essentially "Question: {question}". And then the API's accept the function call definition
  • airocker 3y ago
    But what if there is a bad model that fine tunes with only tools?
    • Joschkabraun 3y ago
      What do you mean with being a bad model? If the model is really good at tool use, then it will broadly useful as it needs capabilities to generate the tool definition. So, there should be some transferability.
      • ec109685 3y ago
        If the exact tools test was part of the training of the model, wouldn’t that throw off the results and not generalize?
• fudoshin2596 3y ago
  No fine tuning. Looks like he's do raw model capabilities with simple prompt. repo: https://github.com/parea-ai/tool-use-benchmark https://github.com/parea-ai/tool-use-benchmark