3 ms·
Ah I see where confusion might come. I don't mean the exact prompt shouldn't matter, but I am saying that we noticed that these series of models picked up on t
by canyon289 2y ago
Ah I see where confusion might come.
I don't mean the exact prompt shouldn't matter, but I am saying that we noticed that these series of models picked up on tool call format quite readily in our various tests, which is what we express in the docs. We tested internally and I hope the independent BFCL results speak for themselves! All their code and evals are public fully public.
> I would imagine training with a specific, perhaps structured, prompt could make the function calling a bit more robust.
This is absolutely true. I show this in a tutorial last year where Gemma2 is finetuned for a specific format, and with some targeted SFT it produces a json output more readily.
https://www.youtube.com/watch?v=YxhzozLH1Dk https://www.youtube.com/watch?v=YxhzozLH1Dk
So this is all to say, Gemma is designed to be a great model for multiple types of users. If you want to use the "out of the box" weights with your own format, go ahead! We hope that makes it easier to integrate with whatever tooling you're using with minimal headache.
If you need specific performance on your bespoke format finetune the model to be your own! Finetuning is supported across many frameworks so pick whatever library you like best.
This is all to say we hope Gemma is flexible and usable for folks like yourself along a variety of dimensions. For myself I'm learning there's big interest in a specific prompt. Again can't thank you enough for the feedback here.