4 ms·
> Rocket 3b uses a slightly different prompt syntax. Wouldn't it be better if llamafile were to standardize the prompt syntax across models?
by davidkunz 3y ago
> Rocket 3b uses a slightly different prompt syntax.
Wouldn't it be better if llamafile were to standardize the prompt syntax across models?
- minimaxir 3y agoThere's currently no standard because there's no one objective best way of handling prompt syntax. There's some libraries which use the OpenAI API syntax as a higher-level abstraction, but for the lower-level precompiled binaries used in in this post that's too much.
- Tostino 3y agoYes there is...HF chat templates is something that is being standardized on, slowly. It's just a jinja template embedded in the tokenizer that the model creator can include.
- jart 3y agollamafile can provide an abstraction, but ultimately it boils down to how the model was trained and/or fine-tuned.