3 ms·
I think they messed up the templates.
by YetAnotherNick 3y ago
I think they messed up the templates.
- detente18 3y agoHere's the template I'm using - https://github.com/BerriAI/litellm/blob/5ca8b23e22139a4f49bd0d4c1d5a2d4a70b49166/litellm/llms/prompt_templates/factory.py#L5 https://github.com/BerriAI/litellm/blob/5ca8b23e22139a4f49bd... Anything I'm doing incorrectly?
- YetAnotherNick 3y agoThe format is really weird for chat models. In your code post message is `<</SYS>>\n [/INST]` which is wrong. Instead the overall format is `[INST] <<SYS>>\n{system}\n<</SYS>>\n\n{instruction} [/INST]` So both the system message and first instruction are in same [INST] block.