3 ms·
I'm still iterating on that. Plugins get complete control over the prompts, so they can handle the various weirdnesses of them. Here's some relevant code: http
by simonw 3y ago
I'm still iterating on that. Plugins get complete control over the prompts, so they can handle the various weirdnesses of them. Here's some relevant code:
https://github.com/simonw/llm-gpt4all/blob/0046e2bf5d0a9c369b804d7125a1ab50bd5878f1/llm_gpt4all.py#L87-L92 https://github.com/simonw/llm-gpt4all/blob/0046e2bf5d0a9c369...
https://github.com/simonw/llm-mlc/blob/b05eec9ba008e700ecc422c69e28395fa8e146fd/llm_mlc.py#L290-L295 https://github.com/simonw/llm-mlc/blob/b05eec9ba008e700ecc42...
https://github.com/simonw/llm-llama-cpp/blob/29ee8d239f5cfbf0b4bc1e55af39045708840b25/llm_llama_cpp.py#L197-L220 https://github.com/simonw/llm-llama-cpp/blob/29ee8d239f5cfbf...
I'm not completely happy with this yet. Part of the problem is that different models on the same architecture may have completely different prompting styles.
I expect I'll eventually evolve the plugins to allow them to be configured in an easier and more flexible way. Ideally I'd like you to be able to run new models on existing architectures using an existing plugin.