4 ms·
Since local models are largest constrained by context window you want to have a tiny system prompt. I know Hax: https://github.com/OleksandrChekhovskyi/hax http
by wyre 14d ago
Since local models are largest constrained by context window you want to have a tiny system prompt. I know Hax: https://github.com/OleksandrChekhovskyi/hax https://github.com/OleksandrChekhovskyi/hax was designed with local models in mind, but I haven't used it.
- khimaros 14d agoi built something with a similar philosophy https://github.com/khimaros/hrns https://github.com/khimaros/hrns