3 ms·
That's some really strange behavior, I don't know why that would cause poor results rather than just poor performance. Can you configure the context size with
by Gracana 2y ago
That's some really strange behavior, I don't know why that would cause poor results rather than just poor performance.
Can you configure the context size with `/set parameter num_ctx N`? On my laptop with an RTX A3000 12GB I can run `yi-coder:9b-chat` (Q4_0) with 32768 context and it produces good results quickly. That uses 11GB of VRAM so it's maxed out for this setup.
- tmikaeld 2y agoSolved, see: https://github.com/01-ai/Yi-Coder/issues/6#issuecomment-2334058066 https://github.com/01-ai/Yi-Coder/issues/6#issuecomment-2334... Works very well now! 65K input tokens with 8192 output tokens is no longer an issue on my 4090. (It maxes out on 22GB/VRAM)
- Gracana 2y agoAwesome! Glad to hear you got it sorted out.