3 ms·
Sounds like you need to check what the max context is set to ...
by spwa4 2mo ago
Sounds like you need to check what the max context is set to ...
- jedbrooke 2mo ago100k is all the context I have ram for, this is with any auto-compact turned off. This is using Cline in vs code. I’m sure I could tune the system prompt and mode switching more to work better with this specific model, but I haven’t gone down the custom harness rabbit hole yet. And this is also specifically for the 1bit quant version. I don’t think the fp8 or even fp4 versions have this issue, but I haven’t tried those much