3 ms·
One thing about local llm projects is that they don't mention the maximum context length which for this project is 4k. I think for it to be used with a harness
by darkoob12 2mo ago
One thing about local llm projects is that they don't mention the maximum context length which for this project is 4k. I think for it to be used with a harness you need at least 200k and reliable tool calling and structured output which obviously degrade at high quantizations.