3 ms·
Yeah but exllama doesn't do grammars so I'm stuck with llama.cpp Also apparently exllama has a few side effects in coherence https://www.reddit.com/r/LocalLLaM
by avereveard 3y ago
Yeah but exllama doesn't do grammars so I'm stuck with llama.cpp
Also apparently exllama has a few side effects in coherence https://www.reddit.com/r/LocalLLaMA/comments/17w57eu/llm_format_comparisonbenchmark_70b_gguf_vs_exl2/ https://www.reddit.com/r/LocalLLaMA/comments/17w57eu/llm_for...