Comparatively, Mistral 7x8b easily handles, even at q4. Not a fundamental LLM limit. Proof here: https://news.ycombinator.com/item?id=39362651 https://news.yco

Comparatively, Mistral 7x8b easily handles, even at q4. Not a fundamental LLM limit. Proof here: https://news.ycombinator.com/item?id=39362651 https://news.ycombinator.com/item?id=39362651 If you want to replicate, use `TheBloke/dolphin-2.5-mixtral-8x7b-GGUF/dolphin-2.5-mixtral-8x7b.Q4_0.gguf`.