Correction: looks like it has to do with the quantization rather: 8bit quantization works while less does not not seem to work. Other working model example (no

Correction: looks like it has to do with the quantization rather: 8bit quantization works while less does not not seem to work. Other working model example (no conversion needed): https://huggingface.co/TheBloke//Yarn-Mistral-7B-64k-GGUF/yarn-mistral-7b-64k.Q8_0.gguf https://huggingface.co/TheBloke//Yarn-Mistral-7B-64k-GGUF/ya...