Yeah it works fine. Mistral is a llama architecture llm. There are quantizations already. I would suggest running it on KoboldCPP, or any of the UIs recommende

Yeah it works fine. Mistral is a llama architecture llm. There are quantizations already. I would suggest running it on KoboldCPP, or any of the UIs recommended on this page: https://huggingface.co/TheBloke/MistralLite-7B-GGUF https://huggingface.co/TheBloke/MistralLite-7B-GGUF Personally, I run the elx2 quantization with exllamav2 because its faster and more coherent in the same amount of vram. You can do that with Aphrodite or Text-gen-ui: https://huggingface.co/models?search=MistralLite%20exl2 https://huggingface.co/models?search=MistralLite%20exl2