4 ms·
It might have something to do with quantization though, I have used the Q6_K version from https://huggingface.co/bartowski/mathstral-7B-v0.1-GGUF https://huggin
by rahduro 2y ago
It might have something to do with quantization though, I have used the Q6_K version from https://huggingface.co/bartowski/mathstral-7B-v0.1-GGUF https://huggingface.co/bartowski/mathstral-7B-v0.1-GGUF with Llamafile. It always shows 9.11 is bigger than 9.9.