3 ms·
Looks like MLX quants are released: - 4bit: https://huggingface.co/mlx-community/QwQ-32B-4bit https://huggingface.co/mlx-community/QwQ-32B-4bit - 6bit: https:
by pickettd 2y ago
Looks like MLX quants are released:
- 4bit: https://huggingface.co/mlx-community/QwQ-32B-4bit https://huggingface.co/mlx-community/QwQ-32B-4bit
- 6bit: https://huggingface.co/mlx-community/QwQ-32B-6bit https://huggingface.co/mlx-community/QwQ-32B-6bit
- seanmcdirmid 2y agoI downloaded the 8 bit quant last night but haven't had a chance to play with it yet.