3 ms·
Here's the huggingface link from that article: https://huggingface.co/mlx-community/OLMo-2-0325-32B-Instruct-4bit https://huggingface.co/mlx-community/OLMo-2-03
by kristianp 2y ago
Here's the huggingface link from that article: https://huggingface.co/mlx-community/OLMo-2-0325-32B-Instruct-4bit https://huggingface.co/mlx-community/OLMo-2-0325-32B-Instruc...
How do mlx quants compare to gguf?
Edit: the thread below says that mlx is faster, but gguf quantisations tend to maintain better text quality.
https://www.reddit.com/r/LocalLLaMA/comments/1gc0t0c/how_does_mlx_quantization_compare_to_gguf/ https://www.reddit.com/r/LocalLLaMA/comments/1gc0t0c/how_doe...