4 ms·
> That too was broken in mlx-lm (it crashed), but has since been fixed on the main branch Unfortunately I have got zero success running gemma with mlx-lm main
by egorfine 6mo ago
> That too was broken in mlx-lm (it crashed), but has since been fixed on the main branch
Unfortunately I have got zero success running gemma with mlx-lm main branch. Can you point me out what is the right way? I have zero experience with mlx-lm.
- Confiks 6mo agoGet into a venv, and run: > pip3 install git+https://github.com/ml-explore/mlx-lm.git https://github.com/ml-explore/mlx-lm.git > ./venv/bin/mlx_lm.generate --model "$MODEL" --temp 1.0 --top-p 0.95 --top-k 64 --max-tokens 128000 --prompt "Hello world" Where $MODEL is an unsloth model like: - unsloth/gemma-4-E4B-it-UD-MLX-4bit - unsloth/gemma-4-26b-a4b-it-UD-MLX-4bit
- egorfine 6mo agoThanks!