4 ms·
Is this Mac only? Or is that an Ollama issue that it only supports this release of models on Mac? It seems like every tag with the MLX badge is only supported o
by Zambyte 4mo ago
Is this Mac only? Or is that an Ollama issue that it only supports this release of models on Mac? It seems like every tag with the MLX badge is only supported on Mac[0], and that includes all of the tags in this release.
[0] https://ollama.com/library/gemma4/tags https://ollama.com/library/gemma4/tags
Edit: MLX being Mac-only is independent of the model being MLX (and therefore Mac) only. The latter is what I am asking about.
- embedding-shape 4mo agoMLX is quite literally macOS-specific technology, for other platforms you want non-MLX. I was sure "MLX" stood for "Metal-something-something" but can't find any reference to that somehow, anywho, "Metal" is hardware-accelerated graphics on Apple platforms FWIW. Edit: about the actual release on Ollama, if you're on non-Apple hardware you probably want the NVFP4 variant ("gemma4:12b-nvfp4") which was uploaded 45 minutes ago, especially if you're with a recent nvidia GPU.
- sambaumann 4mo agoI still get "this model requires macOS" when trying to pull that one
- embedding-shape 4mo agoI don't use Ollama myself anymore, but seems others been having similar issues for quite some time, maybe one of these fit your environment exactly? https://github.com/ollama/ollama/issues?q=is%3Aissue%20state%3Aopen%20%22this%20model%20requires%20macOS%22 https://github.com/ollama/ollama/issues?q=is%3Aissue%20state...
- Patrick_Devine 4mo agoI realize this is a little confusing; we're working w/ the MLX team to bring MLX to other platforms, but we're not quite there yet. The `gemma4:12b-nvfp4` model is specifically for the MLX engine. For the GGUF 4bit variant (i.e. non-macs) you'll need `gemma4:12b-it-q4_K_M` which I just pushed. You'll also need to upgrade to version 0.30.4 which we're just about to release (it's in prerelease and we're running through our last regression tests).
- embedding-shape 4mo agoI gotta say, having both "gemma4:12b-mlx-bf16" and "gemma4:12b-nvfp4" be MLX-specific, and not labeling all of the MLX-specific ones as such, is a bit different than "little confusing" and more "set up to be confusing" :) > You'll also need to upgrade to version 0.30.4 which we're just about to release Interesting, wasn't Google coordinating today's release with you? Considering the blog post seems to have gone out way before the release even been cut.
- Patrick_Devine 4mo agoGiven the model was just republished by Google 15 minutes ago and we're going to have to redo everything (and everyone will have to redownload for all platforms -- not just Ollama), I'll just say that sometimes things don't work out exactly the way you want them to. :-D That said, I think the gemma4:12b-nvfp4 model is pretty solid. It's been tuned with Nvidia's model optimizer. I've been waiting on the results for MMLU-Pro, but I'll have to retrigger that after reconverting.
- embedding-shape 4mo ago> Given the model was just republished by Google 15 minutes ago Hah, missed that! Guess that's slightly neat though, you get a second chance ;) NVFP4 been a blast to use across a wide range of models, seems to work really well, at least with vLLM and a nvidia card.
- spicySpy 4mo agoWould you mind to share the link to `gemma4:12b-it-q4_K_M`?
- jw1224 4mo agoMLX is Apple’s own machine learning framework, designed for Apple Silicon: https://opensource.apple.com/projects/mlx/ https://opensource.apple.com/projects/mlx/
- jasonjmcghee 4mo agoThere's a CUDA backend for MLX now. Not sure about the maturity.
- accountrequired 4mo agohttps://huggingface.co/ggml-org/gemma-4-12B-it-GGUF/tree/main?local-app=ollama https://huggingface.co/ggml-org/gemma-4-12B-it-GGUF/tree/mai...
- Zambyte 4mo agoThe non-MLX versions just dropped on Ollama. gemma4:12b-it-q8_0, gemma4:12b-it-bf16, etc.