3 ms·

I wonder if they were just slightly ahead of this announcement? https://blog.google/innovation-and-ai/technology/developers-tools/quantization-aware-training-g

by dofm 5mo ago

I wonder if they were just slightly ahead of this announcement? https://blog.google/innovation-and-ai/technology/developers-tools/quantization-aware-training-gemma-4/ https://blog.google/innovation-and-ai/technology/developers-... Looks like the 12B model should fit now?

minimaxir 5mo ago
It definitely works in LM Studio, not Edge Gallery yet.
- dofm 5mo ago
  Following up again, in case you see this. I was just in oMLX trying to set up the new 26B QAT models with MTP, and I noticed this message: Kernel iogpu.wired_limit_mb is only 48.0 GB; oMLX can only allocate up to 48.0 GB. Raise it in Terminal: sudo sysctl iogpu.wired_limit_mb=59392 Perhaps if you can increase the wired limit it will fit?