5 ms·

> Can you quote tps? LLM Studio running on a Mac Studio M4 Max with 128GB, gemma-3-27B-it-QAT-Q4_0.gguf with a 4096 token context I get 8.89 tps.

• jychang 1y ago
  That's pretty terrible. I'm getting 18tok/sec Gemma 3 27b QAT on a M1 Max 32gb macbook.
  • pantulis 1y ago
    Yeah, I know. Not sure if this due to something in LLM Studio or whatever.
  • pantulis 1y ago
    I was running the mac in low power mode!!! Getting 24tps now.
• kristianp 1y ago
  Is QAT a different quantisation format to Q4_0? Can you try "gemma-3-27b-it-qat" for a model: https://lmstudio.ai/model/gemma-3-27b-it-qat https://lmstudio.ai/model/gemma-3-27b-it-qat
  • pantulis 1y ago
    Thanks for your suggestion! I think it was just the filename. I tried the model you suggested by opening it in LLMStudio and keep getting 8.3 tps.
    • pantulis 1y ago
      Now this is raising my curiosity, is there anything else I could try and tweak to achieve better tps? Can it be related to being GGUF instead of MLX?
• pantulis 1y ago
  Gah, turns out I was running the Mac in low power mode! I get 24tps in LM Studio now with gemma-3-27b-it-qat.