4 ms·

I tried gemma-4-26B-A4B just to see if it could help me read/sort my emails on a relatively under-powered setup (16GB VRAM + 32GB RAM) and it's not going well..

I tried gemma-4-26B-A4B just to see if it could help me read/sort my emails on a relatively under-powered setup (16GB VRAM + 32GB RAM) and it's not going well.. the model burns 24K tokens just on searching for the right tool and then dumps the email contents into context - i tried to get it to use code-mode to save context but the code-mode implementation can't save files so it was useless and im going to try to switch to "ssh-mode" into my devbox container. Still relatively new to this, so I'm probably doing something wrong

anana_ 4mo ago
Perhaps try a different model? Just from anecdotal experience, I find that the Gemma models smaller than 31B do not tool call as often as they should. Some of the benchmarks appear to back this up [0] Of course, a lot depends how you are using it (inference parameters, harness, prompting, etc.), but the model is quite important too. [0]: https://artificialanalysis.ai/models/open-source/small?models=qwen3-6-27b%2Cqwen3-6-35b-a3b%2Cgemma-4-31b%2Cqwen3-6-27b-non-reasoning%2Cqwen3-5-9b%2Cgemma-4-31b-non-reasoning%2Cqwen3-6-35b-a3b-non-reasoning%2Cgemma-4-26b-a4b%2Cqwen3-5-35b-a3b-non-reasoning%2Cgemma-4-12b#intelligence-evaluations https://artificialanalysis.ai/models/open-source/small?model...
Rzor 4mo ago
So there was a problem with gemma 4 when it comes to tool calling that Google apparently fixed like 2 or 3 days ago. I remember reading something about this.