4 ms·
I'm so excited for this, nice work! Gemma4 edge models were promised to be great for agentic use, but have been really disappointing in all my tests. They fail
by meander_water 5mo ago
I'm so excited for this, nice work!
Gemma4 edge models were promised to be great for agentic use, but have been really disappointing in all my tests. They fail at the most basic tool use scenarios.
Have you run any tool-use benchmarks for Needle, or do you plan to? Would be great if you could add results to the repo if so.
- TobTobXX 5mo agoWait what? I've used DeepSeek V4 flash a lot and compsred to Gemma 4 E2B (ie. the smallest, event at q4), it consistently underperforms. In contrast to DS flash, I've found Gemma 4 to be incredibly precise and consistent with tool use.
- hiroto_lemon 5mo ago[flagged]