3 ms·

The main aider leaderboard includes gemini models. I benchmarked gemma-2-27b-it, but it did so badly that it wasn't worth adding to the leaderboard. https://ai

• Lash-L 2y ago
  Interesting - why do you think that is? can-ai-code seems to have Gemma 2-27b as pretty capable - beating out some of the other models that perform really well on aider: https://huggingface.co/spaces/mike-ravkine/can-ai-code-results https://huggingface.co/spaces/mike-ravkine/can-ai-code-resul...
  • 22c 2y ago
    Not an expert, but I believe it's related to this difference: > LLMs which are good at editing code, not just good at writing code So it may be that Gemma can write new code, but is not particularly good at editing or refactoring code.