2 ms·
I don't know how the coding benchmarks are computed but this model, on its own and outside of agentic loops, definitely doesn't compare to e.g. Qwen3 Coder. I m
by Balinares 1y ago
I don't know how the coding benchmarks are computed but this model, on its own and outside of agentic loops, definitely doesn't compare to e.g. Qwen3 Coder. I might still try that for fun, just to see how it performs given a feedback loop.
On math questions, though, beside a marked tendency towards rambling thinking, it's just plain implausibly good for a 1.5B model. This is probably just rote learning, though. Otherwise this might well be a breakthrough.