3 ms·
Does benchmarks look incredible. Like almost too good to be true, what am I missing? Is this hosted online somewhere so I can try it out?
by Alifatisk 1y ago
Does benchmarks look incredible. Like almost too good to be true, what am I missing?
Is this hosted online somewhere so I can try it out?
- viraptor 1y agoIt's so tiny you can download and run it locally on CPU with llama.cpp. It seems weirdly good at some simple python questions. Definitely better than I'd expect from any model of that size.
- Balinares 1y agoI don't know how the coding benchmarks are computed but this model, on its own and outside of agentic loops, definitely doesn't compare to e.g. Qwen3 Coder. I might still try that for fun, just to see how it performs given a feedback loop. On math questions, though, beside a marked tendency towards rambling thinking, it's just plain implausibly good for a 1.5B model. This is probably just rote learning, though. Otherwise this might well be a breakthrough.