3 ms·

Here is a leader board of some models https://huggingface.co/spaces/mike-ravkine/can-ai-code-results https://huggingface.co/spaces/mike-ravkine/can-ai-code-res

Here is a leader board of some models https://huggingface.co/spaces/mike-ravkine/can-ai-code-results https://huggingface.co/spaces/mike-ravkine/can-ai-code-resul... Don't know how biased this leaderboard is, but I guess you could just give some of them a try and see for yourself.

SparkyMcUnicorn 3y ago
This is a much better leaderboard: https://evalplus.github.io/leaderboard.html https://evalplus.github.io/leaderboard.html I've seen the CanAiCode leaderboard several times (and used many of the models listed), but I wouldn't use it to pick a model. It's not a bad list, but the benchmark is too limited. The results are not accurately ranked from best to worst. For example the deepseek 33b model is ranked 5 spots lower than the 6.7b model, but the 33b model is definitely better. WizardCoder 15b is near the top while WizardCoder 33b is ranked 26 spots lower, which is a wildly inaccurate ranking. It's worth noting that those 33b models score in the 70s for HumanEval and HumanEval+ while the 15b model scores in the 50s.
- SushiHippie 3y ago
  Thanks