3 ms·
I'd recommend those looking for local coding models to go for code-specific tunes. See the EvalPlus leaderboard (HumanEval+ and MBPP+): https://evalplus.github.
by lhl 2y ago
I'd recommend those looking for local coding models to go for code-specific tunes. See the EvalPlus leaderboard (HumanEval+ and MBPP+): https://evalplus.github.io/leaderboard.html https://evalplus.github.io/leaderboard.html
For those looking for less contamination, the LiveCodeBench leaderboard is also good: https://livecodebench.github.io/leaderboard.html https://livecodebench.github.io/leaderboard.html
I did my own testing on the 110B demo and didn't notice any cross-lingual issues (which I've seen with the smaller and past Qwen models), but for my personal testing, while the 110B is significantly better than the 72B, it doesn't punch above its weight (and doesn't perform close to Llama 3 70B Instruct from my testing). https://docs.google.com/spreadsheets/d/e/2PACX-1vRxvmb6227Au4HL0ROH5uxaFPJReCXbG1IU-uMj06d3VgYO5Mu2h_gvf3Fi3gdOT3mXEev3xDf-GkhW/pubhtml https://docs.google.com/spreadsheets/d/e/2PACX-1vRxvmb6227Au...