4 ms·
Are you able explain what the charts mean? Only one of the three has wavecoder at the top.
by austhrow743 3y ago
Are you able explain what the charts mean? Only one of the three has wavecoder at the top.
- ignoramous 3y agoThose charts show pass@k metric (expectation at least k generated samples are correct out of n) on OpenAI and Octopack problem evals for code. WaveCoder: https://arxiv.org/abs/2312.14187 https://arxiv.org/abs/2312.14187 (section 3.2) Octopack: https://github.com/bigcode-project/octopack https://github.com/bigcode-project/octopack
- srikanth235 3y agoWhile testing internally, Mistral worked well. But these models are just starting points. Will add support for the models WaveCoder-Ultra-6.7B, WizardCoder-33B, Magiccoder-S-DS-6.7B, etc soon.