3 ms·

Interestingly, they have withdrawn their model because they discovered that test data leaked into the training data. Quantised versions are still available on

Interestingly, they have withdrawn their model because they discovered that test data leaked into the training data. Quantised versions are still available on huggingface from others though. https://github.com/SLAM-group/newhope https://github.com/SLAM-group/newhope

moffkalast 3y ago
Yeah it seemed suspiciously high for HumanEval and it only ranks 14th for JS and 7th for Python on other benchmarks now: https://huggingface.co/spaces/mike-ravkine/can-ai-code-results https://huggingface.co/spaces/mike-ravkine/can-ai-code-resul... WizardCoder is a bit of a problem since it's not llama 1/2 based but is its own 15B model and as such the support for it in anything practical is near nonexistent. WizardLM v1.2 looks like it may be worth testing out. All of the LLama 2 fine tunes I've tried out so far have weird issues though. Saying unrelated things at times, ignoring parts of the conversation and such. Could be fine tuning or prompt template goofs or Llama 1 may actually be a more self consistent base model overall.