3 ms·
> To evaluate the generalization capabilities of Mistral 7B, we fine-tuned it on instruction datasets publicly available on the Hugging Face repository. Heh, t
by brucethemoose2 3y ago
> To evaluate the generalization capabilities of
Mistral 7B, we fine-tuned it on instruction datasets
publicly available on the Hugging Face repository.
Heh, they won't even say what datasets they used for chat finetuning.
> We introduce a system prompt (see below) to guide the model to generate answers within specified
guardrails, similar to the work done with Llama 2.
This was totally undocumented in the initial model release.
Other than that... Not much really new? We already know it uses SWA, though it works without SWA in current llama implementations, and SWA isnt new either.
If most upcoming base models are this mysterious on release, the field is going to be... weird.
- riedel 3y agoWeird is the right term: do they want to demonstrate with this arxiv paper that they manage reformat a blog post into latex and upload it to a preprint site after publication?