3 ms·
Only 8K context as well, like Mistral. Also, as always, take these benchmarks with a huge grain of salt. Even base model releases are frequently (seemingly) co
by brucethemoose2 3y ago
Only 8K context as well, like Mistral.
Also, as always, take these benchmarks with a huge grain of salt. Even base model releases are frequently (seemingly) contaminated these days.
- tosh 3y agoAgree: will be interesting how Gemma does on ChatBot Arena
- DreamGen 3y agoMistral Instruct v0.2 is 32K.
- Kydlaw 3y agoThey state in their report that they filter evaluation data off their training data, see p.3 - Filtering: "Further, we filter all evaluation sets from our pre-training data mixture, run targeted contamination analyses to check against evaluation set leakage, and reduce the risk of recitation by minimizing proliferation of sensitive outputs."