4 ms·
Mistral doesn't release source code or data just weights. They did include some details of sliding window attention previously
by emadm 3y ago
Mistral doesn't release source code or data just weights. They did include some details of sliding window attention previously
- ebalit 3y agoThey released a reference implementation with Mistral 7B and a fork of a framework they probably used to train the Mixture of Experts for the new one. But it's still early and I guess that they kept just enough mystery to kickstart the conversation around this new model!