4 ms·
There’s an initial version of it already up and running: https://huggingface.co/DiscoResearch/mixtral-7b-8expert https://huggingface.co/DiscoResearch/mixtral-7
by bugglebeetle 3y ago
There’s an initial version of it already up and running:
https://huggingface.co/DiscoResearch/mixtral-7b-8expert https://huggingface.co/DiscoResearch/mixtral-7b-8expert
- ilaksh 3y agoI am doubtful about these efforts because people are just guessing about how it is supposed to work since Mistral didn't release any code or information yet.
- bugglebeetle 3y agoYes, there are probably some implementation pieces missing. But Disco Research also punches way above their weight on open source LLM stuff, so I would gamble it’s not that far off. They also just released a chat fine-tune of the model just moments ago: https://huggingface.co/DiscoResearch/DiscoLM-mixtral-8x7b-v2 https://huggingface.co/DiscoResearch/DiscoLM-mixtral-8x7b-v2
- MacsHeadroom 3y agoPeople had Mixtral 8x7B running on Twitter within 3 hours of the torrent being posted. It's already been independently benchmarked and finetunes are cooking.