4 ms·
It is a great model, and blazing fast, which is actually very useful esp for "reasoning" models, as they produce a lot of tokens. I wish mistral were back into
by freehorse 1y ago
It is a great model, and blazing fast, which is actually very useful esp for "reasoning" models, as they produce a lot of tokens.
I wish mistral were back into making MoE models. I loved their 8x7 mixtral, it was one of the greatest models I could run the time it went out, but it is outdated now. I wish somebody was out making a similar size MoE model, which could comfortably sit in a 64GB ram macbook and be fast. Currently the qwen 30-A3B is the only one I know of, but it would be nice to have something slightly bigger/better (incl a non-reasoning base one). All the other MoE models are just too big to run locally in more standard hardware.