Correction, it should have the performance of a single 12B parameter model: "Concretely, Mixtral has 45B total parameters but only uses 12B parameters per toke

Correction, it should have the performance of a single 12B parameter model: "Concretely, Mixtral has 45B total parameters but only uses 12B parameters per token. It, therefore, processes input and generates output at the same speed and for the same cost as a 12B model." https://mistral.ai/news/mixtral-of-experts/ https://mistral.ai/news/mixtral-of-experts/