2 ms·
Mixtral is also a MoE model, hence the name: mixtral.
by hexomancer 3y ago
Mixtral is also a MoE model, hence the name: mixtral.
- sangnoir 3y agoDespite both being MoEs, thr architectures are different. DBRX has double the number of experts in the pool (16 vs 8 for Mixtral), and doubles the active experts (4 vs 2)