3 ms·

Mixtral 8x22B latest-model benchmarks

• adif_sgaid 3y ago
  *Benchmarks done by the community.
• deleted 3y ago
  [deleted]
• byyoung3 3y ago
  how many shots?
• christianqchung 3y ago
  >"The Mixtral 8x22b is a state-of-the-art language model that leverages a cutting-edge architecture called Mixture of Experts (MoE). In this setup, the model comprises eight individual "expert" models, each with 7 billion parameters." Each with... how many parameters? >"Key Features: 8 expert models, each with 7 billion parameters (totaling 56 billion parameters)" 8x7?? not 8x22?? >"For instance, on the challenging ARC reasoning task, the Mixtral 8x22b achieved an impressive score of 70.5, surpassing even GPT-4's reported score of 68." Not a single score on any benchmark shown from GPT4 is 68. The ARC-C score reported is 96.3. What! Am I going crazy, or is this entire article GPT slop?
  • byyoung3 3y ago
    stock gpt-4 0 shot human eval is 67?