3 ms·
Its funny to see benchmarks where they omit the top performing models like O3 (Which is the best model in many benchmarks currently) and Gemini Pro/Claude 3.7.
by gizmodo59 1y ago
Its funny to see benchmarks where they omit the top performing models like O3 (Which is the best model in many benchmarks currently) and Gemini Pro/Claude 3.7.
- daveguy 1y agoThose are much much larger models, and they are proprietary. Those model providers just don't have the distilled versions identified and available. Notice most of the models they are comparing with are 7B models. The exception is also an open weights model (Qwen-2.5-32B-RL-Zero). Even with 32B parameters the MiMo-7B outperforms it.
- erikig 1y agoI believe the goal here is to compare them against similar models that are being optimized to run offline or on mobile hardware.