3 ms·
Are there any evals or benchmarks of this model?
by ilaksh 2y ago
Are there any evals or benchmarks of this model?
- danielhanchen 2y agoThere's no extensive benchmarks, but I did do a Flappy Bird pass@3 test just to show that a 1.58bit model does in fact work well! The goal was to showcase that MoEs quantized down to 1.58bit without any further training does in fact work!
- ilaksh 2y agoRight and congrats on all of that. But I need to know the evals to know whether this actually makes sense versus alternatives.