4 ms·
VibeBench: Measuring 1k Engineers' Opinions of New Models
- mhi3 5mo ago"Published benchmarks are gamed, optimized, and overfit, and no longer yield a useful signal." Is this true? But I love this concept!
- jpschroeder 5mo agoOh very true. Benchmaxxing itself is basically gaming them.
- memoryleakgame 5mo ago800 commits in a year...
- ramon156 5mo agoLove the idea! Page is incredibly slow on mobile, probably the avatars