3 ms·
Or, 2b: the nerf is real, but benchmarks are gamed and models are trained to excel at them, yet fall flat in real world situations.
by imiric 11mo ago
Or, 2b: the nerf is real, but benchmarks are gamed and models are trained to excel at them, yet fall flat in real world situations.
- metalliqaz 11mo agoI mostly stay out of the LLM space but I thought it was an open secret already that the benchmarks are absolutely gamed.