If you game the benchmark then you always get found out by your users. Yet the practice remains common in hardware. Outright lies are uncommon but misleading an

If you game the benchmark then you always get found out by your users. Yet the practice remains common in hardware. Outright lies are uncommon but misleading and cherry picked numbers are pretty much standard practice. The fact that misleading benchmarks don't even drive profit at Meta didn't seem to stop them doing the same thing, but perhaps this isn't very surprising. I imagine internal incentives are very similar. Unlike the hardware companies though, gaming the benchmark in LLMs seems to involve making the actual performance worse, so perhaps there is more hope that the practice will fade away in this market.