Competition is how the world moves forward. I'm super glad small and big players have competitive models. The thing that makes me a bit sad is how announcement

Competition is how the world moves forward. I'm super glad small and big players have competitive models. The thing that makes me a bit sad is how announcements show the benchmarks but they way they test is tweaked to make their metrics favorable. They aren't apple to apple benchmarks across different paper publications. Super grateful that they openly share the weights and code with Apache license. 25 shot - is having 25 tries and selecting the best answer. Is anyone working on an open benchmark where they take the major models and compare them apples to apples.