4 ms·
I would say it is trust worthy because if it were found to be gamed then Anthropic’s reputation would crater. But, we found out that OpenAI is/was gaming bench
by chairhairair 2y ago
I would say it is trust worthy because if it were found to be gamed then Anthropic’s reputation would crater.
But, we found out that OpenAI is/was gaming benchmarks (https://news.ycombinator.com/item?id=42761648 https://news.ycombinator.com/item?id=42761648) and that seems to be forgotten history now - so I don’t know.
- diggan 2y ago> I would say it is trust worthy because if it were found to be gamed then Anthropic’s reputation would crater. But on the other hand, how would we found out that they've gamed the numbers, if they were gamed? Unless you work at Anthropic and have abnormally high ethics/morals, or otherwise private insight into their business, sounds like we wouldn't be able to find out regardless.