3 ms·
Isn't there an obvious conflict of interest when the model maker is also the creator of a benchmark? I think at the very least it should be from a separate busi
by dcreater 1y ago
Isn't there an obvious conflict of interest when the model maker is also the creator of a benchmark? I think at the very least it should be from a separate business entity under the non profit or from the non profit holding entity itself
- gwd 1y agoI don't think it's necessarily bad to have the benchmark, but the graphs of Gemini and Claude doing worse than o3 did kind of leave a bad taste in my mouth. "Oh look, your models are worse than ours at this very important metric that we just made up! How terrible!"
- progbits 1y agoIt's also barely better, within the error bar on some metrics. So if this is the best they can do on their own benchmark that's pretty pathetic and shows they have no edge anymore.