4 ms·
>They got the dataset from Epoch AI for one of the benchmarks and pinky swore that they wouldn't train on it Is anything here actually false or do you not like
by suddenlybananas 2y ago
>They got the dataset from Epoch AI for one of the benchmarks and pinky swore that they wouldn't train on it
Is anything here actually false or do you not like the conclusions that people may draw from it?
- tedsanders 2y agoThe false statement is “OpenAI got caught [gaming FrontierMath] a while back.” From a primary source: "OpenAI did not use FrontierMath data to guide the development of o1 or o3, at all.... we only downloaded FrontierMath for our evals long after the training data was frozen, and only looked at o3 FrontierMath results after the final announcement checkpoint was already picked." https://x.com/__nmca__/status/1882563755806281986 https://x.com/__nmca__/status/1882563755806281986
- suddenlybananas 2y agoWell, if we trust you. But you had the extra dataset and no one else did.