5 ms·
>> Internal OpenAI models are reportedly solving open problems at a surprisingly fast rate[2] Maybe I'm failing to read that graph properly but the y axis says
by YeGoblynQueenne 1mo ago
>> Internal OpenAI models are reportedly solving open problems at a surprisingly fast rate[2]
Maybe I'm failing to read that graph properly but the y axis says "pass rate" and it only goes up to 0.5. That would mean every single problem is at most half-solved.
I don't know what that means though. What is "0.5 pass rate" in the context of "open math problems" (as in the graph title)?
- red75prime 1mo agoI guess it's a fraction of problems on which a model produces a LEAN proof or a counterexample.
- YeGoblynQueenne 1mo agoWouldn't they just list the number of problems solved then?
- dekhn 1mo agorates beat counts almost always.
- YeGoblynQueenne 1mo agoYou're assuming too much from a single graph with vaguely named axes. I'll wait until there's more concrete information.