4 ms·

I didn't see anyone else ask this but.. isn't the FrontierMath dataset compromised now? At the very least OpenAI now knows the questions if not the answers. I w

I didn't see anyone else ask this but.. isn't the FrontierMath dataset compromised now? At the very least OpenAI now knows the questions if not the answers. I would expect that the next iteration will "magically" get over 80% on the FrontierMath test. I imagine that experiment was pretty closely monitored.

jvanderbot 2y ago
I figured their model was independently evaluated against the questions/answers. That's not to say it's not compromised by "Here's a bag of money" type methods, but I don't even think it'd be a reasonable test if they just handed over the dataset.
- upghost 2y ago
  I'm sure it was independently evaluated, but I'm sure the folks running the test were not given an on-prem installation of ChatGPT to mess with. It was still done via API calls, presumably through the chat interface UI. That means the questions went over the fence to OpenAI. I'm quite certain they are aware of that, and it would be pretty foolish not to take advantage of at least knowing what the questions are.
  - jvanderbot 2y ago
    Now that you put it that way, it is laughably easy.
  - ls612 2y ago
    Depending on the plan the researchers used they may have contractual protections against OpenAI training on their inputs.
    - upghost 2y ago
      Sure, but given the resourcing at OpenAI, it would not be hard to clean[1] the inputs. I'm just trying to be realistic here, there are plenty of ways around contractual obligations and a significant incentive to do so. [1]: https://en.wikipedia.org/wiki/Clean-room_design https://en.wikipedia.org/wiki/Clean-room_design
    - xigoi 2y ago
      Do you think ClosedAI cares about that? It’s not like anyone can tell if they violate the contract.
optimalsolver 2y ago
This was my first thought when I saw the results: https://news.ycombinator.com/item?id=42473470 https://news.ycombinator.com/item?id=42473470
- upghost 2y ago
  Insightful comment. The thing that's extremely frustrating is look at all the energy poured into this conversation around benchmarks. There is a fundamental assumption of honesty and integrity in the benchmarking process by at least some people. But when the dataset is compromised and generation N+1 has miraculous performance gains, how can we see this as anything other than a ploy to pump up valuations? Some people have millions of dollars at stake here and they don't care about the naysayers in the peanut gallery like us.
  - optimalsolver 2y ago
    It's sadly inevitable that when billions in funding and industry hype are tied to performance on a handful of benchmarks, scores will somehow, magically, continue to go up. Needless to say, it doesn't bring us any closer to AGI. The only solution I see here is people crafting their own, private benchmarks that the big players don't care about enough to train on. That, at least, gives you a clearer view of the field.
    - upghost 2y ago
      Not sure why your comment was downvoted, but it certainly shows the pressure going against people who point out fundamental flaws. This is pushing us towards "AVI" rather than AGI-- "Artificially Valued Intelligence". The optimization function here is around the market. I'm being completely serious. You are correct, despite the downvotes, that this could not be pushing us towards AGI because if the dataset is leaked you can't claim the G-- generalizability. The point of the benchmark is to lead is to believe that this is a substantial breakthrough. But a reasonable person would be forced to conclude that the results are misleading to due to optimizing around the training data.

jvanderbot 2y ago

I figured their model was independently evaluated against the questions/answers. That's not to say it's not compromised by "Here's a bag of money" type methods, but I don't even think it'd be a reasonable test if they just handed over the dataset.

upghost 2y ago
I'm sure it was independently evaluated, but I'm sure the folks running the test were not given an on-prem installation of ChatGPT to mess with. It was still done via API calls, presumably through the chat interface UI. That means the questions went over the fence to OpenAI. I'm quite certain they are aware of that, and it would be pretty foolish not to take advantage of at least knowing what the questions are.
- jvanderbot 2y ago
  Now that you put it that way, it is laughably easy.
- ls612 2y ago
  Depending on the plan the researchers used they may have contractual protections against OpenAI training on their inputs.
  - upghost 2y ago
    Sure, but given the resourcing at OpenAI, it would not be hard to clean[1] the inputs. I'm just trying to be realistic here, there are plenty of ways around contractual obligations and a significant incentive to do so. [1]: https://en.wikipedia.org/wiki/Clean-room_design https://en.wikipedia.org/wiki/Clean-room_design
  - xigoi 2y ago
    Do you think ClosedAI cares about that? It’s not like anyone can tell if they violate the contract.

optimalsolver 2y ago

This was my first thought when I saw the results: https://news.ycombinator.com/item?id=42473470 https://news.ycombinator.com/item?id=42473470

upghost 2y ago
Insightful comment. The thing that's extremely frustrating is look at all the energy poured into this conversation around benchmarks. There is a fundamental assumption of honesty and integrity in the benchmarking process by at least some people. But when the dataset is compromised and generation N+1 has miraculous performance gains, how can we see this as anything other than a ploy to pump up valuations? Some people have millions of dollars at stake here and they don't care about the naysayers in the peanut gallery like us.
- optimalsolver 2y ago
  It's sadly inevitable that when billions in funding and industry hype are tied to performance on a handful of benchmarks, scores will somehow, magically, continue to go up. Needless to say, it doesn't bring us any closer to AGI. The only solution I see here is people crafting their own, private benchmarks that the big players don't care about enough to train on. That, at least, gives you a clearer view of the field.
  - upghost 2y ago
    Not sure why your comment was downvoted, but it certainly shows the pressure going against people who point out fundamental flaws. This is pushing us towards "AVI" rather than AGI-- "Artificially Valued Intelligence". The optimization function here is around the market. I'm being completely serious. You are correct, despite the downvotes, that this could not be pushing us towards AGI because if the dataset is leaked you can't claim the G-- generalizability. The point of the benchmark is to lead is to believe that this is a substantial breakthrough. But a reasonable person would be forced to conclude that the results are misleading to due to optimizing around the training data.