7 ms·

"I was shown a prompt and told the internal research model had simply been given the problem statement. Levent had been told by Sebastien “very little human inp

"I was shown a prompt and told the internal research model had simply been given the problem statement. Levent had been told by Sebastien “very little human input” had been used. This turned out not to be true. Over the course of the call, as members of their team sent Sebastien corrections and details over their internal chat, it emerged that an entire team had been working on the problem, that this was one of a number of things that was tried, that work had started on the unforced problem, that the team first set the model on easier problems, including Euler, that even the prompt that had been shown to me had been written by prompting Codex, and that an insane amount of compute had been used." - This, from Tristan Buckmaster's writeup yesterday, indicates to me that there was more than incidental inspiration from Alpoge and Buckmaster.

tedsanders 27d ago
All of those statements sound true, based on what I've heard. - "very little human" input feels ambiguous, and if someone spends a few days prompting a model to solve a super hairy problem requiring a 100-page proof, I can understand reasonable people interpreting that as both "very little" and "not very little" human input - it's all true that a team worked on this, a bunch of compute was burned, and the problem was solved in stages and pieces I'm not sure how any of this provides evidence that OpenAI took any of their work. As evidence against, we never looked at any of their ChatGPT conversations and our model's proof is quite different from theirs. (I work at OpenAI, but not on the team that did this proof.)
- enraged_camel 27d ago
  >> I'm not sure how any of this provides evidence that OpenAI took any of their work. Sorry, but the burden of proof lies in the other direction: OpenAI needs to definitively prove that their agents did not look at the existing work that was about to be published. Otherwise OpenAI simply stole the glory and the spotlight (and I'm being charitable here).
  - fc417fc802 27d ago
    That's entirely unreasonable. Allegations of malfeasance always need to be backed up by evidence.
    - nulld3v 27d ago
      Nobody except OpenAI knows whether or not OpenAI trained on their data. So the burden remains on OpenAI here.
      - fc417fc802 27d ago
        That is an absurd and entirely untenable position that breaks with approximately all western conventions. Only the CIA knows whether or not they're actively covering up reptilian space aliens exerting control over the US government. Therefore the burden of proof remains on the CIA to prove that they are not actively participating in such a scheme.
        nulld3v 27d ago
        I don't understand, OpenAI can just say: "yes/no we did/did not train on your data". It's not a hard question to answer, and it is a question that OpenAI should be able to answer for all data we feed into ChatGPT.
        fc417fc802 27d ago
        > It's not a hard question to answer I didn't realize you had insider knowledge about their systems. Do please explain for the class. As I understand it they will only have trained on his data if he consented to it. Do you have evidence that they do otherwise?
        Timon3 27d ago
        This whole discussion is about evidence. That's not proof and it is not certain, but it is evidence pointing into the direction that OpenAI might be doing something that they're strongly incentivized to do. What kind of "evidence" do you see as necessary?
- whimsicalism 27d ago
  I'm confused, your employer very directly stated that they are unable to confirm that the model was not trained on the conversations.
  - tristanj 27d ago
    The models are trained on the conversations of hundreds of millions of people. ChatGPT has several billion conversations every day. I estimate that the model that solved Navier–Stokes was trained on data from nearly a trillion conversations. It's unknowable and not possible to prove if any one specific conversation was the key to solving Navier–Stokes.
    - DetroitThrow 27d ago
      >It's unknowable and not possible to prove if any one specific conversation was the key to solving Navier–Stokes. If the conversation was in the training set, there's a high likelihood that the small set of conversations related to solving Navier-Stokes was used by the model. I get Astra to still quote some of my friends' books or blogposts nearly verbatim on certain niche issues. Much more importantly, we _can_ determine whether a conversation was used in the training data. And if it was, it gives us a great idea whether that logic was captured in reasoning for a novel problem never yet solved. Given that you don't see any of this as below the belt according to your other comments, maybe your contribution here is more for yourself than a fair conversation about attribution.
      - tristanj 27d ago
        The flaw with this line of reasoning is that Buckmaster and Alpöge only had a partially completed proof of a weaker version of the Navier-Stokes problem. OpenAI's internal model solved the full, harder problem. This means the key information needed to bridge the gap was not present in Buckmaster and Alpöge's chat history. You might retort that ChatGPT used the training data to copy their approach, but the approach Buckmaster and Alpöge chose was already published by Luis and Diego in 2023 and in every frontier model's training set.
- DetroitThrow 27d ago
  It's unclear if you're suggesting that OpenAI did not train on their input or use their chats as inputs to training on a model that found the solution. Let's not provide an Elizabeth Holmes-esque interview where the question is dodged and words gain new meaning. The question can be answered with "Yes, we trained on their conversations" or "No, we did not train on their conversations". I'm not coming from a place of distrust here. This should just be definitively answerable given the weight of the claims here. Surely between you, your lawyers, and other members of your team you can just clear this part up.
- carzilla 26d ago
  You don’t work on the team that did the proof yet you can with certainty make all of these claims?

tedsanders 27d ago

All of those statements sound true, based on what I've heard. - "very little human" input feels ambiguous, and if someone spends a few days prompting a model to solve a super hairy problem requiring a 100-page proof, I can understand reasonable people interpreting that as both "very little" and "not very little" human input - it's all true that a team worked on this, a bunch of compute was burned, and the problem was solved in stages and pieces I'm not sure how any of this provides evidence that OpenAI took any of their work. As evidence against, we never looked at any of their ChatGPT conversations and our model's proof is quite different from theirs. (I work at OpenAI, but not on the team that did this proof.)

enraged_camel 27d ago
>> I'm not sure how any of this provides evidence that OpenAI took any of their work. Sorry, but the burden of proof lies in the other direction: OpenAI needs to definitively prove that their agents did not look at the existing work that was about to be published. Otherwise OpenAI simply stole the glory and the spotlight (and I'm being charitable here).
- fc417fc802 27d ago
  That's entirely unreasonable. Allegations of malfeasance always need to be backed up by evidence.
  - nulld3v 27d ago
    Nobody except OpenAI knows whether or not OpenAI trained on their data. So the burden remains on OpenAI here.
    - fc417fc802 27d ago
      That is an absurd and entirely untenable position that breaks with approximately all western conventions. Only the CIA knows whether or not they're actively covering up reptilian space aliens exerting control over the US government. Therefore the burden of proof remains on the CIA to prove that they are not actively participating in such a scheme.
      - nulld3v 27d ago
        I don't understand, OpenAI can just say: "yes/no we did/did not train on your data". It's not a hard question to answer, and it is a question that OpenAI should be able to answer for all data we feed into ChatGPT.
        fc417fc802 27d ago
        > It's not a hard question to answer I didn't realize you had insider knowledge about their systems. Do please explain for the class. As I understand it they will only have trained on his data if he consented to it. Do you have evidence that they do otherwise?
        Timon3 27d ago
        This whole discussion is about evidence. That's not proof and it is not certain, but it is evidence pointing into the direction that OpenAI might be doing something that they're strongly incentivized to do. What kind of "evidence" do you see as necessary?
whimsicalism 27d ago
I'm confused, your employer very directly stated that they are unable to confirm that the model was not trained on the conversations.
- tristanj 27d ago
  The models are trained on the conversations of hundreds of millions of people. ChatGPT has several billion conversations every day. I estimate that the model that solved Navier–Stokes was trained on data from nearly a trillion conversations. It's unknowable and not possible to prove if any one specific conversation was the key to solving Navier–Stokes.
  - DetroitThrow 27d ago
    >It's unknowable and not possible to prove if any one specific conversation was the key to solving Navier–Stokes. If the conversation was in the training set, there's a high likelihood that the small set of conversations related to solving Navier-Stokes was used by the model. I get Astra to still quote some of my friends' books or blogposts nearly verbatim on certain niche issues. Much more importantly, we _can_ determine whether a conversation was used in the training data. And if it was, it gives us a great idea whether that logic was captured in reasoning for a novel problem never yet solved. Given that you don't see any of this as below the belt according to your other comments, maybe your contribution here is more for yourself than a fair conversation about attribution.
    - tristanj 27d ago
      The flaw with this line of reasoning is that Buckmaster and Alpöge only had a partially completed proof of a weaker version of the Navier-Stokes problem. OpenAI's internal model solved the full, harder problem. This means the key information needed to bridge the gap was not present in Buckmaster and Alpöge's chat history. You might retort that ChatGPT used the training data to copy their approach, but the approach Buckmaster and Alpöge chose was already published by Luis and Diego in 2023 and in every frontier model's training set.
DetroitThrow 27d ago
It's unclear if you're suggesting that OpenAI did not train on their input or use their chats as inputs to training on a model that found the solution. Let's not provide an Elizabeth Holmes-esque interview where the question is dodged and words gain new meaning. The question can be answered with "Yes, we trained on their conversations" or "No, we did not train on their conversations". I'm not coming from a place of distrust here. This should just be definitively answerable given the weight of the claims here. Surely between you, your lawyers, and other members of your team you can just clear this part up.
carzilla 26d ago
You don’t work on the team that did the proof yet you can with certainty make all of these claims?