5 ms·

> Buried under the drama is the fact that OpenAI is claiming that an internal model they’ve been training for less than two weeks is more than twice as capable

> Buried under the drama is the fact that OpenAI is claiming that an internal model they’ve been training for less than two weeks is more than twice as capable in mathematics as Astra. Is this buried under the drama or are the major OpenAI twitter accounts from the people involved in the drama desperately attempting to make this the story after everything else obviously got away from them?

ameliaquining 1mo ago
I don't know what anyone's been saying on Twitter and I don't care. If it's really true that there's a model out there that's that capable two weeks after the start of training, then that's objectively a much bigger deal than a priority dispute, even if the latter involves juicy allegations of espionage and skulduggery.
- 20k 1mo ago
  It isn't a priority dispute, the more concerning allegation is that OpenAI may be training their models on prompts that mathematicians were using to solve this problem, and then surprise surprise OpenAI were able to replicate that work in their latest model What we're really looking at is seemingly a massive plagiarism scandal, which especially brings a lot of the past results into question If OpenAI is training models on researchers' prompts, and then threatening them into staying quiet about it, who knows if anything that's been announced is genuine - or just theft? Edit: OpenAI have admitted they were training on prompts at the time they made their breakthrough https://mastodon.social/@tristanbuckmaster/117236471352470303 https://mastodon.social/@tristanbuckmaster/11723647135247030...
  - ameliaquining 1mo ago
    If you're alleging that they don't actually have a highly capable model and the work they're attributing to it was actually plagiarized from human mathematicians, well, that would be big if true, but I'd be inclined to take the other side of that bet. With most previous splashy AI results, others have subsequently used the model to do other things around the same difficulty level. Also, it would still be necessary to explain why all these famous open problems are suddenly falling like dominoes, if it's not AI solving them. If you're saying that the question of whether they actually have a highly capable model is less important than the question of whether there's a plagiarism scandal, I continue to disagree.
    - 20k 1mo ago
      The issue is that if OpenAI is training on prompts generally, what we really have is the first fully automated luxury plagiarism machine. In that it isn't able to genuinely solve problems, but merely steal the work that other mathematicians have been putting into prompts, and regurgitating that to other users as its own work. That makes them incredibly less useful as research tools The fact that this plagiarism scandal exists underpins the idea that there's actually a mass theft going on, and that these models aren't nearly as capable as is it would seem
      - lotsofpulp 1mo ago
        Do OpenAI’s T&Cs that users accept not allow them to train on prompts people enter into it?
        20k 1mo ago
        OpenAI's T&Cs let them steal your children I'd suspect, that doesn't make it morally correct
        lotsofpulp 1mo ago
        Why would you suspect that? Stealing children is illegal, and involves violating the rights of unwilling parties, whereas prompting openAI (or any LLM) is a business transaction, in which the transfer of money and data is legal.
        fwip 1mo ago
        Terms and conditions are almost entirely about the company doing things that would otherwise be illegal.
        lotsofpulp 1mo ago
        I cannot parse that sentence. If you think a business’s terms and conditions change the illegality of an action, you should consider researching that assumption and speaking to some lawyers.
  - doctoboggan 1mo ago
    I think all he big labs are pretty explicit about when they do and don't train on customer prompts. Is the accusation here that OpenAI trained on prompts when they claimed not to? Or were the mathematicians using one of the interfaces that allows OpenAI to train on the customer data?
  - user43928 1mo ago
    All that I have seen OpenAI employees "admit" is that if you press the Thumbs up button on a response, this can be used as a signal for training. That's it. The rest appears to be wild speculation.
    - jsw97 1mo ago
      Yeah this is a land mine. Even if you opt out of them training on your conversations, giving “feedback on a new version” or answering “how are we doing” or whatever can slurp up all relevant context, which if you think about it can probably be construed to include memories, into the belly of their flying saucer. At least the last time I checked their terms. Never ever touch those requests. If you get a side by side comparison just resend the prompt.
    - lackoftactics 1mo ago
      do you even needs thumbs up? I've been long suspecting that code models get better because they use our data and our results from feedback, status codes, green tests for reinforcement learning
      - user43928 1mo ago
        I was thinking that the training is more curated, so that the methods are learned from experts, and that measurably successful behavior is reinforced. Throwing in random chats with some sentiment analysis doesn't seem like the most promising method to me, but I can only speculate.
        lackoftactics 1mo ago
        hmm, maybe not sentiments, running commands can produce binary results to reinforce, but that's also speculation
  - TZubiri 1mo ago
    >It isn't a priority dispute, the more concerning allegation is that OpenAI may be training their models on prompts that mathematicians were using to solve this problem, I don't want to get epistemiological, but these aren't allegations, and your use of "may be" is more of lack of knowledge on how OpenAI and ChatGPT work. Read the Terms of Service, this is not a secret, OAI doesn't deny it, usage of ChatGPT through the web interface or through its App, including Codex, are shared with OAI and used to train future models. This is one of the ways in which ChatGPT works and improves, welcome to the subject.
  - TZubiri 1mo ago
    >It isn't a priority dispute, the more concerning allegation is that OpenAI may be training their models on prompts that mathematicians were using to solve this problem, I don't want to get epistemiological, but these aren't allegations, and your use of "may be" is more of lack of knowledge on how OpenAI and ChatGPT work. Read the Terms of Service, this is not a secret, OAI doesn't deny it, usage of ChatGPT through the web interface or through its App, including Codex, are shared with OAI and used to train future models. This is one of the ways in which ChatGPT works and improves, it's not something we are learning now, it's something that was always known, welcome to the subject.
  - zem 1mo ago
    even apart from the plagiarism issue, what sort of slimy company thinks "oh, here's someone using our models to work on a problem, let's throw more compute at it and scoop them"?
    - flir 1mo ago
      Training on prompts I can understand - that's kinda baked into the premise, and they've been explicit about it. Publication, though? Slimy is right. But the interesting question to me is: once they had a solution, what should they have done with it? I see two choices: bury it, or contact the mathematicians whose prompts they were listening in on.
- an0malous 1mo ago
  The things you don’t care about are highly relevant to that claim
  - ameliaquining 1mo ago
    Elaborate?
    - an0malous 1mo ago
      If the model started from work that Tristan Buckmaster had already done, and was aided by an entire team of mathematicians as he alleged, then it's not as capable as you seem to be precluding
      - ameliaquining 1mo ago
        See https://news.ycombinator.com/item?id=49614484 https://news.ycombinator.com/item?id=49614484.
- HarHarVeryFunny 1mo ago
  An Astra sized model takes months to train - they are not saying 2 weeks to train from scratch. The only only interpretation of this "2 weeks" claim that is consistent with reality is that they mean 2 weeks of additional training on top of whatever their starting point was, so it's more like this: |--- N months of base model training -->|--- X months of post-training -->(Astra?)|-- 2 weeks more training (on Navier-Stokes adjacent material, perhaps)--> this "new" model

ameliaquining 1mo ago

I don't know what anyone's been saying on Twitter and I don't care. If it's really true that there's a model out there that's that capable two weeks after the start of training, then that's objectively a much bigger deal than a priority dispute, even if the latter involves juicy allegations of espionage and skulduggery.

20k 1mo ago
It isn't a priority dispute, the more concerning allegation is that OpenAI may be training their models on prompts that mathematicians were using to solve this problem, and then surprise surprise OpenAI were able to replicate that work in their latest model What we're really looking at is seemingly a massive plagiarism scandal, which especially brings a lot of the past results into question If OpenAI is training models on researchers' prompts, and then threatening them into staying quiet about it, who knows if anything that's been announced is genuine - or just theft? Edit: OpenAI have admitted they were training on prompts at the time they made their breakthrough https://mastodon.social/@tristanbuckmaster/117236471352470303 https://mastodon.social/@tristanbuckmaster/11723647135247030...
- ameliaquining 1mo ago
  If you're alleging that they don't actually have a highly capable model and the work they're attributing to it was actually plagiarized from human mathematicians, well, that would be big if true, but I'd be inclined to take the other side of that bet. With most previous splashy AI results, others have subsequently used the model to do other things around the same difficulty level. Also, it would still be necessary to explain why all these famous open problems are suddenly falling like dominoes, if it's not AI solving them. If you're saying that the question of whether they actually have a highly capable model is less important than the question of whether there's a plagiarism scandal, I continue to disagree.
  - 20k 1mo ago
    The issue is that if OpenAI is training on prompts generally, what we really have is the first fully automated luxury plagiarism machine. In that it isn't able to genuinely solve problems, but merely steal the work that other mathematicians have been putting into prompts, and regurgitating that to other users as its own work. That makes them incredibly less useful as research tools The fact that this plagiarism scandal exists underpins the idea that there's actually a mass theft going on, and that these models aren't nearly as capable as is it would seem
    - lotsofpulp 1mo ago
      Do OpenAI’s T&Cs that users accept not allow them to train on prompts people enter into it?
      - 20k 1mo ago
        OpenAI's T&Cs let them steal your children I'd suspect, that doesn't make it morally correct
        lotsofpulp 1mo ago
        Why would you suspect that? Stealing children is illegal, and involves violating the rights of unwilling parties, whereas prompting openAI (or any LLM) is a business transaction, in which the transfer of money and data is legal.
        fwip 1mo ago
        Terms and conditions are almost entirely about the company doing things that would otherwise be illegal.
        lotsofpulp 1mo ago
        I cannot parse that sentence. If you think a business’s terms and conditions change the illegality of an action, you should consider researching that assumption and speaking to some lawyers.
- doctoboggan 1mo ago
  I think all he big labs are pretty explicit about when they do and don't train on customer prompts. Is the accusation here that OpenAI trained on prompts when they claimed not to? Or were the mathematicians using one of the interfaces that allows OpenAI to train on the customer data?
- user43928 1mo ago
  All that I have seen OpenAI employees "admit" is that if you press the Thumbs up button on a response, this can be used as a signal for training. That's it. The rest appears to be wild speculation.
  - jsw97 1mo ago
    Yeah this is a land mine. Even if you opt out of them training on your conversations, giving “feedback on a new version” or answering “how are we doing” or whatever can slurp up all relevant context, which if you think about it can probably be construed to include memories, into the belly of their flying saucer. At least the last time I checked their terms. Never ever touch those requests. If you get a side by side comparison just resend the prompt.
  - lackoftactics 1mo ago
    do you even needs thumbs up? I've been long suspecting that code models get better because they use our data and our results from feedback, status codes, green tests for reinforcement learning
    - user43928 1mo ago
      I was thinking that the training is more curated, so that the methods are learned from experts, and that measurably successful behavior is reinforced. Throwing in random chats with some sentiment analysis doesn't seem like the most promising method to me, but I can only speculate.
      - lackoftactics 1mo ago
        hmm, maybe not sentiments, running commands can produce binary results to reinforce, but that's also speculation
- TZubiri 1mo ago
  >It isn't a priority dispute, the more concerning allegation is that OpenAI may be training their models on prompts that mathematicians were using to solve this problem, I don't want to get epistemiological, but these aren't allegations, and your use of "may be" is more of lack of knowledge on how OpenAI and ChatGPT work. Read the Terms of Service, this is not a secret, OAI doesn't deny it, usage of ChatGPT through the web interface or through its App, including Codex, are shared with OAI and used to train future models. This is one of the ways in which ChatGPT works and improves, welcome to the subject.
- TZubiri 1mo ago
  >It isn't a priority dispute, the more concerning allegation is that OpenAI may be training their models on prompts that mathematicians were using to solve this problem, I don't want to get epistemiological, but these aren't allegations, and your use of "may be" is more of lack of knowledge on how OpenAI and ChatGPT work. Read the Terms of Service, this is not a secret, OAI doesn't deny it, usage of ChatGPT through the web interface or through its App, including Codex, are shared with OAI and used to train future models. This is one of the ways in which ChatGPT works and improves, it's not something we are learning now, it's something that was always known, welcome to the subject.
- zem 1mo ago
  even apart from the plagiarism issue, what sort of slimy company thinks "oh, here's someone using our models to work on a problem, let's throw more compute at it and scoop them"?
  - flir 1mo ago
    Training on prompts I can understand - that's kinda baked into the premise, and they've been explicit about it. Publication, though? Slimy is right. But the interesting question to me is: once they had a solution, what should they have done with it? I see two choices: bury it, or contact the mathematicians whose prompts they were listening in on.
an0malous 1mo ago
The things you don’t care about are highly relevant to that claim
- ameliaquining 1mo ago
  Elaborate?
  - an0malous 1mo ago
    If the model started from work that Tristan Buckmaster had already done, and was aided by an entire team of mathematicians as he alleged, then it's not as capable as you seem to be precluding
    - ameliaquining 1mo ago
      See https://news.ycombinator.com/item?id=49614484 https://news.ycombinator.com/item?id=49614484.
HarHarVeryFunny 1mo ago
An Astra sized model takes months to train - they are not saying 2 weeks to train from scratch. The only only interpretation of this "2 weeks" claim that is consistent with reality is that they mean 2 weeks of additional training on top of whatever their starting point was, so it's more like this: |--- N months of base model training -->|--- X months of post-training -->(Astra?)|-- 2 weeks more training (on Navier-Stokes adjacent material, perhaps)--> this "new" model