5 ms·

I think it's a useful analogy to compare OpenAI to a human collaborator. These researchers willingly collaborated with an OpenAI model, giving it ideas, and Ope

by nezi 23d ago

I think it's a useful analogy to compare OpenAI to a human collaborator. These researchers willingly collaborated with an OpenAI model, giving it ideas, and OpenAI provided useful replies. Then, OpenAI goes ahead and publishes work along the lines of this collaboration, without attributing the researchers. If OpenAI was in fact a human researcher, this would be highly unethical. Now, OpenAI is claiming that the model it used to generate the result was not trained on these collaborative communications with the researcher. This is a technical argument that is impossible to verify as an OpenAI outsider, and probably difficult to verify even for internal OpenAI employees. Provenance is hard to track - you would hope OpenAI has very good tools for this, but a full data trail of all inputs is difficult to trace through. Another interesting thing to consider is if instead of OpenAI doing this, it was another research mathematician A using an OpenAI model just like the internal group at OpenAI did to publish these results. What if the model A used was trained with unpublished communications with other researchers B who were working on the same problem? Should researcher A technically include B as coauthors? How could they do this when they do not know the communications B had with OpenAI? In this scenario OpenAI, as a middle man, has laundered information from B to A, stripping out attribution. A scooped B without even knowing it!

jameslars 23d ago
> Provenance is hard to track - you would hope OpenAI has very good tools for this, but a full data trail of all inputs is difficult to trace through. What would OpenAIs incentive for this be? They've gotten away with scraping everything and getting it ruled fair use. It seems like willful ignorance is an affirmative defense today. Why would they want to have some sort of audit trail that could prove otherwise?
- deleted 23d ago
  [deleted]
jjwiseman 23d ago
First, OpenAI is not claiming that the model wasn't trained on those sessions. What they've said is “We (the researchers and the agents) did not see any of their work through any means until they released it publicly — in particular, no specific user data was accessed in order to solve this problem.” and “We did not use their prompts or proofs to prompt our models or direct our agents.” and “While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models.” They also said “Our effort began on September 1st after hearing a rumor which we later realized was related to Levent Alpöge … and Tristan Buckmaster….” They say the rumor was that two Millennium Prize problems had been resolved, and that this prompted them to launch "an effort to evaluate it on all open Millennium Prize problems and a few other high-impact problems." It's not obvious to me that's an unethical thing to do, if it happened as they described.
- efxhoy 23d ago
  > we cannot rule out that de-identified data derived from their usage of our products helped improve our models.” implied the humans sessions could have been (and probably were, why wouldn’t they be?) in the training set? If I was trying to make a model smarter and I had transcripts from the smartest mathematicians in the world I’d make sure the model trained on them.
- tedsanders 23d ago
  We were also curious and we looked further into this. We've determined it was impossible for Dr. Buckmaster’s Codex prompts over the last two months to have influenced the system in any way, including training. This goes beyond what we said earlier, when we were less sure. If prompts were submitted earlier than that and training was not opted out, there may be a chance they made their way into our training pipeline in some form. But this would be a droplet in an ocean and unlikely to have made any difference, in my opinion. (I work at OpenAI.) Source for the updated claim: https://www.nytimes.com/2026/09/10/science/tristan-buckmaster-openai-math-navier-stokes.html https://www.nytimes.com/2026/09/10/science/tristan-buckmaste...
  - nsagent 23d ago
    Can you speak to why in both cases, the problems OpenAI's models solved used the same techniques the mathematicians were exploring, which also happened to be niche approaches to the problem. As an NLP researcher myself, I find that coincidence highly suspect unless the models focused most of their attempts on the predominant approaches (they are trained for MLE after all).
    - fhub 23d ago
      I think for OpenAI to win back some hearts and minds here we should have the option to retrospectively turn off "Help improve our AI models". i.e. Any new model trained would exclude all those user's sessions. This could be technically hard but I'm sure an intelligent AI model could work out how to do it :-) ChatGPT agrees with this too. https://chatgpt.com/share/6aa31959-b0e8-83ec-bee6-851ed18d4594 https://chatgpt.com/share/6aa31959-b0e8-83ec-bee6-851ed18d45...
cjf101 23d ago
If the model was trained proper to the conversation with the researcher took place, there'd be no question of tainting the results. But if any amount of training on the model took place afterward, then yes, everything is thrown into doubt (a core problem with considering anything "original" from a model because of how >a % of everything ever written has been used a corpus for the training).
mcmcmc 23d ago
> I think it's a useful analogy to compare OpenAI to a human collaborator. Frankly I don’t buy this. It’s not a human or a collaborator. It’s a tool. This is like saying it’s not Microsoft’s fault if they extract a bunch of data from people’s Excel sheets because they willingly put it into the program. Anthropomorphizing software is ignorant and foolhardy
- nezi 23d ago
  Tools don’t turn around and scoop you. What OpenAI did here was use the same tool that the researcher did which might have coupled their work together.
  - mcmcmc 23d ago
    You’re right, they don’t. It was scooped by the humans at OpenAI who published the paper. The tool they used to do it isn’t that relevant.
    - fn-mote 23d ago
      > isn’t that relevant “Might not be” that relevant. You’re dismissing the whole controversy without addressing why it’s controversial.
      - mcmcmc 23d ago
        I’m not dismissing the controversy. I’m arguing against shifting the blame away from the culpable parties. It’s a novel form of theft but thats still what it is.
- xdavidliu 23d ago
  i think this line of argument is outdated
  - mcmcmc 23d ago
    Care to explain why?
jimmydddd 23d ago
So, at my company (and most companies I think), we use confidential in-house versions of the AI software. We don't want any confidential information leaking into the public realm. Are these scientists doing that, or are they just using the public version of the software?
- tecleandor 23d ago
  When you say "confidential in-house version", what are you referring to? Local models? Bedrock deployment with "guardrails"? A different thing?
  - AceyMan 23d ago
    Enterprise Agreements can have binding terms for this. When I launch the ChatGPT desktop app, and open the options pane it says "Corpname data is not used for OpenAI training". I would expect academic institutions to require equivalent contractual terms.
    - rainprincess 23d ago
      Sure but they could also rewrite your data to create synthetic reconstructions and many academics, sign up for their own accounts. For example, at school they can have an agreement with Gemini, but the student / academic could have bought an individual pro subscription to any other model provider.
    - buzer 23d ago
      Some of the recent statements have caused at least me to look those claims in a bit more nuanced light. In particular what does OpenAI consider to be "your data"? I would assume input (prompt) to be it at least. However it becomes more murky when you consider other aspects. Is output "your data"? Is the chain of thought that you are not even allowed to see? Can they use these and possibly even inputs to generate synthetic data that is then used? All of these would seem to be "your data", but when they are carefully only including certain aspects (like prompts) in their statements it starts to sound they want to hide something.
raincole 23d ago
The irony is that OpenAI got into this trouble only because they tried to play "nice". They told Buckmaster that he could publish the final result as the author as long as he removed Alpöge from the author list. They wanted to give Buckmaster a chance to be the one solved N-S problem. While this behavior is highly questionable, if OpenAI just published the final result without notifying Buckmaster first and simply cited his previous researches, there would be no ground for anyone to accuse OpenAI for anything. Their self-perceived "generosity" backfired dearly and I'm sure they'll never make the same mistake again. There is probably a policy forbidding any OpenAI employee to contact external researchers like that now.
- throwaway5752 23d ago
  The reality would be the same. They probably used prior session history between the research and Astra to train the internal model, and used it to front run-the researcher. This is the biggest self-own in the history of software. If you can relate to Pixar, OpenAI is Chick Hicks celebrating at the end of the Piston Cup and wondering why he's getting booed. The lack of self-awareness is something to behold, and says a lot about their corporate values.
- JumpCrisscross 23d ago
  > if OpenAI just published the final result without notifying Buckmaster first and simply cited his previous researched, there would be no ground for anyone to accuse OpenAI for anything Yes, there would? They would have left off Buckmaster as a precedent whose work they potentially relied on.
  - deleted 23d ago
    [deleted]
- ozgung 23d ago
  No. 1. Buckmaster contacted OpenAI first. Not the other way. 2. Giving the $1M bounty to a human mathematician for the effort and giving him credit would be excellent PR. They had already burned much more than $1M for the generation. Adding him as author also costs nothing. Purely pragmatical. 3. “As long as he removed Alpöge” part itself is against academic honesty by all means. 4. Buckmaster rejected fame and $1M only because doing (3) would be wrong. That’s a perfect example of honesty. That can’t be overstated. 5. After the rejection OpenAI guy (Sebastien) did’t say, “ok bye”. He threatened Buckmaster to “end his career”. 6. At that point OpenAI was not sure if they really used his conversations in their proof. He basically wanted to buy him to control any damage. 7. They omitted Buckmaster’s published work and any other related work in their References section. Also an academic malpractice. If you see generosity and niceness in all of this you are either too naive or your name is Sebastien.
deleted 23d ago
[deleted]
guelo 23d ago
Bad analogy. OpenAI spent millions on compute to get their result. This is more like if a billionaire heard of your promising mathematical lead and then gathered hundreds of top mathematicians to work on it.
- shye 23d ago
  In the current telling of this story, the billionaire is also giving his hired army copies of your notes he copied without permission. But the worst part in your analogy ain’t omitting the suspected spying and the intimidation that followed, but that your hypothetical mathematical philanthropist won’t be able to hire his army: unlike some OAI employees, no self-respecting mathematician would agree to such unethical task.
jltsiren 23d ago
I think it's better to ignore OpenAI here, because OpenAI didn't do anything. Academic research is a professional field in the traditional sense. Individual researchers are ultimately responsible for their actions. If some OpenAI employees violated academic norms while doing academic research, they should be judged by academic standards. Scooping someone else's result is immoral but not an outright violation of academic norms. But if you are in possession of relevant confidential information, you are expected to steer clear of the topic. It doesn't matter whether you actually used the confidential information to get your results, because outsiders can't know that. The mere fact that there is a plausible suspicion already puts your integrity into question. Tenured professors occasionally lose their jobs over similar scandals (but usually don't). If OpenAI wants to regain some goodwill, it should do a thorough investigation that may lead to firing the individuals in question. If it doesn't find sufficient evidence of wrongdoing to justify any disciplinary action, it probably doesn't gain any goodwill either (as it often happens with similar investigations at universities). And if OpenAI wants to be a trustworthy partner, it should transform into a company of boring gray bureaucrats who provide an essential service without competing with their customers.
- cj 23d ago
  [deleted - misunderstood!]
  - jltsiren 23d ago
    My point was that if someone is at fault, it's the individual OpenAI employees. Because they chose to engage in a professional field, they can't use "boss told me to do so" as a defense.
    - gregw2 22d ago
      Err, no. If someone is at fault, it is definitely the company (OpenAI) not the employee in this case. Just think, isn't the whole point of a company, of incorporating, is that the liability shifts from the employee to the firm? It is 100% fair to hold the company responsible, and doubly so when the questionable thing the employee is doing is something that A) benefits the company, and B) is on company time (aand with company resources.) I know you weren't making a legal-minutia point but even the high level principles involved are and should be the opposite of what you suggest. Companies may want to have their cake and eat it too (have employees do their dirty work but then push the blame on the employee) but that's not how it works and it's generally not a society you'd want to live in the more we head in that direction.
amelius 23d ago
The problem here is that OAI (and others) pretend or claim that this is uncharted legal territory, where in fact it is very simple. We have a machine that is fed data, and produces new data as a result. If that new data depends (in any way) on the fed data, then from a legal viewpoint it is derived from that data. Whether they anthropomorphize the operation performed by the machine does not matter. They can anthropomorphize when/if the law is updated to include such terms, but right now they certainly cannot.
- fn-mote 23d ago
  > in fact it is very simple Even if this opinion were backed up by a court ruling, it would definitely not be “simple”. It will be a very ugly case if it is ever litigated. A lot of money will be spent and no guarantee at all the plaintiff wins.
- magicalhippo 22d ago
  > If that new data depends (in any way) on the fed data, then from a legal viewpoint it is derived from that data. The "in any way" part is either so broad it makes everything derivative, or not, in which case things are no longer simple. If everything is derivative then it seizes to be meaningful. The words I write are derivative, I literally copied them from someone else, yet my sentences as a whole can be fully novel.
  - amelius 22d ago
    > If everything is derivative then it seizes to be meaningful. That's why we tolerate it for humans, and also because we cannot prove it. But yes, if you go too far in this, you will see legal consequences.
Eji1700 23d ago
> Provenance is hard to track Right, which is going to open a lot of doors to a lot of questions. I don't think there's any legal ramifications on this, just ethical ones about when and how you publish research, but it's yet another point in favor of "if provenance is hard to track, should we be using this for things where it needs to be". Obviously copyright/trademark is a huge discussion on this, and I could absolutely see this devolving into that as well with how certain findings wind up monetized. We have a response in this topic from someone claiming to be from OpenAI and linking an article where they, roughly, say "we are sure nothing from the 2 month period made its way into the solution". If that is true, that should mean it is provable, but leads to some more open ended questions like "well what data did it use then?". Is this still okay if someone close to the author did plug data into open AI and it extrapolated it? Obviously that's probably an unreasonable expectation for these models to track and prove, but it also used to be an unreasonable expectation to scrape every single piece of digital and physical info for consolidated data. If I opine to a friend on a park bench about a story I'm writing, do they get to pull it from the flock feed, shove it in the model, and then provide it to disney? Legally, right now, probably. But there's going to need to be a serious look at laws and standards. Or a major shift in what is and isn't discussed in public if literally every breath and move you make can become monetized.
rolandog 23d ago
Then there's the possibility of indirect training via modern spy devices ("smart" IoT devices like LG TV's) feeding the transcribed ambient conversation data for summarization to an agent [0]. [0]: https://youtu.be/6IFVTcM28KA https://youtu.be/6IFVTcM28KA
lmeyerov 23d ago
OpenAI says deidentified data from the private sessions go into training. (Well, explicitly said they will not rule that out.) That changes a lot of the conversation.
BobbyTables2 23d ago
The AI not being human doesn’t escape ethical consideration - OpenAI employees are culpable for what they build. This was academic research. Could just have easily been trade secrets and proprietary data.
timcobb 22d ago
> but a full data trail of all inputs is difficult to trace through. Great use case for AI agents
tw04 22d ago
> you would hope OpenAI has very good tools for this, but a full data trail of all inputs is difficult to trace through. They have a financial incentive not to track any of this, so why would they? OpenAI’s entire business model is predicated on stealing other people’s work and selling it to the masses.
Agentlien 22d ago
I think this move by OpenAI is crazy. At best, if all unconfirmed accusations are unfounded, they still heard a rumour that someone had solved a huge million dollar problem and was about to make a name for themselves. Then, they decided this was a good opportunity to pour millions of dollars into trying to snag the glory while the researchers were busy cleaning up their notes and polishing the announcement. That still sounds highly unethical.
- jasonfarnon 22d ago
  Would this be unethical if it was a human who heard rumors about a solution then attacked the problem, solved it and published first? Often knowing of the mere existence of a solution carries a lot of information--you would know the problem is accessible, you would expect clues in recent progress (the two Spanish researchers in this case), you would probably have a sense if the solution is a counterexample or positive proof, and so on. I think there are similar examples where we think of them as maybe unsporting but not quite unethical. Does it change if it's openAI and not a human?
  - timmytokyo 22d ago
    The problem with your counter-hypothetical is that not only is it unrealistic, it's utterly impossible. No human would be able to do in such a short timeframe what the LLM did. Part of what makes the OpenAI move so egregious is how bullying it was. It was the big guy coming along with their nearly infinite resources and squashing the little guy who's devoted a good chunk of his career to the problem.
    - jasonfarnon 22d ago
      Actually my hypothetical is completely realistic as I've been involved in such scenarios. It's unrealistic maybe for a millennium problem to come in on a rumor and still front-run but not at all for the many other problems we work on and which manifest our ethical code. If you're saying ethical rules change depending on the prize be clear about it, because I can see arguments that they change to favor either side.
enyone 22d ago
I think OP's analogy is bad. The difference of OpenAI when comparing to human collaborator is the possibility to replicate once learned skill. Imagine if any single human collaborator learns a skill it is immediately a skill of any human collaborator.
tgv 22d ago
> I think it's a useful analogy to compare OpenAI to a human collaborator. A human collaborator who stands to win or loose a couple of $100B.
pred_ 22d ago
Another scenario also just surfaced: https://news.ycombinator.com/item?id=49657499 https://news.ycombinator.com/item?id=49657499 – the construction used by Anthropic for the Jacobian Conjecture counterexample appears to have existed in an unpublished but publically available document. This document of course wasn't referenced in the announcement (nor did we ever get to see the prompt or reasoning traces).
- pfortuny 22d ago
  Wow, this is really interesting. Thanks.

by nezi 23d ago

jameslars 23d ago

> Provenance is hard to track - you would hope OpenAI has very good tools for this, but a full data trail of all inputs is difficult to trace through. What would OpenAIs incentive for this be? They've gotten away with scraping everything and getting it ruled fair use. It seems like willful ignorance is an affirmative defense today. Why would they want to have some sort of audit trail that could prove otherwise?

deleted 23d ago
[deleted]

jjwiseman 23d ago

First, OpenAI is not claiming that the model wasn't trained on those sessions. What they've said is “We (the researchers and the agents) did not see any of their work through any means until they released it publicly — in particular, no specific user data was accessed in order to solve this problem.” and “We did not use their prompts or proofs to prompt our models or direct our agents.” and “While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models.” They also said “Our effort began on September 1st after hearing a rumor which we later realized was related to Levent Alpöge … and Tristan Buckmaster….” They say the rumor was that two Millennium Prize problems had been resolved, and that this prompted them to launch "an effort to evaluate it on all open Millennium Prize problems and a few other high-impact problems." It's not obvious to me that's an unethical thing to do, if it happened as they described.

efxhoy 23d ago
> we cannot rule out that de-identified data derived from their usage of our products helped improve our models.” implied the humans sessions could have been (and probably were, why wouldn’t they be?) in the training set? If I was trying to make a model smarter and I had transcripts from the smartest mathematicians in the world I’d make sure the model trained on them.
tedsanders 23d ago
We were also curious and we looked further into this. We've determined it was impossible for Dr. Buckmaster’s Codex prompts over the last two months to have influenced the system in any way, including training. This goes beyond what we said earlier, when we were less sure. If prompts were submitted earlier than that and training was not opted out, there may be a chance they made their way into our training pipeline in some form. But this would be a droplet in an ocean and unlikely to have made any difference, in my opinion. (I work at OpenAI.) Source for the updated claim: https://www.nytimes.com/2026/09/10/science/tristan-buckmaster-openai-math-navier-stokes.html https://www.nytimes.com/2026/09/10/science/tristan-buckmaste...
- nsagent 23d ago
  Can you speak to why in both cases, the problems OpenAI's models solved used the same techniques the mathematicians were exploring, which also happened to be niche approaches to the problem. As an NLP researcher myself, I find that coincidence highly suspect unless the models focused most of their attempts on the predominant approaches (they are trained for MLE after all).
  - fhub 23d ago
    I think for OpenAI to win back some hearts and minds here we should have the option to retrospectively turn off "Help improve our AI models". i.e. Any new model trained would exclude all those user's sessions. This could be technically hard but I'm sure an intelligent AI model could work out how to do it :-) ChatGPT agrees with this too. https://chatgpt.com/share/6aa31959-b0e8-83ec-bee6-851ed18d4594 https://chatgpt.com/share/6aa31959-b0e8-83ec-bee6-851ed18d45...

cjf101 23d ago

If the model was trained proper to the conversation with the researcher took place, there'd be no question of tainting the results. But if any amount of training on the model took place afterward, then yes, everything is thrown into doubt (a core problem with considering anything "original" from a model because of how >a % of everything ever written has been used a corpus for the training).

mcmcmc 23d ago

> I think it's a useful analogy to compare OpenAI to a human collaborator. Frankly I don’t buy this. It’s not a human or a collaborator. It’s a tool. This is like saying it’s not Microsoft’s fault if they extract a bunch of data from people’s Excel sheets because they willingly put it into the program. Anthropomorphizing software is ignorant and foolhardy

nezi 23d ago
Tools don’t turn around and scoop you. What OpenAI did here was use the same tool that the researcher did which might have coupled their work together.
- mcmcmc 23d ago
  You’re right, they don’t. It was scooped by the humans at OpenAI who published the paper. The tool they used to do it isn’t that relevant.
  - fn-mote 23d ago
    > isn’t that relevant “Might not be” that relevant. You’re dismissing the whole controversy without addressing why it’s controversial.
    - mcmcmc 23d ago
      I’m not dismissing the controversy. I’m arguing against shifting the blame away from the culpable parties. It’s a novel form of theft but thats still what it is.
xdavidliu 23d ago
i think this line of argument is outdated
- mcmcmc 23d ago
  Care to explain why?

jimmydddd 23d ago

So, at my company (and most companies I think), we use confidential in-house versions of the AI software. We don't want any confidential information leaking into the public realm. Are these scientists doing that, or are they just using the public version of the software?

tecleandor 23d ago
When you say "confidential in-house version", what are you referring to? Local models? Bedrock deployment with "guardrails"? A different thing?
- AceyMan 23d ago
  Enterprise Agreements can have binding terms for this. When I launch the ChatGPT desktop app, and open the options pane it says "Corpname data is not used for OpenAI training". I would expect academic institutions to require equivalent contractual terms.
  - rainprincess 23d ago
    Sure but they could also rewrite your data to create synthetic reconstructions and many academics, sign up for their own accounts. For example, at school they can have an agreement with Gemini, but the student / academic could have bought an individual pro subscription to any other model provider.
  - buzer 23d ago
    Some of the recent statements have caused at least me to look those claims in a bit more nuanced light. In particular what does OpenAI consider to be "your data"? I would assume input (prompt) to be it at least. However it becomes more murky when you consider other aspects. Is output "your data"? Is the chain of thought that you are not even allowed to see? Can they use these and possibly even inputs to generate synthetic data that is then used? All of these would seem to be "your data", but when they are carefully only including certain aspects (like prompts) in their statements it starts to sound they want to hide something.

raincole 23d ago

The irony is that OpenAI got into this trouble only because they tried to play "nice". They told Buckmaster that he could publish the final result as the author as long as he removed Alpöge from the author list. They wanted to give Buckmaster a chance to be the one solved N-S problem. While this behavior is highly questionable, if OpenAI just published the final result without notifying Buckmaster first and simply cited his previous researches, there would be no ground for anyone to accuse OpenAI for anything. Their self-perceived "generosity" backfired dearly and I'm sure they'll never make the same mistake again. There is probably a policy forbidding any OpenAI employee to contact external researchers like that now.

throwaway5752 23d ago
The reality would be the same. They probably used prior session history between the research and Astra to train the internal model, and used it to front run-the researcher. This is the biggest self-own in the history of software. If you can relate to Pixar, OpenAI is Chick Hicks celebrating at the end of the Piston Cup and wondering why he's getting booed. The lack of self-awareness is something to behold, and says a lot about their corporate values.
JumpCrisscross 23d ago
> if OpenAI just published the final result without notifying Buckmaster first and simply cited his previous researched, there would be no ground for anyone to accuse OpenAI for anything Yes, there would? They would have left off Buckmaster as a precedent whose work they potentially relied on.
- deleted 23d ago
  [deleted]
ozgung 23d ago
No. 1. Buckmaster contacted OpenAI first. Not the other way. 2. Giving the $1M bounty to a human mathematician for the effort and giving him credit would be excellent PR. They had already burned much more than $1M for the generation. Adding him as author also costs nothing. Purely pragmatical. 3. “As long as he removed Alpöge” part itself is against academic honesty by all means. 4. Buckmaster rejected fame and $1M only because doing (3) would be wrong. That’s a perfect example of honesty. That can’t be overstated. 5. After the rejection OpenAI guy (Sebastien) did’t say, “ok bye”. He threatened Buckmaster to “end his career”. 6. At that point OpenAI was not sure if they really used his conversations in their proof. He basically wanted to buy him to control any damage. 7. They omitted Buckmaster’s published work and any other related work in their References section. Also an academic malpractice. If you see generosity and niceness in all of this you are either too naive or your name is Sebastien.

deleted 23d ago

[deleted]

guelo 23d ago

Bad analogy. OpenAI spent millions on compute to get their result. This is more like if a billionaire heard of your promising mathematical lead and then gathered hundreds of top mathematicians to work on it.

shye 23d ago
In the current telling of this story, the billionaire is also giving his hired army copies of your notes he copied without permission. But the worst part in your analogy ain’t omitting the suspected spying and the intimidation that followed, but that your hypothetical mathematical philanthropist won’t be able to hire his army: unlike some OAI employees, no self-respecting mathematician would agree to such unethical task.

jltsiren 23d ago

I think it's better to ignore OpenAI here, because OpenAI didn't do anything. Academic research is a professional field in the traditional sense. Individual researchers are ultimately responsible for their actions. If some OpenAI employees violated academic norms while doing academic research, they should be judged by academic standards. Scooping someone else's result is immoral but not an outright violation of academic norms. But if you are in possession of relevant confidential information, you are expected to steer clear of the topic. It doesn't matter whether you actually used the confidential information to get your results, because outsiders can't know that. The mere fact that there is a plausible suspicion already puts your integrity into question. Tenured professors occasionally lose their jobs over similar scandals (but usually don't). If OpenAI wants to regain some goodwill, it should do a thorough investigation that may lead to firing the individuals in question. If it doesn't find sufficient evidence of wrongdoing to justify any disciplinary action, it probably doesn't gain any goodwill either (as it often happens with similar investigations at universities). And if OpenAI wants to be a trustworthy partner, it should transform into a company of boring gray bureaucrats who provide an essential service without competing with their customers.

cj 23d ago
[deleted - misunderstood!]
- jltsiren 23d ago
  My point was that if someone is at fault, it's the individual OpenAI employees. Because they chose to engage in a professional field, they can't use "boss told me to do so" as a defense.
  - gregw2 22d ago
    Err, no. If someone is at fault, it is definitely the company (OpenAI) not the employee in this case. Just think, isn't the whole point of a company, of incorporating, is that the liability shifts from the employee to the firm? It is 100% fair to hold the company responsible, and doubly so when the questionable thing the employee is doing is something that A) benefits the company, and B) is on company time (aand with company resources.) I know you weren't making a legal-minutia point but even the high level principles involved are and should be the opposite of what you suggest. Companies may want to have their cake and eat it too (have employees do their dirty work but then push the blame on the employee) but that's not how it works and it's generally not a society you'd want to live in the more we head in that direction.

amelius 23d ago

The problem here is that OAI (and others) pretend or claim that this is uncharted legal territory, where in fact it is very simple. We have a machine that is fed data, and produces new data as a result. If that new data depends (in any way) on the fed data, then from a legal viewpoint it is derived from that data. Whether they anthropomorphize the operation performed by the machine does not matter. They can anthropomorphize when/if the law is updated to include such terms, but right now they certainly cannot.

fn-mote 23d ago
> in fact it is very simple Even if this opinion were backed up by a court ruling, it would definitely not be “simple”. It will be a very ugly case if it is ever litigated. A lot of money will be spent and no guarantee at all the plaintiff wins.
magicalhippo 22d ago
> If that new data depends (in any way) on the fed data, then from a legal viewpoint it is derived from that data. The "in any way" part is either so broad it makes everything derivative, or not, in which case things are no longer simple. If everything is derivative then it seizes to be meaningful. The words I write are derivative, I literally copied them from someone else, yet my sentences as a whole can be fully novel.
- amelius 22d ago
  > If everything is derivative then it seizes to be meaningful. That's why we tolerate it for humans, and also because we cannot prove it. But yes, if you go too far in this, you will see legal consequences.

Eji1700 23d ago

> Provenance is hard to track Right, which is going to open a lot of doors to a lot of questions. I don't think there's any legal ramifications on this, just ethical ones about when and how you publish research, but it's yet another point in favor of "if provenance is hard to track, should we be using this for things where it needs to be". Obviously copyright/trademark is a huge discussion on this, and I could absolutely see this devolving into that as well with how certain findings wind up monetized. We have a response in this topic from someone claiming to be from OpenAI and linking an article where they, roughly, say "we are sure nothing from the 2 month period made its way into the solution". If that is true, that should mean it is provable, but leads to some more open ended questions like "well what data did it use then?". Is this still okay if someone close to the author did plug data into open AI and it extrapolated it? Obviously that's probably an unreasonable expectation for these models to track and prove, but it also used to be an unreasonable expectation to scrape every single piece of digital and physical info for consolidated data. If I opine to a friend on a park bench about a story I'm writing, do they get to pull it from the flock feed, shove it in the model, and then provide it to disney? Legally, right now, probably. But there's going to need to be a serious look at laws and standards. Or a major shift in what is and isn't discussed in public if literally every breath and move you make can become monetized.

rolandog 23d ago

Then there's the possibility of indirect training via modern spy devices ("smart" IoT devices like LG TV's) feeding the transcribed ambient conversation data for summarization to an agent [0]. [0]: https://youtu.be/6IFVTcM28KA https://youtu.be/6IFVTcM28KA

lmeyerov 23d ago

OpenAI says deidentified data from the private sessions go into training. (Well, explicitly said they will not rule that out.) That changes a lot of the conversation.

BobbyTables2 23d ago

The AI not being human doesn’t escape ethical consideration - OpenAI employees are culpable for what they build. This was academic research. Could just have easily been trade secrets and proprietary data.

timcobb 22d ago

> but a full data trail of all inputs is difficult to trace through. Great use case for AI agents

tw04 22d ago

> you would hope OpenAI has very good tools for this, but a full data trail of all inputs is difficult to trace through. They have a financial incentive not to track any of this, so why would they? OpenAI’s entire business model is predicated on stealing other people’s work and selling it to the masses.

Agentlien 22d ago

I think this move by OpenAI is crazy. At best, if all unconfirmed accusations are unfounded, they still heard a rumour that someone had solved a huge million dollar problem and was about to make a name for themselves. Then, they decided this was a good opportunity to pour millions of dollars into trying to snag the glory while the researchers were busy cleaning up their notes and polishing the announcement. That still sounds highly unethical.

jasonfarnon 22d ago
Would this be unethical if it was a human who heard rumors about a solution then attacked the problem, solved it and published first? Often knowing of the mere existence of a solution carries a lot of information--you would know the problem is accessible, you would expect clues in recent progress (the two Spanish researchers in this case), you would probably have a sense if the solution is a counterexample or positive proof, and so on. I think there are similar examples where we think of them as maybe unsporting but not quite unethical. Does it change if it's openAI and not a human?
- timmytokyo 22d ago
  The problem with your counter-hypothetical is that not only is it unrealistic, it's utterly impossible. No human would be able to do in such a short timeframe what the LLM did. Part of what makes the OpenAI move so egregious is how bullying it was. It was the big guy coming along with their nearly infinite resources and squashing the little guy who's devoted a good chunk of his career to the problem.
  - jasonfarnon 22d ago
    Actually my hypothetical is completely realistic as I've been involved in such scenarios. It's unrealistic maybe for a millennium problem to come in on a rumor and still front-run but not at all for the many other problems we work on and which manifest our ethical code. If you're saying ethical rules change depending on the prize be clear about it, because I can see arguments that they change to favor either side.

enyone 22d ago

I think OP's analogy is bad. The difference of OpenAI when comparing to human collaborator is the possibility to replicate once learned skill. Imagine if any single human collaborator learns a skill it is immediately a skill of any human collaborator.

tgv 22d ago

> I think it's a useful analogy to compare OpenAI to a human collaborator. A human collaborator who stands to win or loose a couple of $100B.

pred_ 22d ago

Another scenario also just surfaced: https://news.ycombinator.com/item?id=49657499 https://news.ycombinator.com/item?id=49657499 – the construction used by Anthropic for the Jacobian Conjecture counterexample appears to have existed in an unpublished but publically available document. This document of course wasn't referenced in the announcement (nor did we ever get to see the prompt or reasoning traces).

pfortuny 22d ago
Wow, this is really interesting. Thanks.