5 ms·

Full mastodon post: https://mathstodon.xyz/@tristanbuckmaster@mastodon.social/117233413735526010 https://mathstodon.xyz/@tristanbuckmaster@mastodon.social/11...

Full mastodon post: https://mathstodon.xyz/@tristanbuckmaster@mastodon.social/117233413735526010 https://mathstodon.xyz/@tristanbuckmaster@mastodon.social/11... Mathematical explanation by Terrance Tao: https://mathstodon.xyz/@tao/117233527638291447 https://mathstodon.xyz/@tao/117233527638291447 It seems there is much background drama behind this, and this is what I've pieced together of what happened: Over the past year, Buckmaster and Alpöge have been using AI to work on fluid dynamics maths problems. Alpöge works at Anthropic, which will cause future issues. In mid-August, they found a counterexample for a simpler version of the Navier-Stokes problem. They spend the next few weeks preparing their paper. In early September, rumors start spreading on X that Anthropic has solved a Millennium prize problem (and that it's Navier-Stokes). Buckmaster reaches out to OpenAI to explain this is their own personal research, not an Anthropic project. A few days later, OpenAI gets back to him, and tells him an internal model found has a counterexample for Navier–Stokes, potentially worth the $1 million Millennium prize. The proof uses the same method that Buckmaster and Alpöge chose to work on. They don't show him the proof. Buckmaster pressed them for more details. OpenAI reveals they had an entire team had been working on the problem, and that they started work in the past few days, after the rumors that Anthropic had solved a Millennium prize problem. Buckmaster says OpenAI talked about a shared publication timeline. They want to Buckmaster to publish first, then give Buckmaster shared credit for the Millennium Prize when they publish the full result. But they want to exclude Alpöge as an author because he works at Anthropic. An agreement is not reached. Buckmaster had been using OpenAI Codex to draft/check his work, and asks if his private AI chats were used to accelerate OpenAI's result. Buckmaster and Alpöge think they have found a counterexample for Navier-Stokes, but the paper is not yet presentable. It's unclear what date they found this result. Because of the situation with OpenAI, they published their existing papers earlier than planned (today), alongside this statement announcing they have a tentative result on Navier-Stokes and revealing the OpenAI drama. The post is missing context from both sides, and this isn't my field, so hopefully someone else can unpack what's happening here.

akersten 1mo ago
> They were coordinating with OpenAI regarding a publishing timeline, but could not come to an agreement, Skimming the PDFs it seems much more dramatic than that? It sounds like at least one of them is concerned OpenAI "solved" the problem by having their internal model use the chats of the independent researchers and want to claim the credit instead? I don't know. The tone is pretty accusational though: > the one Levent and I had quietly chosen to attack. Almost nobody else I know of was working on it. It is not the direction one arrives at in a few days by giving a model the problem statement. When I heard “forced,” it was a bright red flag. > I was shown a prompt and told the internal research model had simply been given the problem statement. Levent had been told by Sebastien “very little human input” had been used. This turned out not to be true. Over the course of the call, as members of their team sent Sebastien corrections and details over their internal chat, it emerged that an entire team had been working on the problem, that this was one of a number of things that was tried, that work had started on the unforced problem, that the team first set the model on easier problems, including Euler, that even the prompt that had been shown to me had been written by prompting Codex, and that an insane amount of compute had been used. > I asked when the first prompt had been sent by them. This question was not answered directly by OpenAI for some time. Eventually it was agreed that it had been sent in the past few days, after information about our work had reached OpenAI. > I asked whether the model had been trained on, or had access to, our sessions in Codex, into which we had been putting all our drafts for the whole of this project. I was told the model did not look up user data. I asked again, about training, and I did not get an answer. [0] [0]: https://cims.nyu.edu/~tristanb/statement.pdf https://cims.nyu.edu/~tristanb/statement.pdf
- tristanj 1mo ago
  Yeah these are major accusations. But the story is incomplete, the conversation is missing a lot of details. It's not clear who was working on what, and when. The entire thing feels rushed, like they wanted to get this result published and out the door quickly.
  - deleted 1mo ago
    [deleted]
- cma 1mo ago
  Does he claim to have opted out of training too?
  - kzrdude 1mo ago
    There are various forces at play here, academic honesty requires them to disclose any inputs regardless of license or ToS circumstances. While common sense reminds us here that if you send your data to an external entity’s computer, you are no longer in control of said data. The lines have blurred here clearly over the last decade, but that should have made the theory yet more clear to everyone involved: your data will be vacuumed up unless you keep it sealed. Use your own computer if you want to be in control.
    - cma 1mo ago
      But if they didn't opt out of training, did they want OpenAI to opt out for them? Also they need to audit anyone they sent drafts to to make sure they opted out before submitting it. I'd prefer things be opt in, and especially not start opt out, then try to trick you opt in with a popup defaulting to opt-in, like Anthropic did on consumer plans, but if they submitted anything on an opted-in plan it's not reasonable to be mad it trained on it. Even still, I also believe for significant reasons that OpenAI would ignore the opt-out in selective cases and could be in the wrong here. And the threats and terms they offered seem wrong either way, pending more context.
dumberquestions 1mo ago
>...has solved a millennium problem and is sitting on the result Someone correct me if I'm wrong, but the work involved here is not the actual millennium problem, but it concerns versions with an added external force that the author thinks is a path that may help toward solving the harder unforced problem.
- modeless 1mo ago
  Apparently forcing is allowed in the Millenium Prize problem statement. So OpenAI's claimed proof could win the prize. OTOH the results Tristan and Levent are publishing here do not go far enough to win the prize, though apparently they are suggestive of a general approach that could produce a solution, which seems likely to be the general approach OpenAI's proof uses. The question is whether OpenAI's pursuit of this direction happened spontaneously, or as a result of them learning about Tristan's work somehow. To be clear, while the tone of this post seems quite accusatory, Tristan does not claim to know for sure whether OpenAI unfairly benefited from his work. Sholto Douglas from Anthropic is also on record saying the suggestion that OpenAI used Tristan's codex transcripts somehow is extremely unlikely to be true[1], which I agree with, though it doesn't rule out them learning of Tristan's work some other way. I am sure OpenAI will have a statement out tomorrow clarifying their position. [1] https://x.com/_sholtodouglas/status/2097218240397410733 https://x.com/_sholtodouglas/status/2097218240397410733
  - dumberquestions 1mo ago
    Why do you find it to be extremely unlikely?
    - tristanj 1mo ago
      Because very few people actually have access to these logs, all access is monitored and recorded, and improper access will get you fired. It's not worth risking your job over something like this.
      - dumberquestions 1mo ago
instagraham 1mo ago
> A few days later, OpenAI gets back to him, and tells him an internal model found a counterexample for Navier–Stokes Why is OpenAI chatting with him at all at this stage? Is the discussion along the lines of "hey we used the work you are famous for to do a bigger piece of work, just thought you should know" or "heyyy....so we kinda liked what you were typing in your private chat, and thought we'd develop those ideas a bit. and yeah we solved Navier-Stokes in the process. But it's our finding, so do you want like an honorary acknowledgement or do you want to go to court?"
- traes 1mo ago
  Perhaps out of a sense of academic good will, knowing that he got there first? It seems like the timeline according to OpenAI is that: 1. Buckmaster developed a counterexample to a reduced version of Navier-Stokes with Anthropic employee Levent 2. Rumors start spreading that Anthropic has solved Navier-Stokes 3. OpenAI learns this and starts throwing a ridiculous amount of compute at it, now knowing it's within reach of LLMs 4. Their LLMs (with human assistance) get FARTHER than Buckmaster, using the exact same method. 5. OpenAI reaches out to Buckmaster to negotiate a fair way to publish both results and properly assign credit Perhaps they simply and honestly feel he is owed credit. I can't help but imagine it at least played some small part. I expect that's what Bubeck is going to claim: https://xcancel.com/SebastienBubeck/status/2097214122471432349 https://xcancel.com/SebastienBubeck/status/20972141224714323...
  - denverllc 1mo ago
    The only problem with this narrative is that they refused to allow the other coauthor to be listed because he worked at Anthropic. That is absolutely *ridiculous* in academia to deny authorship because of affiliation of the author worked on a substantial portion. You’d be ostracized because nobody would ever want to work with you again.
Davidzheng 1mo ago
"Buckmaster and Alpöge think they have found a counterexample for Navier-Stokes, but the paper is not yet presentable. " Are you sure about this? I'm far far from the area but it doesn't look like it to me on first viewing (hypo-dispersive seems like a sizable difference to me and not covered in the clay prize description)
- tristanj 1mo ago
  Yes, it's a separate problem. That's a mistake in my post.
OceanSeth 1mo ago
This reminds me of a conversation I had with an engineer at google when I was angrily saying "it's not end to end encryption if you get my emails and can train your models on them" to which he said "we try not to do that". What a statement!

akersten 1mo ago

> They were coordinating with OpenAI regarding a publishing timeline, but could not come to an agreement, Skimming the PDFs it seems much more dramatic than that? It sounds like at least one of them is concerned OpenAI "solved" the problem by having their internal model use the chats of the independent researchers and want to claim the credit instead? I don't know. The tone is pretty accusational though: > the one Levent and I had quietly chosen to attack. Almost nobody else I know of was working on it. It is not the direction one arrives at in a few days by giving a model the problem statement. When I heard “forced,” it was a bright red flag. > I was shown a prompt and told the internal research model had simply been given the problem statement. Levent had been told by Sebastien “very little human input” had been used. This turned out not to be true. Over the course of the call, as members of their team sent Sebastien corrections and details over their internal chat, it emerged that an entire team had been working on the problem, that this was one of a number of things that was tried, that work had started on the unforced problem, that the team first set the model on easier problems, including Euler, that even the prompt that had been shown to me had been written by prompting Codex, and that an insane amount of compute had been used. > I asked when the first prompt had been sent by them. This question was not answered directly by OpenAI for some time. Eventually it was agreed that it had been sent in the past few days, after information about our work had reached OpenAI. > I asked whether the model had been trained on, or had access to, our sessions in Codex, into which we had been putting all our drafts for the whole of this project. I was told the model did not look up user data. I asked again, about training, and I did not get an answer. [0] [0]: https://cims.nyu.edu/~tristanb/statement.pdf https://cims.nyu.edu/~tristanb/statement.pdf

tristanj 1mo ago
Yeah these are major accusations. But the story is incomplete, the conversation is missing a lot of details. It's not clear who was working on what, and when. The entire thing feels rushed, like they wanted to get this result published and out the door quickly.
- deleted 1mo ago
  [deleted]
cma 1mo ago
Does he claim to have opted out of training too?
- kzrdude 1mo ago
  There are various forces at play here, academic honesty requires them to disclose any inputs regardless of license or ToS circumstances. While common sense reminds us here that if you send your data to an external entity’s computer, you are no longer in control of said data. The lines have blurred here clearly over the last decade, but that should have made the theory yet more clear to everyone involved: your data will be vacuumed up unless you keep it sealed. Use your own computer if you want to be in control.
  - cma 1mo ago
    But if they didn't opt out of training, did they want OpenAI to opt out for them? Also they need to audit anyone they sent drafts to to make sure they opted out before submitting it. I'd prefer things be opt in, and especially not start opt out, then try to trick you opt in with a popup defaulting to opt-in, like Anthropic did on consumer plans, but if they submitted anything on an opted-in plan it's not reasonable to be mad it trained on it. Even still, I also believe for significant reasons that OpenAI would ignore the opt-out in selective cases and could be in the wrong here. And the threats and terms they offered seem wrong either way, pending more context.

dumberquestions 1mo ago

>...has solved a millennium problem and is sitting on the result Someone correct me if I'm wrong, but the work involved here is not the actual millennium problem, but it concerns versions with an added external force that the author thinks is a path that may help toward solving the harder unforced problem.

modeless 1mo ago
Apparently forcing is allowed in the Millenium Prize problem statement. So OpenAI's claimed proof could win the prize. OTOH the results Tristan and Levent are publishing here do not go far enough to win the prize, though apparently they are suggestive of a general approach that could produce a solution, which seems likely to be the general approach OpenAI's proof uses. The question is whether OpenAI's pursuit of this direction happened spontaneously, or as a result of them learning about Tristan's work somehow. To be clear, while the tone of this post seems quite accusatory, Tristan does not claim to know for sure whether OpenAI unfairly benefited from his work. Sholto Douglas from Anthropic is also on record saying the suggestion that OpenAI used Tristan's codex transcripts somehow is extremely unlikely to be true[1], which I agree with, though it doesn't rule out them learning of Tristan's work some other way. I am sure OpenAI will have a statement out tomorrow clarifying their position. [1] https://x.com/_sholtodouglas/status/2097218240397410733 https://x.com/_sholtodouglas/status/2097218240397410733
- dumberquestions 1mo ago
  Why do you find it to be extremely unlikely?
  - tristanj 1mo ago
    Because very few people actually have access to these logs, all access is monitored and recorded, and improper access will get you fired. It's not worth risking your job over something like this.
    - dumberquestions 1mo ago

instagraham 1mo ago

> A few days later, OpenAI gets back to him, and tells him an internal model found a counterexample for Navier–Stokes Why is OpenAI chatting with him at all at this stage? Is the discussion along the lines of "hey we used the work you are famous for to do a bigger piece of work, just thought you should know" or "heyyy....so we kinda liked what you were typing in your private chat, and thought we'd develop those ideas a bit. and yeah we solved Navier-Stokes in the process. But it's our finding, so do you want like an honorary acknowledgement or do you want to go to court?"

traes 1mo ago
Perhaps out of a sense of academic good will, knowing that he got there first? It seems like the timeline according to OpenAI is that: 1. Buckmaster developed a counterexample to a reduced version of Navier-Stokes with Anthropic employee Levent 2. Rumors start spreading that Anthropic has solved Navier-Stokes 3. OpenAI learns this and starts throwing a ridiculous amount of compute at it, now knowing it's within reach of LLMs 4. Their LLMs (with human assistance) get FARTHER than Buckmaster, using the exact same method. 5. OpenAI reaches out to Buckmaster to negotiate a fair way to publish both results and properly assign credit Perhaps they simply and honestly feel he is owed credit. I can't help but imagine it at least played some small part. I expect that's what Bubeck is going to claim: https://xcancel.com/SebastienBubeck/status/2097214122471432349 https://xcancel.com/SebastienBubeck/status/20972141224714323...
- denverllc 1mo ago
  The only problem with this narrative is that they refused to allow the other coauthor to be listed because he worked at Anthropic. That is absolutely *ridiculous* in academia to deny authorship because of affiliation of the author worked on a substantial portion. You’d be ostracized because nobody would ever want to work with you again.

Davidzheng 1mo ago

"Buckmaster and Alpöge think they have found a counterexample for Navier-Stokes, but the paper is not yet presentable. " Are you sure about this? I'm far far from the area but it doesn't look like it to me on first viewing (hypo-dispersive seems like a sizable difference to me and not covered in the clay prize description)

tristanj 1mo ago
Yes, it's a separate problem. That's a mistake in my post.

OceanSeth 1mo ago

This reminds me of a conversation I had with an engineer at google when I was angrily saying "it's not end to end encryption if you get my emails and can train your models on them" to which he said "we try not to do that". What a statement!