3 ms·

> It's not obvious to me that's an unethical thing to do In terms of work in mathematics, something I personally would not do based on ethical grounds would be

> It's not obvious to me that's an unethical thing to do In terms of work in mathematics, something I personally would not do based on ethical grounds would be to hear a rumor that some researchers are taking a certain approach and may be nearing a solution, use a model that was possibly contaminated with intimate knowledge about that approach (though later they investigated and think it wasn't), and then commit millions to tens of millions of dollars and untold amounts of hardware to try to beat them to it. If I had done this, I also wouldn't have pestered the researchers on a Sunday night to meet immediately so we could negotiate a nice way of presenting the actions I had decided to take. Even if you don't think it was unethical, it was never going to be received well in the community that was especially going to care about this work, and who are very much peers to many of the people working on this solution, so it was at the least an enormous (and well-deserved) own-goal that their unveiling of their solution to NS went like this.

dwaltrip 22d ago
I heard they also tried to strong-arm them into removing the name of their collaborator who happened to work at a different company (Anthropic)... I haven't looked into it myself, but if true, that seems incredibly scummy.
- luma 22d ago
  The flip side is that the Anthropic researcher is clearly pushing the case against Open AI and one might have reason to suspect their motivation and version of events for the same reasons. What a mess.
  - SecretDreams 22d ago
    What's that saying about wrestling with pigs?
  - alexgoodhart 22d ago
    I think people are too reserved in their unwillingness to operationalize ambiguity. Ambiguity is constantly being thrown in our face, with internal audits and other laughable attestations of virtue that amount to a pantomime of transparency / good faith. Why should I care if a company claims they find no evidence of wrongdoing? Is that the threshold for privacy/trust? “We don’t care if it appears that we’ve been dishonest unless there’s hard proof.” They can simply design proof keeping to terminate at the places their dishonesty is implemented. For me, when there is a clear motive to be dishonest, a corporation should be assumed to be dishonest unless there are robust transparency measures and a regulatory environment shown to be providing a cost to dishonesty. Without it, all you do is burden yourself while the powerful entity moves ahead with its selective dishonesty and the rewards there reaped.
  - joshuamorton 22d ago
    > The flip side is that the Anthropic researcher is clearly pushing the case against Open AI and one might have reason to suspect their motivation and version of events for the same reasons. I haven't seen any evidence of this. Much of the anger is coming from the unaffiliated researcher. levent (the anthropic employee) has mostly constrained his comments to basically "I would have been happy to collaborate w/ folks from OAI"
  - magicalist 22d ago
    > the Anthropic researcher is clearly pushing the case against Open AI Things like the nytimes interview are with Buckmaster, who works at NYU, not Alpöge. I saw a couple of tweets from him over the last week. Any chance of clarifying what makes you think he's "clearly pushing the case"?
- user43928 22d ago
  That's also incorrect. My understanding is that they asked the independent researcher to improve OpenAI's AI generated proof and be the lead author of the paper to publish OpenAI's result. This is the paper where they did not want the Anthropic employee collaborating. Not their work.
  - snaking0776 22d ago
    I think both Seb and Sam have said that it would’ve been simpler if the coauthor hadn’t worked at Anthropic so they’ve largely admitted they didn’t invite the collaborator as a coauthor because it would’ve look bad to have an Anthropic employee on the paper.
    - jerkstate 22d ago
      seems pretty short-sighted - "our models are so good that even our competitors use them for the most advanced tasks" is pretty powerful marketing
    - user43928 22d ago
      Yes. The key point being that this concerns a new paper about OpenAI's result rather than the paper Buckmaster and Alpöge were working on.
    - charlieyu1 22d ago
      Looks like a lot of insecurity from OpenAI. At the top level researchers move around at their own will, you don’t decide who they work for
keeda 22d ago
But by OpenAI's telling they heard a rumor that the problem had already been solved. So they reached out to the other researchers as an attempt to share the credit, and in fact have at least one of them be the lead author (which is when they found out the AI had solved a broader problem than the researchers.) Seems pretty ethically palatable. I suspect the main reason the community is not receiving it well is largely the same reason many developers are not receiving coding agents well.
- ZYbCRq22HbJ2y7 22d ago
  > I suspect the main reason the community is not receiving it well is largely the same reason many developers are not receiving coding agents well. Because the training data is millions of hours human efforts being distilled into a cascading hierarchy of enrichment by interested parties without providing attribution or compensation?
  - keeda 22d ago
    I'm sure that's part of the reason for many, yes.
  - eru 22d ago
    I don't see the cascading hierarchy of enrichment. I mean, they are certainly trying, but so far there's too much competition so the surplus mostly goes to customers.
    - zecken 22d ago
      Well, the investment dollars are spent on the customers for the most part, though also on salaries and equipment. But the lions share of the value is going to the shareholders (eg employees and investors)... and they have liquidated and will continue to liquidate a disproportionate value to what they have spent on us. By some estimations at least. It's very possible $1 into this machine to feed your queries is worth $10+ to a shareholder based on whatever new valuation they get. So I'd say there is a hierarchy of enrichment.
    - 22d ago
cgio 22d ago
Hearing that something is solvable is already a hint. I don’t think leveraging this knowledge is ethical. They could go after a different problem but didn’t.

dwaltrip 22d ago

I heard they also tried to strong-arm them into removing the name of their collaborator who happened to work at a different company (Anthropic)... I haven't looked into it myself, but if true, that seems incredibly scummy.

luma 22d ago
The flip side is that the Anthropic researcher is clearly pushing the case against Open AI and one might have reason to suspect their motivation and version of events for the same reasons. What a mess.
- SecretDreams 22d ago
  What's that saying about wrestling with pigs?
- alexgoodhart 22d ago
  I think people are too reserved in their unwillingness to operationalize ambiguity. Ambiguity is constantly being thrown in our face, with internal audits and other laughable attestations of virtue that amount to a pantomime of transparency / good faith. Why should I care if a company claims they find no evidence of wrongdoing? Is that the threshold for privacy/trust? “We don’t care if it appears that we’ve been dishonest unless there’s hard proof.” They can simply design proof keeping to terminate at the places their dishonesty is implemented. For me, when there is a clear motive to be dishonest, a corporation should be assumed to be dishonest unless there are robust transparency measures and a regulatory environment shown to be providing a cost to dishonesty. Without it, all you do is burden yourself while the powerful entity moves ahead with its selective dishonesty and the rewards there reaped.
- joshuamorton 22d ago
  > The flip side is that the Anthropic researcher is clearly pushing the case against Open AI and one might have reason to suspect their motivation and version of events for the same reasons. I haven't seen any evidence of this. Much of the anger is coming from the unaffiliated researcher. levent (the anthropic employee) has mostly constrained his comments to basically "I would have been happy to collaborate w/ folks from OAI"
- magicalist 22d ago
  > the Anthropic researcher is clearly pushing the case against Open AI Things like the nytimes interview are with Buckmaster, who works at NYU, not Alpöge. I saw a couple of tweets from him over the last week. Any chance of clarifying what makes you think he's "clearly pushing the case"?
user43928 22d ago
That's also incorrect. My understanding is that they asked the independent researcher to improve OpenAI's AI generated proof and be the lead author of the paper to publish OpenAI's result. This is the paper where they did not want the Anthropic employee collaborating. Not their work.
- snaking0776 22d ago
  I think both Seb and Sam have said that it would’ve been simpler if the coauthor hadn’t worked at Anthropic so they’ve largely admitted they didn’t invite the collaborator as a coauthor because it would’ve look bad to have an Anthropic employee on the paper.
  - jerkstate 22d ago
    seems pretty short-sighted - "our models are so good that even our competitors use them for the most advanced tasks" is pretty powerful marketing
  - user43928 22d ago
    Yes. The key point being that this concerns a new paper about OpenAI's result rather than the paper Buckmaster and Alpöge were working on.
  - charlieyu1 22d ago
    Looks like a lot of insecurity from OpenAI. At the top level researchers move around at their own will, you don’t decide who they work for

keeda 22d ago

But by OpenAI's telling they heard a rumor that the problem had already been solved. So they reached out to the other researchers as an attempt to share the credit, and in fact have at least one of them be the lead author (which is when they found out the AI had solved a broader problem than the researchers.) Seems pretty ethically palatable. I suspect the main reason the community is not receiving it well is largely the same reason many developers are not receiving coding agents well.

ZYbCRq22HbJ2y7 22d ago
> I suspect the main reason the community is not receiving it well is largely the same reason many developers are not receiving coding agents well. Because the training data is millions of hours human efforts being distilled into a cascading hierarchy of enrichment by interested parties without providing attribution or compensation?
- keeda 22d ago
  I'm sure that's part of the reason for many, yes.
- eru 22d ago
  I don't see the cascading hierarchy of enrichment. I mean, they are certainly trying, but so far there's too much competition so the surplus mostly goes to customers.
  - zecken 22d ago
    Well, the investment dollars are spent on the customers for the most part, though also on salaries and equipment. But the lions share of the value is going to the shareholders (eg employees and investors)... and they have liquidated and will continue to liquidate a disproportionate value to what they have spent on us. By some estimations at least. It's very possible $1 into this machine to feed your queries is worth $10+ to a shareholder based on whatever new valuation they get. So I'd say there is a hierarchy of enrichment.
  - 22d ago

cgio 22d ago

Hearing that something is solvable is already a hint. I don’t think leveraging this knowledge is ethical. They could go after a different problem but didn’t.