4 ms·

I mean, here is a relevant passage from the paper, linked in another comment: https://news.ycombinator.com/item?id=38373684 https://news.ycombinator.com/item?id

I mean, here is a relevant passage from the paper, linked in another comment: https://news.ycombinator.com/item?id=38373684 https://news.ycombinator.com/item?id=38373684 If I were the CEO of OpenAI, I'd be pretty pissed if a member of my own board was shitting on the organization she was a member of while puffing up a competitor. But the tone of that paper makes me think that the schism must go back much earlier (other reporting said things really started to split a year ago when ChatGPT was first released), and it sounds to me like Toner was needling because she was pissed with the direction OpenAI was headed. I'm thinking of a good previous comment I read when the whole Timnit Gebru situation at Google blew up and the Ethical AI team at Google was disbanded. The basic argument was on some of the inherent incompatibilities between an "academic ombudsman" mindset, and a "corporate growth" mindset. I'm not saying which one was "right" in this situation given OpenAI's frankenstein org structure, but just that this kind of conflict was probably inevitable.

lossolo 3y ago
> If I were the CEO of OpenAI, I'd be pretty pissed if a member of my own board was shitting on the organization she was a member of while puffing up a competitor. Considering what's in the charter, it seems like she didn't do anything wrong? > We will attempt to directly build safe and beneficial AGI, but will also consider our mission fulfilled if our work aids others to achieve this outcome. >We commit to use any influence we obtain over AGI’s deployment to ensure it is used for the benefit of all, and to avoid enabling uses of AI or AGI that harm humanity or unduly concentrate power. more here: https://news.ycombinator.com/item?id=38372769 https://news.ycombinator.com/item?id=38372769
- xigency 3y ago
  > Considering what's in the charter, it seems like she didn't do anything wrong? It’s incredibly disingenuous to slap your name on an ethics paper claiming a company is doing malfeasance such as triggering a race-to-the-bottom for AI ethics when you have an active hand in steering the company. It’s shameful. Either she should have resigned from the company, for ethical reasons, or recused herself from the paper, for conflict of interest.
  - kamranjon 3y ago
    Did we read different things? All it said was that they had been accused of these things, which is true. If your charter involves ethical AI I’d imagine the first step is telling the truth?
    - xigency 3y ago
      From the PDF: While the system card itself has been well received among researchers interested in understanding GPT-4’s risk profile, it appears to have been less successful as a broader signal of OpenAI’s commitment to safety. The reason for this unintended outcome is that the company took other actions that overshadowed the import of the system card: most notably, the blockbuster release of ChatGPT four months earlier. Intended as a relatively inconspicuous “research preview,” the original ChatGPT was built using a less advanced LLM called GPT-3.5, which was already in widespread use by other OpenAI customers. GPT-3.5’s prior circulation is presumably why OpenAI did not feel the need to perform or publish such detailed safety testing in this instance. Nonetheless, one major effect of ChatGPT’s release was to spark a sense of urgency inside major tech companies.149 To avoid falling behind OpenAI amid the wave of customer enthusiasm about chatbots, competitors sought to accelerate or circumvent internal safety and ethics review processes, with Google creating a fast-track “green lane” to allow products to be released more quickly.150 This result seems strikingly similar to the race- to-the-bottom dynamics that OpenAI and others have stated that they wish to avoid. OpenAI has also drawn criticism for many other safety and ethics issues related to the launches of ChatGPT and GPT-4, including regarding copyright issues, labor conditions for data annotators, and the susceptibility of their products to “jailbreaks” that allow users to bypass safety controls.151 This muddled overall picture provides an example of how the messages sent by deliberate signals can be overshadowed by actions that were not designed to reveal intent.
      - webmaven 3y ago
        > This […] provides an example of how the messages sent by deliberate signals can be overshadowed by actions that were not designed to reveal intent. What a lovely turn of phrase! I'm stealing it for use later, in place of "actions speak louder than words".
- jorams 3y ago
  I agree and none of the other passages people are quoting from the paper (which I admittedly haven't read yet) seem at all controversial. She's saying OpenAI's messaging about AI safety is not landing because it's simultaneously launching products that are taking the spotlight, and Anthropic is doing a better job at signaling their commitment to safety. That's true, obvious, and entirely in line with the charter she's supposed to uphold.
BryantD 3y ago
Just spot checking: did anyone comment on this paper when it was published? Did any media outlet say “hey, a member of the OpenAI board is criticizing OpenAI and showing a conflict of interest?” Did any of the people who cover AI (Zvi, say) discuss this as a problem? These are serious questions, not gotchas. I don’t know the answers, and I think having those answers would make it easier to evaluate whether or not the paper was a significant conflict of interest. The opinions we have formed now are shaped by our biases about current events. It didn’t make HN.
- dmix 3y ago
  Gambling that no one reads academics papers by fringe Open AI board members no one has heard of before is probably a safe bet but it's still a risk if some doomer AI people on Twitter pumped it up, some journo discovered the tweets and sells it as "concern in the industry including one of OpenAI's own board" and it got swept up in some lawyer-style grilling by Congress which Sam just had to go through.
- triplebankshot 3y ago
  Who cares if the paper was covered in the media or not? Think tanks write policy papers for regulators, not HN. And it's the regulators that Sam was worried about. From the article: > Mr. Altman said that he had reprimanded Ms. Toner for the paper and that it was dangerous to the company, particularly at a time, he added, when the Federal Trade Commission was investigating OpenAI over the data used to build its technology.

lossolo 3y ago

> If I were the CEO of OpenAI, I'd be pretty pissed if a member of my own board was shitting on the organization she was a member of while puffing up a competitor. Considering what's in the charter, it seems like she didn't do anything wrong? > We will attempt to directly build safe and beneficial AGI, but will also consider our mission fulfilled if our work aids others to achieve this outcome. >We commit to use any influence we obtain over AGI’s deployment to ensure it is used for the benefit of all, and to avoid enabling uses of AI or AGI that harm humanity or unduly concentrate power. more here: https://news.ycombinator.com/item?id=38372769 https://news.ycombinator.com/item?id=38372769

xigency 3y ago
> Considering what's in the charter, it seems like she didn't do anything wrong? It’s incredibly disingenuous to slap your name on an ethics paper claiming a company is doing malfeasance such as triggering a race-to-the-bottom for AI ethics when you have an active hand in steering the company. It’s shameful. Either she should have resigned from the company, for ethical reasons, or recused herself from the paper, for conflict of interest.
- kamranjon 3y ago
  Did we read different things? All it said was that they had been accused of these things, which is true. If your charter involves ethical AI I’d imagine the first step is telling the truth?
  - xigency 3y ago
    From the PDF: While the system card itself has been well received among researchers interested in understanding GPT-4’s risk profile, it appears to have been less successful as a broader signal of OpenAI’s commitment to safety. The reason for this unintended outcome is that the company took other actions that overshadowed the import of the system card: most notably, the blockbuster release of ChatGPT four months earlier. Intended as a relatively inconspicuous “research preview,” the original ChatGPT was built using a less advanced LLM called GPT-3.5, which was already in widespread use by other OpenAI customers. GPT-3.5’s prior circulation is presumably why OpenAI did not feel the need to perform or publish such detailed safety testing in this instance. Nonetheless, one major effect of ChatGPT’s release was to spark a sense of urgency inside major tech companies.149 To avoid falling behind OpenAI amid the wave of customer enthusiasm about chatbots, competitors sought to accelerate or circumvent internal safety and ethics review processes, with Google creating a fast-track “green lane” to allow products to be released more quickly.150 This result seems strikingly similar to the race- to-the-bottom dynamics that OpenAI and others have stated that they wish to avoid. OpenAI has also drawn criticism for many other safety and ethics issues related to the launches of ChatGPT and GPT-4, including regarding copyright issues, labor conditions for data annotators, and the susceptibility of their products to “jailbreaks” that allow users to bypass safety controls.151 This muddled overall picture provides an example of how the messages sent by deliberate signals can be overshadowed by actions that were not designed to reveal intent.
    - webmaven 3y ago
      > This […] provides an example of how the messages sent by deliberate signals can be overshadowed by actions that were not designed to reveal intent. What a lovely turn of phrase! I'm stealing it for use later, in place of "actions speak louder than words".
jorams 3y ago
I agree and none of the other passages people are quoting from the paper (which I admittedly haven't read yet) seem at all controversial. She's saying OpenAI's messaging about AI safety is not landing because it's simultaneously launching products that are taking the spotlight, and Anthropic is doing a better job at signaling their commitment to safety. That's true, obvious, and entirely in line with the charter she's supposed to uphold.

BryantD 3y ago

Just spot checking: did anyone comment on this paper when it was published? Did any media outlet say “hey, a member of the OpenAI board is criticizing OpenAI and showing a conflict of interest?” Did any of the people who cover AI (Zvi, say) discuss this as a problem? These are serious questions, not gotchas. I don’t know the answers, and I think having those answers would make it easier to evaluate whether or not the paper was a significant conflict of interest. The opinions we have formed now are shaped by our biases about current events. It didn’t make HN.

dmix 3y ago
Gambling that no one reads academics papers by fringe Open AI board members no one has heard of before is probably a safe bet but it's still a risk if some doomer AI people on Twitter pumped it up, some journo discovered the tweets and sells it as "concern in the industry including one of OpenAI's own board" and it got swept up in some lawyer-style grilling by Congress which Sam just had to go through.
triplebankshot 3y ago
Who cares if the paper was covered in the media or not? Think tanks write policy papers for regulators, not HN. And it's the regulators that Sam was worried about. From the article: > Mr. Altman said that he had reprimanded Ms. Toner for the paper and that it was dangerous to the company, particularly at a time, he added, when the Federal Trade Commission was investigating OpenAI over the data used to build its technology.