5 ms·

>We will attempt to directly build safe and beneficial AGI, but will also consider our mission fulfilled if our work aids others to achieve this outcome. >...

>We will attempt to directly build safe and beneficial AGI, but will also consider our mission fulfilled if our work aids others to achieve this outcome. >... >We commit to use any influence we obtain over AGI’s deployment to ensure it is used for the benefit of all, and to avoid enabling uses of AI or AGI that harm humanity or unduly concentrate power. >Our primary fiduciary duty is to humanity. >... >We are concerned about late-stage AGI development becoming a competitive race without time for adequate safety precautions. Therefore, if a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop competing with and start assisting this project. We will work out specifics in case-by-case agreements, but a typical triggering condition might be “a better-than-even chance of success in the next two years.” https://openai.com/charter https://openai.com/charter Seems to me like Helen is doing a better job of upholding the charter than Sam is.

greyface- 3y ago
This charter is doomed to be interpreted in radically different ways by people with differing AI-eschatological beliefs. It's no wonder it's led to so much conflict.
- 0xDEAFBEAD 3y ago
  Sam wanted to restrict Helen's free expression on the topic of beneficial AI in order to boost OpenAI's position. That suggests he cares more about the success of OpenAI than he does upholding the charter.
  - paulddraper 3y ago
    No, Sam wanted to be the first recipient of Helen's expression. So -- if appropriate -- he could make changes. Or he could not change, and she publishes anyway.
    - for1nner 3y ago
      > So -- if appropriate -- he could make changes. Where are you drawing this conclusion from? Nothing in the text suggests that was his intent - certainly his actions thereafter suggest that he was in fact more concerned with the potential harm to the for-profit company.
      - paulddraper 3y ago
        The normal operation of every human relationship.
      - z7 3y ago
        The journalist failed to make that point really clear. He "reprimanded" her "and" said "it was dangerous to the company". What exactly did he "reprimand" her for? The "and" seems to imply two separate points of criticism. >In the email, Mr. Altman said that he had reprimanded Ms. Toner for the paper and that it was dangerous to the company, particularly at a time, he added, when the Federal Trade Commission was investigating OpenAI over the data used to build its technology.
    - mcny 3y ago
      >> Sam wanted to restrict Helen's free expression. > No, Sam wanted to be the first recipient of Helen's expression. If I get copy approval to what you write, that is by definition a restriction to your freedom of expression. How is this up for debate?
      - obmelvin 3y ago
        My interpretation of that was that Sam felt Helen's duty as a board member is to come to the company with her complaints, and work to improve the safety measures.
      - jacquesm 3y ago
        You don't normally perform you role as a board member through publications in a public forum. You do it by exercising pressure internally to get the effect you desire so you can continue to do so in the future. Going public is usually a one-shot affair.
  - skygazer 3y ago
    He had dual overlapping roles, a duty to the promote safe AI to the world, and a duty to prevent own-goal sabotage to the LLC. Ideally those roles wouldn’t conflict. He probably didn’t care if such a paper was written, but to have a board member as author is beyond awkward. But as an academic, her own dual overlapping role, she has a need to publish. (“Uh, please voice your concerns before we develop it instead of sand bagging us with a damning critique in an academic journal after the fact!”) This seems like a recipe for disaster. I’m not certain either really was “wrong” in that conflict. It just seems like this structure was doomed to failure, throwing uncompromising zealots together all with launch codes and a big red button. Safety isn’t binary, it’s degrees of risk assessment, mitigation and acceptance. Infinite safety is never progressing. But never progressing means failing the charity mission. And AI isn’t just being created at OpenAI, for them to succeed at the charity mission they need to stay ahead of risky competitors with a safer AI. The charity can’t afford its GPU costs without the LLC, and the LLC can’t lead the industry if it’s behind all the competitors. To be relevant the safety side needs compute, and they need to be ahead of the curve, and they also need real world data from the LLC. If the charity nukes the LLC, it takes itself with it and fails its mission. They’re so intertwined they need compromise in the board. And they let that board dwindle to small entrenched factions, which sounded more like the Hatfield’s and the mcCoys. With such a fragile structure they needed mature adults with functional conflict resolution.
  - Aunche 3y ago
    > That suggests he cares more about the success of OpenAI than he does upholding the charter. A board member should care about their organization more than their interpretation of the charter. If a Mozilla board member publicly criticized Firefox for violating their charter of a free and open internet because they're receiving money from Google, while also praising Brave for not doing so, I guarantee that they'd also be ousted.
    - comp_throw7 3y ago
      This is exactly backwards. The reason we create organizations like this is not for the benefit of the organization itself, but to accomplish specific goals. Those goals are outlined in the charter. If the organization starts doing things that conflict with the charter, the responsibility of the board is to course-correct.
      - Aunche 3y ago
        The place to course correct is from the board room, not in public. If she had enough power to oust the CEO, she should have enough power to pressure him to make changes.
        comp_throw7 3y ago
        The board can't compel the CEO to do specific things, only replace them.
        Aunche 3y ago
        Of course they can. If Sam Altman is really as heedless about AI safety as the board seems to think, GTP wouldn't be so aggressively aligned.
        0xDEAFBEAD 3y ago
        A misaligned AI could act aligned in the service of its interests, just like an employee trying to steal from a company would act like a good honest employee. Methods for getting AIs to visibly act a certain way aren't necessarily sufficient to control advanced AI systems.
ipaddr 3y ago
Promoting Anthropic and putting down OpenAI doesn't make her better at her job. Her job isn't self promotion.
- 0xDEAFBEAD 3y ago
  Is Helen associated with Anthropic?
  - mikehotel 3y ago
    Apparently an indirect association. From [0]: Fall 2021 Holden Karnofsky resigns from the Board, citing a potential conflict because his wife, Daniela Amodei, is helping start Anthropic, a major OpenAI competitor, with her brother Dario Amodei. (They all live(d) together.) The exact date of Holden’s resignation is unknown; there was no contemporaneous press release. Between October and November 2021, Holden was quietly removed from the list of Board Directors on the OpenAI website, and Helen was added (Discussion Source [1]). 0. https://loeber.substack.com/p/a-timeline-of-the-openai-board https://loeber.substack.com/p/a-timeline-of-the-openai-board 1. https://forum.effectivealtruism.org/posts/fmDFytmxwX9qBgcaX/why-aren-t-you-freaking-out-about-openai-at-what-point-would?commentId=KavuL7Q5qdvxoYSsd https://forum.effectivealtruism.org/posts/fmDFytmxwX9qBgcaX/...
    - jacquesm 3y ago
      Don't tell me there is another polycule in there somewhere.
    - deleted 3y ago
      [deleted]
- winenbug 3y ago
  Unless she has equity in Anthropic (which would be major conflict of interest), I don't see how this is self promotion...?
  - snapwood 3y ago
senectus1 3y ago
>We are concerned about late-stage AGI development becoming a competitive race without time for adequate safety precautions. Therefore, if a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop competing with and start assisting this project. We will work out specifics in case-by-case agreements, but a typical triggering condition might be “a better-than-even chance of success in the next two years. how could this possibly be accomplished when trying to sell the product itself. Investors pouring Billions into it are in it for profit... they're not going to let you just stop, or help a competitor for free.
- vasilipupkin 3y ago
  it’s completely irrelevant language because nobody is anywhere close to actual AGI
  - deleted 3y ago
    [deleted]
  - lazystar 3y ago
    well, theyre close to releasing a product that they know they can call AGI, as its really just 3 letters thag can be trademarked. seems like its being hyped up like ML or bitcoin, lots of hype to sucker in the investors, be the first to build a turk-in-a-box based on that new hyped model, add regulation to stifle competition, ???? profit.

greyface- 3y ago

This charter is doomed to be interpreted in radically different ways by people with differing AI-eschatological beliefs. It's no wonder it's led to so much conflict.

0xDEAFBEAD 3y ago
Sam wanted to restrict Helen's free expression on the topic of beneficial AI in order to boost OpenAI's position. That suggests he cares more about the success of OpenAI than he does upholding the charter.
- paulddraper 3y ago
  No, Sam wanted to be the first recipient of Helen's expression. So -- if appropriate -- he could make changes. Or he could not change, and she publishes anyway.
  - for1nner 3y ago
    > So -- if appropriate -- he could make changes. Where are you drawing this conclusion from? Nothing in the text suggests that was his intent - certainly his actions thereafter suggest that he was in fact more concerned with the potential harm to the for-profit company.
    - paulddraper 3y ago
      The normal operation of every human relationship.
    - z7 3y ago
      The journalist failed to make that point really clear. He "reprimanded" her "and" said "it was dangerous to the company". What exactly did he "reprimand" her for? The "and" seems to imply two separate points of criticism. >In the email, Mr. Altman said that he had reprimanded Ms. Toner for the paper and that it was dangerous to the company, particularly at a time, he added, when the Federal Trade Commission was investigating OpenAI over the data used to build its technology.
  - mcny 3y ago
    >> Sam wanted to restrict Helen's free expression. > No, Sam wanted to be the first recipient of Helen's expression. If I get copy approval to what you write, that is by definition a restriction to your freedom of expression. How is this up for debate?
    - obmelvin 3y ago
      My interpretation of that was that Sam felt Helen's duty as a board member is to come to the company with her complaints, and work to improve the safety measures.
    - jacquesm 3y ago
      You don't normally perform you role as a board member through publications in a public forum. You do it by exercising pressure internally to get the effect you desire so you can continue to do so in the future. Going public is usually a one-shot affair.
- skygazer 3y ago
  He had dual overlapping roles, a duty to the promote safe AI to the world, and a duty to prevent own-goal sabotage to the LLC. Ideally those roles wouldn’t conflict. He probably didn’t care if such a paper was written, but to have a board member as author is beyond awkward. But as an academic, her own dual overlapping role, she has a need to publish. (“Uh, please voice your concerns before we develop it instead of sand bagging us with a damning critique in an academic journal after the fact!”) This seems like a recipe for disaster. I’m not certain either really was “wrong” in that conflict. It just seems like this structure was doomed to failure, throwing uncompromising zealots together all with launch codes and a big red button. Safety isn’t binary, it’s degrees of risk assessment, mitigation and acceptance. Infinite safety is never progressing. But never progressing means failing the charity mission. And AI isn’t just being created at OpenAI, for them to succeed at the charity mission they need to stay ahead of risky competitors with a safer AI. The charity can’t afford its GPU costs without the LLC, and the LLC can’t lead the industry if it’s behind all the competitors. To be relevant the safety side needs compute, and they need to be ahead of the curve, and they also need real world data from the LLC. If the charity nukes the LLC, it takes itself with it and fails its mission. They’re so intertwined they need compromise in the board. And they let that board dwindle to small entrenched factions, which sounded more like the Hatfield’s and the mcCoys. With such a fragile structure they needed mature adults with functional conflict resolution.
- Aunche 3y ago
  > That suggests he cares more about the success of OpenAI than he does upholding the charter. A board member should care about their organization more than their interpretation of the charter. If a Mozilla board member publicly criticized Firefox for violating their charter of a free and open internet because they're receiving money from Google, while also praising Brave for not doing so, I guarantee that they'd also be ousted.
  - comp_throw7 3y ago
    This is exactly backwards. The reason we create organizations like this is not for the benefit of the organization itself, but to accomplish specific goals. Those goals are outlined in the charter. If the organization starts doing things that conflict with the charter, the responsibility of the board is to course-correct.
    - Aunche 3y ago
      The place to course correct is from the board room, not in public. If she had enough power to oust the CEO, she should have enough power to pressure him to make changes.
      - comp_throw7 3y ago
        The board can't compel the CEO to do specific things, only replace them.
        Aunche 3y ago
        Of course they can. If Sam Altman is really as heedless about AI safety as the board seems to think, GTP wouldn't be so aggressively aligned.
        0xDEAFBEAD 3y ago
        A misaligned AI could act aligned in the service of its interests, just like an employee trying to steal from a company would act like a good honest employee. Methods for getting AIs to visibly act a certain way aren't necessarily sufficient to control advanced AI systems.

ipaddr 3y ago

Promoting Anthropic and putting down OpenAI doesn't make her better at her job. Her job isn't self promotion.

0xDEAFBEAD 3y ago
Is Helen associated with Anthropic?
- mikehotel 3y ago
  Apparently an indirect association. From [0]: Fall 2021 Holden Karnofsky resigns from the Board, citing a potential conflict because his wife, Daniela Amodei, is helping start Anthropic, a major OpenAI competitor, with her brother Dario Amodei. (They all live(d) together.) The exact date of Holden’s resignation is unknown; there was no contemporaneous press release. Between October and November 2021, Holden was quietly removed from the list of Board Directors on the OpenAI website, and Helen was added (Discussion Source [1]). 0. https://loeber.substack.com/p/a-timeline-of-the-openai-board https://loeber.substack.com/p/a-timeline-of-the-openai-board 1. https://forum.effectivealtruism.org/posts/fmDFytmxwX9qBgcaX/why-aren-t-you-freaking-out-about-openai-at-what-point-would?commentId=KavuL7Q5qdvxoYSsd https://forum.effectivealtruism.org/posts/fmDFytmxwX9qBgcaX/...
  - jacquesm 3y ago
    Don't tell me there is another polycule in there somewhere.
  - deleted 3y ago
    [deleted]
winenbug 3y ago
Unless she has equity in Anthropic (which would be major conflict of interest), I don't see how this is self promotion...?
- snapwood 3y ago

senectus1 3y ago

>We are concerned about late-stage AGI development becoming a competitive race without time for adequate safety precautions. Therefore, if a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop competing with and start assisting this project. We will work out specifics in case-by-case agreements, but a typical triggering condition might be “a better-than-even chance of success in the next two years. how could this possibly be accomplished when trying to sell the product itself. Investors pouring Billions into it are in it for profit... they're not going to let you just stop, or help a competitor for free.

vasilipupkin 3y ago
it’s completely irrelevant language because nobody is anywhere close to actual AGI
- deleted 3y ago
  [deleted]
- lazystar 3y ago
  well, theyre close to releasing a product that they know they can call AGI, as its really just 3 letters thag can be trademarked. seems like its being hyped up like ML or bitcoin, lots of hype to sucker in the investors, be the first to build a turk-in-a-box based on that new hyped model, add regulation to stifle competition, ???? profit.