27 ms·

Before Altman’s ouster, OpenAI’s board was divided and feuding

304 points by vthommeret 3y ago

tempusalaria 3y ago
This claims that Anthropic founders also tried to throw Sam out before leaving. They claim three sources but not clear how strong - technically the current board could be three sources
- deleted 3y ago
  [deleted]
- gumballindie 3y ago
  Yeah thats an interesting statement. People leaving or wanting to get him out usually indicates that the person - altman in this case - is quite toxic. Being a toxic person yet preferred by microsoft and others indicates he’s “willing”, meaning he will do whatever they ask of him. And that’s not in the best interest of openai.
- simonw 3y ago
  Skilled journalists (and this piece was written by skilled journalists) would not use the three members of the board as their only three sources. A big part of this kind of journalism is taking information from sources, considering the potential bias of each source and carefully correlating it with information from other sources before publishing it. That bit specifically says: "After they failed, they gave up and departed, according to three people familiar with the attempt to push Mr. Altman out." My guess is that at least one of those people comes from the Anthropic side, who would be someone with a clear insight into what happened.
  - simonw 3y ago
    This inspired me to write an article about how to decipher clues like "three people familiar with the attempt": https://simonwillison.net/2023/Nov/22/deciphering-clues/ https://simonwillison.net/2023/Nov/22/deciphering-clues/
  - JoshuaDavid 3y ago
    > Skilled journalists (and this piece was written by skilled journalists) would not use the three members of the board as their only three sources. Depends on whether their goal is to inform or to convince.
PepperdineG 3y ago
>Ms. Toner disagreed. The board’s mission is to ensure that the company creates artificial intelligence that “benefits all of humanity,” and if the company was destroyed, she said, that mission would be fulfilled. Now we know where that came from
- jacquesm 3y ago
  That's a bit naive, to put it mildly. It presumes that nobody else would be able to replicate the effort and that the parties that are able to replicate it would also destroy theirs after proving that it could be done. Fat chance.
  - barrkel 3y ago
    Indeed. The only way to retain control of the leading ship in the race is to keep it together as you steer it. If the ship disintegrates, then you're no longer in control of the leading ship, and someone else will win the race.
    - 0xDEAFBEAD 3y ago
      The charter states that OpenAI should allow another company to win the race under certain circumstances: >We are concerned about late-stage AGI development becoming a competitive race without time for adequate safety precautions. Therefore, if a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop competing with and start assisting this project. We will work out specifics in case-by-case agreements, but a typical triggering condition might be “a better-than-even chance of success in the next two years.” https://openai.com/charter https://openai.com/charter That's why Sam pressuring Helen to shut up about Anthropic is so concerning.
      - jacquesm 3y ago
        > we commit to stop competing with and start assisting this project That bit I never understood. It doesn't even have a qualification as to who is stewarding the project as if any AI should be assisted no matter who ends up owning it. The whole thing is just so utterly disconnected from the real world.
CSMastermind 3y ago
The only specific accusation made in the article is that Sam criticized Helen Toner for writing a paper: https://cset.georgetown.edu/publication/decoding-intentions/ https://cset.georgetown.edu/publication/decoding-intentions/ That says Anthropic has a better approach to AI safety than OpenAI. Sam apparently said she should have come to him directly if she had concerns about the company's approach and pointed out that as a board member her words have weight at a time when he was trying to navigate a tricky relationship with the FTC. She apparently told him to kick rocks and he started to look for ways to get her off the board. All of that ... seems completely reasonable? Like I've heard a lot of vague accusations thrown at Sam over the last few days and yet based on this account I think he reacted the exact same way any CEO would. I'm much more interested in how Helen managed to get on this board at all.
- deleted 3y ago
  [deleted]
- jacquesm 3y ago
  It's interesting but it may well be they both have a point: Helen for telling him to get lost and Sam for attempting to remove her before she would damage the company. But she could have made that point more forcefully by not comparing Anthropic to OpenAI, after all who better than her to steer OpenAI in the right direction. I noted in a comment elsewhere that all of these board members appear to have had at least one and some many more conflicts of interest. Helen probably believes that her loyalty is not to OpenAI but to something higher than that based on her remark that destroying the company would serve to fulfil its mission (which is a very strange point of view to begin with). But that doesn't automatically mean that she's able to place it in context, within OpenAI, within the USA, the Western world and the world as a whole. It's like saying the atomic bomb would have never been invented if the people at Los Alamos didn't do it. They did it in three years after it became known that it could be done in principle. Others tried and failed but without the same resources. I suspect that if the USA had not done it that eventually France, the UK and Russia would have gotten there as well and later on China. Israel would not have had the bomb without the USA (willing or unwilling) and India and Pakistan would have achieved it but much later as well. So we'd end up with the same situation that we have today modulo some timing differences and with another last chapter on WWII. Better? Maybe. But it is also possible that the Russians would have launched a first strike on the USA if they were unopposed. It almost happened as it was! The open question then is: does she really believe that no other entity has the resources to match OpenAI and does she believe that if such an entity does exist that it too will self destruct rather than to go through with the development? And does she believe that this will hold true for all time? That they and their colleagues are so unique that they hold the key to something that can otherwise not be replicated.
throw20away 3y ago
Ms. Toner’s motive is now clearer for being criticized about her research. Mr. D’Angelo had a competing commercialization product with Poe. Mr. Sutskever seems easy to manipulate emotionally and is in constant ideological battles. Mrs. McCauley AKA Joseph Gordon-Levitt’s wife, what’s her motive?
- brigadier132 3y ago
  Apparently she is also an "Effective Altruist" (along with Toner) so they are ideologically aligned.
  - alvah 3y ago
    EA hadn’t had a good run in the corporate world, over the last year or so
    - 0xDEAFBEAD 3y ago
      First SBF gets busted for being too greedy, and now Helen Toner gets busted because she's not greedy enough. EA just can't catch a break.
      - alvah 3y ago
        People with a child’s view of the world, in charge of $billions of other people’s money, is the common theme.
        0xDEAFBEAD 3y ago
        There's been a ton of criticism of Helen here in this thread and elsewhere, but I haven't seen anyone clearly articulate what Helen should have done. Everyone says she did the wrong thing, but no one has said what the right thing actually was. Here's my guess: I think OpenAI should have prioritized character more in its hiring. Ending up with an organization full of employees who cared more about making money than upholding the charter was a mistake. Ultimately it's not enough for the people on the board to have character. The CEO and employees need to have character too.
DueDilligence 3y ago
[dead]
kragen 3y ago
keep in mind that the lead reporter on this, cade metz, is the one who wrote the character assassination piece on scott alexander repeatedly insinuating he was a neo-nazi (and de-anonymizing him, costing him his job due to some kind of weird psychiatrist ethical code) so while probably nothing in here is literally false, it's quite likely calculated to give false impressions; read with caution (well, it's literally false that "Greg Brockman (...) quit his role[] as (...) board chairman" but only slightly; that was the role he was fired from, as explained in the next paragraph of the article; that's not the kind of lies to watch out for)
- dekhn 3y ago
  wasn't that piece never published?
  - comp_throw7 3y ago
    No, it's right here: https://www.nytimes.com/2021/02/13/technology/slate-star-codex-rationalists.html https://www.nytimes.com/2021/02/13/technology/slate-star-cod... (See also Scott's response: https://www.astralcodexten.com/p/statement-on-new-york-times-article https://www.astralcodexten.com/p/statement-on-new-york-times...)
    - kragen 3y ago
      (-)
      - comp_throw7 3y ago
        My bad, edited.
    - dekhn 3y ago
      It's a pretty interesting article if you know how to read around the NYTimes's rhetoric. """The issue, it was clear to me, was that I told him I could not guarantee him the anonymity he’d been writing with. In fact, his real name was easy to find because people had shared it online for years and he had used it on a piece he’d written for a scientific journal. I did a Google search for Scott Alexander and one of the first results I saw in the auto-complete list was Scott Alexander Siskind.""" Reading the response, I don't think this is condeming Metz to live in infamy.
pasltd 3y ago
Here you go: https://www.nytimes.com/2023/11/21/technology/openai-altman-board-fight.html?unlocked_article_code=1.AU0.0Zli.ur35-H19FBLR&smid=url-share https://www.nytimes.com/2023/11/21/technology/openai-altman-...
- dang 3y ago
  I moved your comment from https://news.ycombinator.com/item?id=38372541 https://news.ycombinator.com/item?id=38372541 to here, where it seems like it can help more readers. I hope that's ok!
  - jessenaser 3y ago
    Thank you.
breadwinner 3y ago
The main takeaway from this whole saga is, be careful who you allow on your board. Nearly everyone on OpenAI board is young, inexperienced and unqualified to be on the board of one of the most important companies on the planet. Here's Ms. Toner's linkedin: https://www.linkedin.com/in/helen-toner-4162439a/ https://www.linkedin.com/in/helen-toner-4162439a/
- jjulius 3y ago
  >Nearly everyone on OpenAI board is young, inexperienced and unqualified to be on the board of one of the most important companies on the planet. Jumping off of that, I'm genuinely curious what, in Sam's resume, suggests he should be in charge of "one of the most important companies on the planet".
  - avarun 3y ago
    The fact that he's a founder and the one that made it "one of the most important companies on the planet"?
    - jjulius 3y ago
      Not quite true - OpenAI's Wikipedia article says... >The organization was founded in December 2015 by Ilya Sutskever, Greg Brockman, Trevor Blackwell, Vicki Cheung, Andrej Karpathy, Durk Kingma, Jessica Livingston, John Schulman, Pamela Vagata, and Wojciech Zaremba, with Sam Altman and Elon Musk serving as the initial board members. Altman didn't join as CEO until 2019. Edit: All those names and we're going to say stuff like Altman was "the one" who made the company so important?
      - breadwinner 3y ago
        Not true. This page dated December 11, 2015 mentions Sam Altman and Elon Musk as co-chairs: https://openai.com/blog/introducing-openai https://openai.com/blog/introducing-openai
neonate 3y ago
https://archive.ph/eN5PY https://archive.ph/eN5PY
hintymad 3y ago
>Ms. Toner disagreed. The board’s mission is to ensure that the company creates artificial intelligence that “benefits all of humanity,” and if the company was destroyed, she said, that mission would be fulfilled. Doesn’t this sound like Toner herself is playing god? That is, only she is the gatekeeper of humanity and only she knows what is good for the entire humanity, when many of us believe that OpenAI’s tech is perfectly safe and amazingly beneficial?
- hollerith 3y ago
  >Doesn’t this sound like Toner herself is playing god? That is, only she is the gatekeeper of humanity and only she knows what is good for the entire humanity, when many of us believe that OpenAI’s tech is perfectly safe and amazingly beneficial? Yeah, well, hate to burst your nice warm Silicon-Valley bubble, but most Americans believe that continuing to create smarter and smarter AIs is dangerous, so who's playing god now? (No one I know thinks it is dangerous to fine-tune GPT-4 or to integrate GPT-4 deeply into the economy: the danger is the plans for the creation of much bigger models and algorithmic improvements.)
  - TheOsiris 3y ago
    > but most Americans believe that continuing to create smarter and smarter AIs is dangerous, so who's playing god now? Aren't you just making this up? I haven't seen any surveys on what "most Americans believe" re AI. I know a lot of people that are concerned. But I'll bet good money that "most Americans" don't give a crap
    - comp_throw7 3y ago
      Googling "americans AI polling" produces this as the first result: https://www.pewresearch.org/short-reads/2023/11/21/what-the-data-says-about-americans-views-of-artificial-intelligence/ https://www.pewresearch.org/short-reads/2023/11/21/what-the-... I don't put much stock into polls like this, when it comes to actually deciding whether things are dangerous, but I think you should spend five seconds doing an incredibly trivial sanity-check before accusing people of making things up.
Bukhmanizer 3y ago
I think it’s getting to the point where the noise is overwhelming the signal in this story.
- Philpax 3y ago
  Really? I found this to be a very insightful article; it made it much clearer for me why the board acted as they did. It doesn't have all of the details, but it's much more than I had yesterday.
jessenaser 3y ago
> Mr. Altman complained that the research paper seemed to criticize OpenAI’s efforts to keep its A.I. technologies safe while praising the approach taken by Anthropic, according to an email that Mr. Altman wrote to colleagues and that was viewed by The New York Times. > OpenAI's board of directors approached rival Anthropic's CEO about replacing chief Sam Altman and potentially merging the two AI startups, according to two people briefed on the matter. (https://www.reuters.com/technology/openais-board-approached-anthropic-ceo-about-top-job-merger-sources-2023-11-21/ https://www.reuters.com/technology/openais-board-approached-...) It all makes sense.
- soderfoo 3y ago
  And the whole time no one thought to phone Satya Nadella regarding his $10 billion investment and to get his input.
  - wnoise 3y ago
    That's literally not the job of this board.
  - HDThoreaun 3y ago
    Why would the board care about that?
  - function_seven 3y ago
    Good. His money came with no strings*. In fact, if they had run this past him before making their decision, that would have been a breach of their duty as outlined in their charter. They may have made mistakes with timing (making this decision too late), or with execution (not having a replacement CEO solidly lined-up and in the loop), but the core decision—to remove Sam—is definitely not something they should ask donors about. It's not Satya's business how the board makes decisions at this level. * Other than a capped profit return if there is any.
  - dkjaudyeqooe 3y ago
    Who cares what he thinks, his money is gone and he has no control over it. In fact the most positive outcome is if Altman and the rest of the staff went to MS and did their thing and OpenAI started from scratch with the $13B they've come into. That would double the chances of something useful emerging from the OpenAI work so far. I personally think Altman is very much less than a genius (maybe at extracting financial advantage) so all OpenAI's eggs shouldn't be placed in that particular basket.
winenbug 3y ago
https://openai.com/our-structure https://openai.com/our-structure This whole thing was so, SO poorly executed, but the independent people on the board were gathered specifically to prioritize humanity & AI safety over OpenAI. It sounds like Sam forgot just that when he criticized Helen for her research (given how many people were posting ways to "get around" ChatGPT's guardrails, she probably had some firm grounds to stand on). Yes, Sam made LLMs mainstream and is the face of AI, but if the board believes that that course of action could destroy humanity it's literally the board's mission to stop it — whether that means destroying OpenAI or not. What this really shows us is that this "for-profit wrapper around a non-profit" shenanigans was doomed to fail in the first place. I don't think either side is purely in the wrong here, but they're two sides of an incredibly badly thought-of charter.
- 0xDEAFBEAD 3y ago
  >they're two sides of an incredibly badly thought-of charter. It's easy to say this with the benefit of hindsight, but I haven't seen anyone in this discussion even suggest an alternative model that they claim would've been superior.
  - winenbug 3y ago
    Agreed, I'm not saying I have a better alternative, just that this is something we all should now realize; given i'm sure we were all wondering for a long time what the whole governance structure of OpenAI really meant (capped for-profit with non-profit mission etc.)
- lll-o-lll 3y ago
  > It sounds like Sam forgot just that when he criticized Helen for her research (given how many people were posting ways to "get around" ChatGPT's guardrails, she probably had some firm grounds to stand on). Sam didn’t forget anything. He is a brilliant Machiavellian operator. Just look at the Reddit reverse takeover as an example; Machiavelli would be in awe. > What this really shows us is that this "for-profit wrapper around a non-profit" shenanigans was doomed to fail in the first place. No. It shows this structure is doomed to fail if you have a genius schemer as a CEO, playing the long game to gain unrestricted control.
1vuio0pswjnm7 3y ago
"OpenAI was started in 2015 with an ambitious plan to one day create a superintelligent automated system that can do everything a human brain can do." Sounds absurd. No one even knows everything the human brain does. It is poorly understood.
6gvONxR4sf7o 3y ago
tldr: The new part of the story this adds is that Altman’s firing was partly in response to him trying to kick out one of the other board members.
brigandish 3y ago
As an aside, I must say that I'm fascinated by this use of the word ouster. In British English I've not heard it used this way, we'd use ousting in that place and the ouster would be the one doing the ousting, i.e. the person doing the pushing. I thought I'd check with a quick search of the Guardian, and on two different days it used both in the same sense: From the 18th of November edition, technology section:[1] > The crisis at OpenAI deepened this weekend when the company said Altman’s ousting was for allegedly misleading the board. From 17th of November edition, also technology section[2]: > The announcement blindsided employees, many of whom learned of the sudden ouster from an internal announcement and the company’s public facing blog. I could only find one instance in the Telegraph prior to Altman's erm, ousting[3]: > Johnson dropped into the COP27 UN climate change conference in October, joking unusual summer heat had played a part in his ouster, and has vowed to keep championing Ukraine. They stick to ousting every other time. I wonder if it may be an artefact of newspapers using news services to get copy from, or a stylistic rule for international editions, as the Guardian does use ouster several other times but always in stories regarding US news. Also fascinating that we still don't know who that is. Neither do most of the staff, and even the board, apparently. A true mastermind! Maybe it is GPT5 after all… <cue ominous music, or perhaps, a Guns n' Roses album?> [1] https://www.theguardian.com/technology/2023/nov/18/earthquake-at-chatgpt-developer-as-senior-staff-quit-after-sacking-of-boss-sam-altman https://www.theguardian.com/technology/2023/nov/18/earthquak... [2] https://www.theguardian.com/technology/2023/nov/17/openai-ceo-sam-altman-fired https://www.theguardian.com/technology/2023/nov/17/openai-ce... [3] https://www.telegraph.co.uk/news/2022/12/27/boris-johnson-will-make-political-comeback-2023/ https://www.telegraph.co.uk/news/2022/12/27/boris-johnson-wi...
crmd 3y ago
I thought the drug dealers in my hometown were ruthless sociopaths until I had to start doing deals with change-the-world mild mannered old navy-wearing Stanford Silicon Valley dudes.
1vuio0pswjnm7 3y ago
"In the email, Mr. Altman said that he had reprimanded Ms. Toner for the paper and that it was dangerous to the company, particularly at a time, he added, when the Federal Trade Commission was investigating OpenAI over the data used to build its technology."
QuantumGood 3y ago
It sounds like the board got ever more concentrated towards less sanity over time.
gwern 3y ago
None of the comments thus far seem to clearly explain why this matters. Let me summarize the implications: Sam Altman expelling Toner with the pretext of an inoffensive page (https://cset.georgetown.edu/wp-content/uploads/CSET-Decoding-Intentions.pdf#page=30 https://cset.georgetown.edu/wp-content/uploads/CSET-Decoding...) in a paper no one read* would have given him a temporary majority with which to appoint a replacement director, and then further replacement directors. These directors would, naturally, agree with Sam Altman, and he would have a full, perpetual board majority - the board, which is the only oversight on the OA CEO. Obviously, as an extremely experienced VC and CEO, he knew all this and how many votes he (thought he) had on the board, and the board members knew this as well - which is why they had been unable to agree on replacement board members all this time. So when he 'reprimanded' her for her 'dangerous' misconduct and started talking seriously about how 'inappropriate' it was for a 'board member' to write anything which was not cheerleading, and started leading discussions about "whether Ms Toner should be removed"... * I actually read CSET papers, and I still hadn't bothered to read this one, nor would I have found anything remarkable about that page, which Altman says was so bad that she needed to be expelled immediately from the board.
- abi 3y ago
  I mean it's not cool for board members to publicly criticize the company.
  - kragen 3y ago
    it's not cool for companies to try to shut down academic freedom of inquiry in scholarly publishing in order to improve their public image her independence from openai was the nominal reason she was asked to supervise it in the first place, wasn't it
    - foota 3y ago
      I'm not sure you should get to have your cake and eat it too.
      - 3y ago
lwneal 3y ago
The relevant passage from the paper co-written by board member Helen Toner: "OpenAI has also drawn criticism for many other safety and ethics issues related to the launches of ChatGPT and GPT-4, including regarding copyright issues, labor conditions for data annotators, and the susceptibility of their products to "jailbreaks" that allow users to bypass safety controls... A different approach to signaling in the private sector comes from Anthropic, one of OpenAI's primary competitors. Anthropic's desire to be perceived as a company that values safety shines through across its communications, beginning from its tagline: "an AI safety and research company." A careful look at the company's decision-making reveals that this commitment goes beyond words." [1] https://cset.georgetown.edu/publication/decoding-intentions/ https://cset.georgetown.edu/publication/decoding-intentions/
- murakamiiq84 3y ago
  I think this is heavily editoralized. if you look at the 3 pages in question that the quotes are pulled from (28-30 in doc, 29-31 in pdf), they appear to be given as examples in pretty boring academic discussions explicating the theories of costly signaling in the context of AI.It also has lines like: "The system card provides evidence of several kinds of costs that OpenAI was willing to bear in order to release GPT-4 safely.These include the time and financial cost..." "Returning to our framework of costly signals, OpenAI’s decision to create and publish the GPT4 system card could be considered an example of tying hands as well as reducible costs. By publishing such a thorough, frank assessment of its model’s shortcomings, OpenAI has to some extent tied its own hands—creating an expectation that the company will produce and publish similar risk assessments for major new releases in the future. OpenAI also paid a price ..." "While the system card itself has been well received among researchers interested in understanding GPT-4’s risk profile, it appears to have been less successful as a broader signal of OpenAI’s commitment to safety" And the conclusion: "Yet where OpenAI’s attempt at signaling may have been drowned out by other, even more conspicuous actions taken by the company, Anthropic’s signal may have simply failed to cut through the noise. By burying the explanation of Claude’s delayed release in the middle of a long, detailed document posted to the company’s website, Anthropic appears to have ensured that this signal of its intentions around AI safety has gone largely unnoticed. Taken together, these two case studies therefore provide further evidence that signaling around AI may be even more complex than signaling in previous eras."
jafitc 3y ago
Sam Altman's Actions - Sam complained that Helen Toner's research paper criticized OpenAI's safety approach and praised Anthropic's, seeing it as dangerous for OpenAI - He reprimanded her for the paper, saying as a board member she should have brought concerns to him first rather than publishing something critical right when OpenAI was in a tricky spot with the FTC - He then began looking for ways to remove her from the board in response to the paper --- Helen Toner's Perspective - She believes the board's mission is to ensure OpenAI makes AI that benefits humanity, so destroying the company would fulfill that mission - This suggests she prioritizes that mission over the company itself, seeing humanitarian concerns as more important than OpenAI's success --- Microsoft Partnership - The Microsoft partnership concentrated too much power in one company and went against the mission of OpenAI being "open" - It gave Microsoft full access to OpenAI's core technologies against the safety-focused mission --- Governance Issues - The conflict shows the adversarial tensions inherent in OpenAI's structure between nonprofit and for-profit sides - The board's mandate to act as a check and balance on OpenAI seems to be working as intended in this case --- Criticisms of Players - Altman appears reckless in his actions, while Toner seems naive about consequences of destroying OpenAI - Their behavior calls into question whether anyone should have this kind of power over the development of AI --- Future of AI Development - Attempts at alignment and safeguards by companies like OpenAI may be ineffective if other actors are developing AI without such considerations - Who controls advanced AI is more important than whether the AI is friendly. - Nationalization of AI projects may occur
- jacquesm 3y ago
  Nice summary.
  - mpalmer 3y ago
    "Thank you! If you have any other questions, feel free to ask. I'm here to help!" - the author of the summary, probably
- richk449 3y ago
  Is there a name for this type of analysis?
jdprgm 3y ago
I really don't understand why the three board members have just been completely silent online since things hit the fan with no activity or privating profiles. You would think if this was some sort of ideological based coup and pre-meditated you would have a full PR plan in place and pushing your message aggressively. This whole thing has just been bizarre and it still feels like there has to be some big key piece missing that somehow nobody has revealed.
- jacquesm 3y ago
  In a word: liability.
- superultra 3y ago
  I think anyone that has worked in academia or the non-profit world is familiar with how this board appears to be operating and certainly its speed, or lack thereof. Compared to tech, it’s glacial. Altman et al has media training, the full weight of Microsoft PR/comms. What do the three board members have? Barely anyone even really knew some of their names until this weekend.
foota 3y ago
Definitely seems like Helen is emerging as the main leader of the coup, "Ms. Toner disagreed. The board’s mission is to ensure that the company creates artificial intelligence that “benefits all of humanity,” and if the company was destroyed, she said, that could be consistent with its mission. In the board’s view, OpenAI would be stronger without Mr. Altman."
- comp_throw7 3y ago
  If the events happened as described (https://news.ycombinator.com/item?id=38373572 https://news.ycombinator.com/item?id=38373572), calling it a "coup" sure is an interesting framing.
- zucker42 3y ago
  The "leader of the coup" has changed each day. First it was Ilya, then D'Angelo because of Poe, then Toner because he criticized her paper.
- dkjaudyeqooe 3y ago
  How is it a coup when it's the boards job to hire and fire the CEO?
jejeyyy77 3y ago
Looks like the board did the right thing after all.
webappguy 3y ago
How can anyone not think Adam D'Angelo is upset about open AI crushing his shitty Poe
fnordpiglet 3y ago
“””Ms. Toner disagreed. The board’s mission is to ensure that the company creates artificial intelligence that “benefits all of humanity,” and if the company was destroyed, she said, that could be consistent with its mission.””” This is remarkable and unhinged. I gave the board the benefit of the doubt at first, but as this unfolds, it becomes clear that the board is held hostage by ideological zealots who can’t compromise. The story also seems to paint Ilya as a manipulated figure, being bent against his beliefs by crazies like Toner exploiting his concerns about AI safety. What an absolute shame. I am entirely ambivalent about Altman overall, but he becomes more sympathetic as the days go by.
- swatcoder 3y ago
  > he becomes more sympathetic as the days go by PR in action! He's had an excellent PR machine running since Friday and the opposing faction seems to have exactly none (which makes sense given their relative roles and backgrounds). So reporters and Twitter get carefully crafted leaks, tips, and comments from one side and nobody on the other side has the experience, connections, and confidence to push back with a different narrative. And so the story drifts in their favor. This is why professional PR is a thing and earns people lots of money.
  - orpheansodality 3y ago
    Sometimes a "PR machine" is just social capital at work. If there's anything else I've learned in this saga it's that Sam has had a consistent track record of building and maintaining highly positive relationships with anyone near his orbit, this board aside, and that it consistently pays him dividends.
- leroy_masochist 3y ago
  She literally wanted to destroy OpenAI in order to save it. Unbelievable. We need a total and complete shutdown of EAs serving on startup boards until we figure out what the hell is going on.
- comp_throw7 3y ago
  It's not unhinged, it's explicitly within the scope of OpenAI's charter. Sam Altman himself said that "Development of superhuman machine intelligence (SMI) is probably the greatest threat to the continued existence of humanity." not even a year before OpenAI was founded: https://blog.samaltman.com/machine-intelligence-part-1 https://blog.samaltman.com/machine-intelligence-part-1
deleted 3y ago
[deleted]
clnq 3y ago
A lot of people in tech say that executives are excessively diplomatic and do not speak their truth. But this is what happens when they do too much, too ardently, too often. This is why diplomacy and tact is so important in these roles. Things do not go well if everyone keeps poking each other with sticks and cannot let their own frame of reference go for the sake of the bigger picture. Ultimately, I don’t think Altman doesn’t believe ethics and safety is important. And I don’t think Toner fails to realize that OpenAI is only in a place to dictate what AI will be due to its commercial principles. And they probably both agree that there is a conflict there. But what tactful leadership would have done is found a solution behind closed doors. Yet from their communication, it doesn’t even look like they defined the problem statement — everyone offers a different idea of the problem that they had to face together. It looks more like it was more like immature people shouting past each other for a year (not saying it was that, but it looks that way). Moral of the story: tact, grace, and diplomacy are important. So is speaking one’s truth, but there is a tactful time, place, and manner. And also, no matter how brilliant someone is, if they can’t develop these traits, they end up rocking the boat a lot.
- jacquesm 3y ago
  Spot on.
Uptrenda 3y ago
How do I filter news of this.
jibal 3y ago
why are so many people so keen on this prepper?
oraphalous 3y ago
Fascinating! Apparently central to the complete farce going on at OpenAI - is this paper by one of the board members: https://cset.georgetown.edu/publication/decoding-intentions/ https://cset.georgetown.edu/publication/decoding-intentions/ It's about - my favourite topic: Costly Signaling! How curious - that such a topic would be the possible catalyst for one of the most extraordinary collapses of a leading tech company. Specifically - the paper is about using costly signalling as a means to align (or demonstrate algnment) various kinds of AI interested entities (governments, private corporations, etc) with the public good. The gist of costly signalling - to try and convince others you really mean what you say - you use a signal that is very expensive to you in some respect. You don't just ask a girl to marry you, you buy a big diamond ring! The idea being - cheaters are much less likely to suffer such expense. Apparently the troubles at OpenAI escalated when one of the board members - Helen Toner - published this paper. It is critical of OpenAI, and Sam Altman was pissed at the reputational damage to the company and wanted her removed. The board instead removed him. The gist of the paper's criticisms is that while OpenAI has invested in some costly signals to indicate its alignment with AI safety, overall, it judges those signals were ultimately rather weak (or cheap). Now here is what I find fascinating about all this: up until reading this paper I had found the actions of the OpenAI board completely baffling, but now suddenly their actions make a kind of insane sense. They are just taking their thinking on costly signalling to its logical conclusion. By putting the entire fate of the company and its amazing market position at risk - they are sending THE COSTLIEST SIGNAL possible, relatively speaking: willingness to suffer self-annihilation. Academics truly are wondrous people... that they can lose themselves in a system of thought so deeply, in a way regular people can't. I can't help but have a genuine, sublime appreciation for this, even while thinking they are some of the silliest people on this planet. Here's where I feel they went wrong. Costly signals by and large should be without explicit intention. If you are consciously sending various signals that are costly - you are probably a weirdo. Systems of costly signalling work because they are implicit, shared and in many respects, innate. That's why even insects can engage in costly signalling. But these folk see costly signals as an explicit activity to be engaged in as part of explicit public policy - and unsurprisingly, see it riddled with ambiguity. Of course it would be - individual agents can't just make signals up, and expect the system to understand them. Semiotics biatch.... But rather than reflect on this they double down on signalling as an explicit policy choice. How do they propose to reduce ambiguity? Even costlier signals! It's no wonder then they see it as entirely rational to accept self destruction as a possibility. That's how they escape the horrid existential dread of being doubted by the other. In biology though, no creatures survived in the long-run to reproduce where they invested in costly signals that that didn't confer at least as much, if not more benefit to them in excess of what they paid in the first place. Those that ignore this basic cost-benefit analysis in their signalling will suffer the ignomony of being perceived as ABSOLUTE NUTTERS. Which is exactly how the world is thinking about the OpenAI board. The world doesn't see a group of highly safety aligned AI leaders. The world sees a bunch of disfunctional crazy people.
alex_young 3y ago
This is the most amazing part of the article imo: Ms. Toner disagreed. The board’s mission is to ensure that the company creates artificial intelligence that “benefits all of humanity,” and if the company was destroyed, she said, that could be consistent with its mission. In the board’s view, OpenAI would be stronger without Mr. Altman. Corporate suicidal ideation.
- jacquesm 3y ago
  Par for the course for EA and the nine dwarfs of Eschatology.
- dkjaudyeqooe 3y ago
  If your mission is to not harm humanity and you think your charity is going to harm humanity then its destruction fulfils that mission.
GreedClarifies 3y ago
Sounds like Sam was attempting to fix the board membership. OpenAI had grown massively since some of the board members were installed. Some of them were simply not the caliber of people that one would have running such a prestigious institution, especially not with the weight they had due to the board being depopulated. Sam realized this and maybe was attempting to address the issue. Some of the members (ahem, Helen, Tasha and to a lesser extent Adam) liked their positions and struck first, probably convincing poor Ilya that this was about AI safety. Being lightweights they did not do any pre-work or planning, they just plowed ahead. They didn't think through that Sam and Greg have added tremendous value to the company and that the company would favor them far over the board that added zero value. They didn't think through that tech in general would see rain makers and value creators being cut loose and side with them instead of figureheads. They didn't think that partners and customers, who dealt with Sam and Greg daily would be find the move disconcerting (at a minimum). They didn't even think through who would be the next CEO. Maybe they didn't think it through since they didn't care. There was only upside for them since Sam was going to get rid of them sooner or later. They didn't see that having been on the OpenAI board was an honor and enormous career boost. Or maybe their ambition was so great that nothing mattered but controlling OpenAI. Further, they thought that if they slandered Sam that he would be cowed and that they would retain their power. I wonder how many times they had pulled this stunt in the past and it worked?
zoogeny 3y ago
> Mr. Sutskever’s frustration with Mr. Altman echoed what had happened in 2021 when another senior A.I. scientist left OpenAI to form the company Anthropic. That scientist and other researchers went to the board to try to push Mr. Altman out. So Altman faced another similar challenge to his authority and prevailed. I recall hearing that Anthropic started because the people who had left were unhappy with OpenAIs track record on AI safety. > In the email, Mr. Altman said that he had reprimanded Ms. Toner for the paper and that it was dangerous to the company, > Mr. Altman complained that the research paper seemed to criticize OpenAI’s efforts to keep its A.I. technologies safe > Senior OpenAI leaders [...] later discussed whether Ms. Toner should be removed That paints a pretty bleak picture that isn't favorable to Altman. Two times he was challenged about OpenAIs safety and both times he worked to purge those who opposed him. I can't tell if this is a contention between accelerationism and deccelerationism or if it is a contention between safety and recklessness. Is Altman ignoring the warnings his employees/peers are giving him and retaliating against them? Or is he facing insubordination? I wish OpenAI would split neatly into two. But based on the heightened emotions caused by the insane amount of PR I only see two outcomes. Altman returns and OpenAI stays unified vs. Altman stays at MS and OpenAI is obliterated. I am guessing Altman is hoping that senior management will choose a unified OpenAI at all costs, including ignoring the red flags above. He has engineered a situation where the only way OpenAI remains unified is if he returns.
- jiggawatts 3y ago
  > I recall hearing that Anthropic started because the people who had left were unhappy with OpenAIs track record on AI safety. Anthropic announced Claude 2.1 just recently, to which HN members notably responded by saying that their models are "overly restricted", to the point of uselessness. E.g.: https://news.ycombinator.com/item?id=38366134 https://news.ycombinator.com/item?id=38366134 and: https://news.ycombinator.com/item?id=38368072 https://news.ycombinator.com/item?id=38368072 etc.: https://news.ycombinator.com/item?id=38370590 https://news.ycombinator.com/item?id=38370590 It seems that those people left Open AI because they're insane and think that AI safety means to completely lobotomize the models to the point of uselessness. That last bit is not hyperbole: right here on HN many people were complaining that the Claude model is useless to them because it refuses to obey orders and keeps interjecting some absurd refusal such as "killing a process is too violent" instead of providing Python code for this harmless activity. It's hard to put into words exactly, but I feel like that there's a disconnect in the minds of these people and the reality on the ground, and those false axioms or assumptions when driven to their logical conclusion lead to nonsense. I suspect that Sam Altman doesn't share these delusions, and is more sane and commercially minded. Models like GPT3, 4, and the upcoming v5 have no risk of "escaping" and taking over humanity. None. Zero. Zip. Similarly, they're not much more (in practice) than a fancy search engine. Search engines let their users search for whatever they want, including hateful content, violent content, racist, sexist, or whatever. So why lobotomise the AI? That's undermining their utility, making them much less useful! An here's the thing: their potential utility is enormous. But not if they're dumbed down to the point of uselessness! Microsoft is throwing 10+ billion dollars at OpenAI and Sam Altman precisely because of this potential utility. I can see this straightforward commercial mindset of utility => $$$ being incompatible with what is rapidly turning into an anti-useful-AI mindset that is bordering on insane religious zealotry.
didibus 3y ago
> Ms. Toner disagreed. The board’s mission is to ensure that the company creates artificial intelligence that “benefits all of humanity,” and if the company was destroyed, she said, that could be consistent with its mission We're in true sci-fi land, when we're discussing it might be best to destroy SkyNet before it's too late
jacquesm 3y ago
I'm beginning to strongly associate Effective Altruism with out-of-control naivety.
coffeeshopgoth 3y ago
Having served on boards, the one thing I have not seen that would be really key (and maybe I just have not been lucky in my searches), is something that answers the question of what is the language around releasing information regarding the company to the press or in a research capacity? Typically there are rules that are set up to give the board notice of anything going out to the public, be it an article or research paper, that allows for a heads up or gives time to discuss the implications of such a thing. While whistleblowers are necessary, was there a need to be a sort of whistleblower in this case? Was there adequate board discussion around the subject and the paper before the release? If not, nobody needs a rogue board member like that and it was definitely not in the interest of the company - she is the one at fault here. If that process did happen, she definitely did the right thing and shame on the board for not getting out in front of it.

304 points by vthommeret 3y ago

This claims that Anthropic founders also tried to throw Sam out before leaving. They claim three sources but not clear how strong - technically the current board could be three sources

deleted 3y ago
[deleted]
gumballindie 3y ago
Yeah thats an interesting statement. People leaving or wanting to get him out usually indicates that the person - altman in this case - is quite toxic. Being a toxic person yet preferred by microsoft and others indicates he’s “willing”, meaning he will do whatever they ask of him. And that’s not in the best interest of openai.
simonw 3y ago
Skilled journalists (and this piece was written by skilled journalists) would not use the three members of the board as their only three sources. A big part of this kind of journalism is taking information from sources, considering the potential bias of each source and carefully correlating it with information from other sources before publishing it. That bit specifically says: "After they failed, they gave up and departed, according to three people familiar with the attempt to push Mr. Altman out." My guess is that at least one of those people comes from the Anthropic side, who would be someone with a clear insight into what happened.
- simonw 3y ago
  This inspired me to write an article about how to decipher clues like "three people familiar with the attempt": https://simonwillison.net/2023/Nov/22/deciphering-clues/ https://simonwillison.net/2023/Nov/22/deciphering-clues/
- JoshuaDavid 3y ago
  > Skilled journalists (and this piece was written by skilled journalists) would not use the three members of the board as their only three sources. Depends on whether their goal is to inform or to convince.

PepperdineG 3y ago

jacquesm 3y ago
That's a bit naive, to put it mildly. It presumes that nobody else would be able to replicate the effort and that the parties that are able to replicate it would also destroy theirs after proving that it could be done. Fat chance.
- barrkel 3y ago
  Indeed. The only way to retain control of the leading ship in the race is to keep it together as you steer it. If the ship disintegrates, then you're no longer in control of the leading ship, and someone else will win the race.
  - 0xDEAFBEAD 3y ago
    The charter states that OpenAI should allow another company to win the race under certain circumstances: >We are concerned about late-stage AGI development becoming a competitive race without time for adequate safety precautions. Therefore, if a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop competing with and start assisting this project. We will work out specifics in case-by-case agreements, but a typical triggering condition might be “a better-than-even chance of success in the next two years.” https://openai.com/charter https://openai.com/charter That's why Sam pressuring Helen to shut up about Anthropic is so concerning.
    - jacquesm 3y ago
      > we commit to stop competing with and start assisting this project That bit I never understood. It doesn't even have a qualification as to who is stewarding the project as if any AI should be assisted no matter who ends up owning it. The whole thing is just so utterly disconnected from the real world.

CSMastermind 3y ago

The only specific accusation made in the article is that Sam criticized Helen Toner for writing a paper: https://cset.georgetown.edu/publication/decoding-intentions/ https://cset.georgetown.edu/publication/decoding-intentions/ That says Anthropic has a better approach to AI safety than OpenAI. Sam apparently said she should have come to him directly if she had concerns about the company's approach and pointed out that as a board member her words have weight at a time when he was trying to navigate a tricky relationship with the FTC. She apparently told him to kick rocks and he started to look for ways to get her off the board. All of that ... seems completely reasonable? Like I've heard a lot of vague accusations thrown at Sam over the last few days and yet based on this account I think he reacted the exact same way any CEO would. I'm much more interested in how Helen managed to get on this board at all.

deleted 3y ago
[deleted]
jacquesm 3y ago
It's interesting but it may well be they both have a point: Helen for telling him to get lost and Sam for attempting to remove her before she would damage the company. But she could have made that point more forcefully by not comparing Anthropic to OpenAI, after all who better than her to steer OpenAI in the right direction. I noted in a comment elsewhere that all of these board members appear to have had at least one and some many more conflicts of interest. Helen probably believes that her loyalty is not to OpenAI but to something higher than that based on her remark that destroying the company would serve to fulfil its mission (which is a very strange point of view to begin with). But that doesn't automatically mean that she's able to place it in context, within OpenAI, within the USA, the Western world and the world as a whole. It's like saying the atomic bomb would have never been invented if the people at Los Alamos didn't do it. They did it in three years after it became known that it could be done in principle. Others tried and failed but without the same resources. I suspect that if the USA had not done it that eventually France, the UK and Russia would have gotten there as well and later on China. Israel would not have had the bomb without the USA (willing or unwilling) and India and Pakistan would have achieved it but much later as well. So we'd end up with the same situation that we have today modulo some timing differences and with another last chapter on WWII. Better? Maybe. But it is also possible that the Russians would have launched a first strike on the USA if they were unopposed. It almost happened as it was! The open question then is: does she really believe that no other entity has the resources to match OpenAI and does she believe that if such an entity does exist that it too will self destruct rather than to go through with the development? And does she believe that this will hold true for all time? That they and their colleagues are so unique that they hold the key to something that can otherwise not be replicated.

throw20away 3y ago

Ms. Toner’s motive is now clearer for being criticized about her research. Mr. D’Angelo had a competing commercialization product with Poe. Mr. Sutskever seems easy to manipulate emotionally and is in constant ideological battles. Mrs. McCauley AKA Joseph Gordon-Levitt’s wife, what’s her motive?

brigadier132 3y ago
Apparently she is also an "Effective Altruist" (along with Toner) so they are ideologically aligned.
- alvah 3y ago
  EA hadn’t had a good run in the corporate world, over the last year or so
  - 0xDEAFBEAD 3y ago
    First SBF gets busted for being too greedy, and now Helen Toner gets busted because she's not greedy enough. EA just can't catch a break.
    - alvah 3y ago
      People with a child’s view of the world, in charge of $billions of other people’s money, is the common theme.
      - 0xDEAFBEAD 3y ago
        There's been a ton of criticism of Helen here in this thread and elsewhere, but I haven't seen anyone clearly articulate what Helen should have done. Everyone says she did the wrong thing, but no one has said what the right thing actually was. Here's my guess: I think OpenAI should have prioritized character more in its hiring. Ending up with an organization full of employees who cared more about making money than upholding the charter was a mistake. Ultimately it's not enough for the people on the board to have character. The CEO and employees need to have character too.

DueDilligence 3y ago

[dead]

kragen 3y ago

keep in mind that the lead reporter on this, cade metz, is the one who wrote the character assassination piece on scott alexander repeatedly insinuating he was a neo-nazi (and de-anonymizing him, costing him his job due to some kind of weird psychiatrist ethical code) so while probably nothing in here is literally false, it's quite likely calculated to give false impressions; read with caution (well, it's literally false that "Greg Brockman (...) quit his role[] as (...) board chairman" but only slightly; that was the role he was fired from, as explained in the next paragraph of the article; that's not the kind of lies to watch out for)

dekhn 3y ago
wasn't that piece never published?
- comp_throw7 3y ago
  No, it's right here: https://www.nytimes.com/2021/02/13/technology/slate-star-codex-rationalists.html https://www.nytimes.com/2021/02/13/technology/slate-star-cod... (See also Scott's response: https://www.astralcodexten.com/p/statement-on-new-york-times-article https://www.astralcodexten.com/p/statement-on-new-york-times...)
  - kragen 3y ago
    (-)
    - comp_throw7 3y ago
      My bad, edited.
  - dekhn 3y ago
    It's a pretty interesting article if you know how to read around the NYTimes's rhetoric. """The issue, it was clear to me, was that I told him I could not guarantee him the anonymity he’d been writing with. In fact, his real name was easy to find because people had shared it online for years and he had used it on a piece he’d written for a scientific journal. I did a Google search for Scott Alexander and one of the first results I saw in the auto-complete list was Scott Alexander Siskind.""" Reading the response, I don't think this is condeming Metz to live in infamy.

pasltd 3y ago

Here you go: https://www.nytimes.com/2023/11/21/technology/openai-altman-board-fight.html?unlocked_article_code=1.AU0.0Zli.ur35-H19FBLR&smid=url-share https://www.nytimes.com/2023/11/21/technology/openai-altman-...

dang 3y ago
I moved your comment from https://news.ycombinator.com/item?id=38372541 https://news.ycombinator.com/item?id=38372541 to here, where it seems like it can help more readers. I hope that's ok!
- jessenaser 3y ago
  Thank you.

breadwinner 3y ago

The main takeaway from this whole saga is, be careful who you allow on your board. Nearly everyone on OpenAI board is young, inexperienced and unqualified to be on the board of one of the most important companies on the planet. Here's Ms. Toner's linkedin: https://www.linkedin.com/in/helen-toner-4162439a/ https://www.linkedin.com/in/helen-toner-4162439a/

jjulius 3y ago
>Nearly everyone on OpenAI board is young, inexperienced and unqualified to be on the board of one of the most important companies on the planet. Jumping off of that, I'm genuinely curious what, in Sam's resume, suggests he should be in charge of "one of the most important companies on the planet".
- avarun 3y ago
  The fact that he's a founder and the one that made it "one of the most important companies on the planet"?
  - jjulius 3y ago
    Not quite true - OpenAI's Wikipedia article says... >The organization was founded in December 2015 by Ilya Sutskever, Greg Brockman, Trevor Blackwell, Vicki Cheung, Andrej Karpathy, Durk Kingma, Jessica Livingston, John Schulman, Pamela Vagata, and Wojciech Zaremba, with Sam Altman and Elon Musk serving as the initial board members. Altman didn't join as CEO until 2019. Edit: All those names and we're going to say stuff like Altman was "the one" who made the company so important?
    - breadwinner 3y ago
      Not true. This page dated December 11, 2015 mentions Sam Altman and Elon Musk as co-chairs: https://openai.com/blog/introducing-openai https://openai.com/blog/introducing-openai

neonate 3y ago

https://archive.ph/eN5PY https://archive.ph/eN5PY

hintymad 3y ago

>Ms. Toner disagreed. The board’s mission is to ensure that the company creates artificial intelligence that “benefits all of humanity,” and if the company was destroyed, she said, that mission would be fulfilled. Doesn’t this sound like Toner herself is playing god? That is, only she is the gatekeeper of humanity and only she knows what is good for the entire humanity, when many of us believe that OpenAI’s tech is perfectly safe and amazingly beneficial?

hollerith 3y ago
>Doesn’t this sound like Toner herself is playing god? That is, only she is the gatekeeper of humanity and only she knows what is good for the entire humanity, when many of us believe that OpenAI’s tech is perfectly safe and amazingly beneficial? Yeah, well, hate to burst your nice warm Silicon-Valley bubble, but most Americans believe that continuing to create smarter and smarter AIs is dangerous, so who's playing god now? (No one I know thinks it is dangerous to fine-tune GPT-4 or to integrate GPT-4 deeply into the economy: the danger is the plans for the creation of much bigger models and algorithmic improvements.)
- TheOsiris 3y ago
  > but most Americans believe that continuing to create smarter and smarter AIs is dangerous, so who's playing god now? Aren't you just making this up? I haven't seen any surveys on what "most Americans believe" re AI. I know a lot of people that are concerned. But I'll bet good money that "most Americans" don't give a crap
  - comp_throw7 3y ago
    Googling "americans AI polling" produces this as the first result: https://www.pewresearch.org/short-reads/2023/11/21/what-the-data-says-about-americans-views-of-artificial-intelligence/ https://www.pewresearch.org/short-reads/2023/11/21/what-the-... I don't put much stock into polls like this, when it comes to actually deciding whether things are dangerous, but I think you should spend five seconds doing an incredibly trivial sanity-check before accusing people of making things up.

Bukhmanizer 3y ago

I think it’s getting to the point where the noise is overwhelming the signal in this story.

Philpax 3y ago
Really? I found this to be a very insightful article; it made it much clearer for me why the board acted as they did. It doesn't have all of the details, but it's much more than I had yesterday.

jessenaser 3y ago

> Mr. Altman complained that the research paper seemed to criticize OpenAI’s efforts to keep its A.I. technologies safe while praising the approach taken by Anthropic, according to an email that Mr. Altman wrote to colleagues and that was viewed by The New York Times. > OpenAI's board of directors approached rival Anthropic's CEO about replacing chief Sam Altman and potentially merging the two AI startups, according to two people briefed on the matter. (https://www.reuters.com/technology/openais-board-approached-anthropic-ceo-about-top-job-merger-sources-2023-11-21/ https://www.reuters.com/technology/openais-board-approached-...) It all makes sense.

soderfoo 3y ago
And the whole time no one thought to phone Satya Nadella regarding his $10 billion investment and to get his input.
- wnoise 3y ago
  That's literally not the job of this board.
- HDThoreaun 3y ago
  Why would the board care about that?
- function_seven 3y ago
  Good. His money came with no strings*. In fact, if they had run this past him before making their decision, that would have been a breach of their duty as outlined in their charter. They may have made mistakes with timing (making this decision too late), or with execution (not having a replacement CEO solidly lined-up and in the loop), but the core decision—to remove Sam—is definitely not something they should ask donors about. It's not Satya's business how the board makes decisions at this level. * Other than a capped profit return if there is any.
- dkjaudyeqooe 3y ago
  Who cares what he thinks, his money is gone and he has no control over it. In fact the most positive outcome is if Altman and the rest of the staff went to MS and did their thing and OpenAI started from scratch with the $13B they've come into. That would double the chances of something useful emerging from the OpenAI work so far. I personally think Altman is very much less than a genius (maybe at extracting financial advantage) so all OpenAI's eggs shouldn't be placed in that particular basket.

winenbug 3y ago

https://openai.com/our-structure https://openai.com/our-structure This whole thing was so, SO poorly executed, but the independent people on the board were gathered specifically to prioritize humanity & AI safety over OpenAI. It sounds like Sam forgot just that when he criticized Helen for her research (given how many people were posting ways to "get around" ChatGPT's guardrails, she probably had some firm grounds to stand on). Yes, Sam made LLMs mainstream and is the face of AI, but if the board believes that that course of action could destroy humanity it's literally the board's mission to stop it — whether that means destroying OpenAI or not. What this really shows us is that this "for-profit wrapper around a non-profit" shenanigans was doomed to fail in the first place. I don't think either side is purely in the wrong here, but they're two sides of an incredibly badly thought-of charter.

0xDEAFBEAD 3y ago
>they're two sides of an incredibly badly thought-of charter. It's easy to say this with the benefit of hindsight, but I haven't seen anyone in this discussion even suggest an alternative model that they claim would've been superior.
- winenbug 3y ago
  Agreed, I'm not saying I have a better alternative, just that this is something we all should now realize; given i'm sure we were all wondering for a long time what the whole governance structure of OpenAI really meant (capped for-profit with non-profit mission etc.)
lll-o-lll 3y ago
> It sounds like Sam forgot just that when he criticized Helen for her research (given how many people were posting ways to "get around" ChatGPT's guardrails, she probably had some firm grounds to stand on). Sam didn’t forget anything. He is a brilliant Machiavellian operator. Just look at the Reddit reverse takeover as an example; Machiavelli would be in awe. > What this really shows us is that this "for-profit wrapper around a non-profit" shenanigans was doomed to fail in the first place. No. It shows this structure is doomed to fail if you have a genius schemer as a CEO, playing the long game to gain unrestricted control.

1vuio0pswjnm7 3y ago

"OpenAI was started in 2015 with an ambitious plan to one day create a superintelligent automated system that can do everything a human brain can do." Sounds absurd. No one even knows everything the human brain does. It is poorly understood.

6gvONxR4sf7o 3y ago

tldr: The new part of the story this adds is that Altman’s firing was partly in response to him trying to kick out one of the other board members.

brigandish 3y ago

As an aside, I must say that I'm fascinated by this use of the word ouster. In British English I've not heard it used this way, we'd use ousting in that place and the ouster would be the one doing the ousting, i.e. the person doing the pushing. I thought I'd check with a quick search of the Guardian, and on two different days it used both in the same sense: From the 18th of November edition, technology section:[1] > The crisis at OpenAI deepened this weekend when the company said Altman’s ousting was for allegedly misleading the board. From 17th of November edition, also technology section[2]: > The announcement blindsided employees, many of whom learned of the sudden ouster from an internal announcement and the company’s public facing blog. I could only find one instance in the Telegraph prior to Altman's erm, ousting[3]: > Johnson dropped into the COP27 UN climate change conference in October, joking unusual summer heat had played a part in his ouster, and has vowed to keep championing Ukraine. They stick to ousting every other time. I wonder if it may be an artefact of newspapers using news services to get copy from, or a stylistic rule for international editions, as the Guardian does use ouster several other times but always in stories regarding US news. Also fascinating that we still don't know who that is. Neither do most of the staff, and even the board, apparently. A true mastermind! Maybe it is GPT5 after all… <cue ominous music, or perhaps, a Guns n' Roses album?> [1] https://www.theguardian.com/technology/2023/nov/18/earthquake-at-chatgpt-developer-as-senior-staff-quit-after-sacking-of-boss-sam-altman https://www.theguardian.com/technology/2023/nov/18/earthquak... [2] https://www.theguardian.com/technology/2023/nov/17/openai-ceo-sam-altman-fired https://www.theguardian.com/technology/2023/nov/17/openai-ce... [3] https://www.telegraph.co.uk/news/2022/12/27/boris-johnson-will-make-political-comeback-2023/ https://www.telegraph.co.uk/news/2022/12/27/boris-johnson-wi...

crmd 3y ago

I thought the drug dealers in my hometown were ruthless sociopaths until I had to start doing deals with change-the-world mild mannered old navy-wearing Stanford Silicon Valley dudes.

1vuio0pswjnm7 3y ago

"In the email, Mr. Altman said that he had reprimanded Ms. Toner for the paper and that it was dangerous to the company, particularly at a time, he added, when the Federal Trade Commission was investigating OpenAI over the data used to build its technology."

QuantumGood 3y ago

It sounds like the board got ever more concentrated towards less sanity over time.

gwern 3y ago

None of the comments thus far seem to clearly explain why this matters. Let me summarize the implications: Sam Altman expelling Toner with the pretext of an inoffensive page (https://cset.georgetown.edu/wp-content/uploads/CSET-Decoding-Intentions.pdf#page=30 https://cset.georgetown.edu/wp-content/uploads/CSET-Decoding...) in a paper no one read* would have given him a temporary majority with which to appoint a replacement director, and then further replacement directors. These directors would, naturally, agree with Sam Altman, and he would have a full, perpetual board majority - the board, which is the only oversight on the OA CEO. Obviously, as an extremely experienced VC and CEO, he knew all this and how many votes he (thought he) had on the board, and the board members knew this as well - which is why they had been unable to agree on replacement board members all this time. So when he 'reprimanded' her for her 'dangerous' misconduct and started talking seriously about how 'inappropriate' it was for a 'board member' to write anything which was not cheerleading, and started leading discussions about "whether Ms Toner should be removed"... * I actually read CSET papers, and I still hadn't bothered to read this one, nor would I have found anything remarkable about that page, which Altman says was so bad that she needed to be expelled immediately from the board.

abi 3y ago
I mean it's not cool for board members to publicly criticize the company.
- kragen 3y ago
  it's not cool for companies to try to shut down academic freedom of inquiry in scholarly publishing in order to improve their public image her independence from openai was the nominal reason she was asked to supervise it in the first place, wasn't it
  - foota 3y ago
    I'm not sure you should get to have your cake and eat it too.
    - 3y ago

lwneal 3y ago

The relevant passage from the paper co-written by board member Helen Toner: "OpenAI has also drawn criticism for many other safety and ethics issues related to the launches of ChatGPT and GPT-4, including regarding copyright issues, labor conditions for data annotators, and the susceptibility of their products to "jailbreaks" that allow users to bypass safety controls... A different approach to signaling in the private sector comes from Anthropic, one of OpenAI's primary competitors. Anthropic's desire to be perceived as a company that values safety shines through across its communications, beginning from its tagline: "an AI safety and research company." A careful look at the company's decision-making reveals that this commitment goes beyond words." [1] https://cset.georgetown.edu/publication/decoding-intentions/ https://cset.georgetown.edu/publication/decoding-intentions/

murakamiiq84 3y ago
I think this is heavily editoralized. if you look at the 3 pages in question that the quotes are pulled from (28-30 in doc, 29-31 in pdf), they appear to be given as examples in pretty boring academic discussions explicating the theories of costly signaling in the context of AI.It also has lines like: "The system card provides evidence of several kinds of costs that OpenAI was willing to bear in order to release GPT-4 safely.These include the time and financial cost..." "Returning to our framework of costly signals, OpenAI’s decision to create and publish the GPT4 system card could be considered an example of tying hands as well as reducible costs. By publishing such a thorough, frank assessment of its model’s shortcomings, OpenAI has to some extent tied its own hands—creating an expectation that the company will produce and publish similar risk assessments for major new releases in the future. OpenAI also paid a price ..." "While the system card itself has been well received among researchers interested in understanding GPT-4’s risk profile, it appears to have been less successful as a broader signal of OpenAI’s commitment to safety" And the conclusion: "Yet where OpenAI’s attempt at signaling may have been drowned out by other, even more conspicuous actions taken by the company, Anthropic’s signal may have simply failed to cut through the noise. By burying the explanation of Claude’s delayed release in the middle of a long, detailed document posted to the company’s website, Anthropic appears to have ensured that this signal of its intentions around AI safety has gone largely unnoticed. Taken together, these two case studies therefore provide further evidence that signaling around AI may be even more complex than signaling in previous eras."

jafitc 3y ago

Sam Altman's Actions - Sam complained that Helen Toner's research paper criticized OpenAI's safety approach and praised Anthropic's, seeing it as dangerous for OpenAI - He reprimanded her for the paper, saying as a board member she should have brought concerns to him first rather than publishing something critical right when OpenAI was in a tricky spot with the FTC - He then began looking for ways to remove her from the board in response to the paper --- Helen Toner's Perspective - She believes the board's mission is to ensure OpenAI makes AI that benefits humanity, so destroying the company would fulfill that mission - This suggests she prioritizes that mission over the company itself, seeing humanitarian concerns as more important than OpenAI's success --- Microsoft Partnership - The Microsoft partnership concentrated too much power in one company and went against the mission of OpenAI being "open" - It gave Microsoft full access to OpenAI's core technologies against the safety-focused mission --- Governance Issues - The conflict shows the adversarial tensions inherent in OpenAI's structure between nonprofit and for-profit sides - The board's mandate to act as a check and balance on OpenAI seems to be working as intended in this case --- Criticisms of Players - Altman appears reckless in his actions, while Toner seems naive about consequences of destroying OpenAI - Their behavior calls into question whether anyone should have this kind of power over the development of AI --- Future of AI Development - Attempts at alignment and safeguards by companies like OpenAI may be ineffective if other actors are developing AI without such considerations - Who controls advanced AI is more important than whether the AI is friendly. - Nationalization of AI projects may occur

jacquesm 3y ago
Nice summary.
- mpalmer 3y ago
  "Thank you! If you have any other questions, feel free to ask. I'm here to help!" - the author of the summary, probably
richk449 3y ago
Is there a name for this type of analysis?

jdprgm 3y ago

I really don't understand why the three board members have just been completely silent online since things hit the fan with no activity or privating profiles. You would think if this was some sort of ideological based coup and pre-meditated you would have a full PR plan in place and pushing your message aggressively. This whole thing has just been bizarre and it still feels like there has to be some big key piece missing that somehow nobody has revealed.

jacquesm 3y ago
In a word: liability.
superultra 3y ago
I think anyone that has worked in academia or the non-profit world is familiar with how this board appears to be operating and certainly its speed, or lack thereof. Compared to tech, it’s glacial. Altman et al has media training, the full weight of Microsoft PR/comms. What do the three board members have? Barely anyone even really knew some of their names until this weekend.

foota 3y ago

Definitely seems like Helen is emerging as the main leader of the coup, "Ms. Toner disagreed. The board’s mission is to ensure that the company creates artificial intelligence that “benefits all of humanity,” and if the company was destroyed, she said, that could be consistent with its mission. In the board’s view, OpenAI would be stronger without Mr. Altman."

comp_throw7 3y ago
If the events happened as described (https://news.ycombinator.com/item?id=38373572 https://news.ycombinator.com/item?id=38373572), calling it a "coup" sure is an interesting framing.
zucker42 3y ago
The "leader of the coup" has changed each day. First it was Ilya, then D'Angelo because of Poe, then Toner because he criticized her paper.
dkjaudyeqooe 3y ago
How is it a coup when it's the boards job to hire and fire the CEO?

jejeyyy77 3y ago

Looks like the board did the right thing after all.

webappguy 3y ago

How can anyone not think Adam D'Angelo is upset about open AI crushing his shitty Poe

fnordpiglet 3y ago

“””Ms. Toner disagreed. The board’s mission is to ensure that the company creates artificial intelligence that “benefits all of humanity,” and if the company was destroyed, she said, that could be consistent with its mission.””” This is remarkable and unhinged. I gave the board the benefit of the doubt at first, but as this unfolds, it becomes clear that the board is held hostage by ideological zealots who can’t compromise. The story also seems to paint Ilya as a manipulated figure, being bent against his beliefs by crazies like Toner exploiting his concerns about AI safety. What an absolute shame. I am entirely ambivalent about Altman overall, but he becomes more sympathetic as the days go by.

swatcoder 3y ago
> he becomes more sympathetic as the days go by PR in action! He's had an excellent PR machine running since Friday and the opposing faction seems to have exactly none (which makes sense given their relative roles and backgrounds). So reporters and Twitter get carefully crafted leaks, tips, and comments from one side and nobody on the other side has the experience, connections, and confidence to push back with a different narrative. And so the story drifts in their favor. This is why professional PR is a thing and earns people lots of money.
- orpheansodality 3y ago
  Sometimes a "PR machine" is just social capital at work. If there's anything else I've learned in this saga it's that Sam has had a consistent track record of building and maintaining highly positive relationships with anyone near his orbit, this board aside, and that it consistently pays him dividends.
leroy_masochist 3y ago
She literally wanted to destroy OpenAI in order to save it. Unbelievable. We need a total and complete shutdown of EAs serving on startup boards until we figure out what the hell is going on.
comp_throw7 3y ago
It's not unhinged, it's explicitly within the scope of OpenAI's charter. Sam Altman himself said that "Development of superhuman machine intelligence (SMI) is probably the greatest threat to the continued existence of humanity." not even a year before OpenAI was founded: https://blog.samaltman.com/machine-intelligence-part-1 https://blog.samaltman.com/machine-intelligence-part-1

deleted 3y ago

[deleted]

clnq 3y ago

A lot of people in tech say that executives are excessively diplomatic and do not speak their truth. But this is what happens when they do too much, too ardently, too often. This is why diplomacy and tact is so important in these roles. Things do not go well if everyone keeps poking each other with sticks and cannot let their own frame of reference go for the sake of the bigger picture. Ultimately, I don’t think Altman doesn’t believe ethics and safety is important. And I don’t think Toner fails to realize that OpenAI is only in a place to dictate what AI will be due to its commercial principles. And they probably both agree that there is a conflict there. But what tactful leadership would have done is found a solution behind closed doors. Yet from their communication, it doesn’t even look like they defined the problem statement — everyone offers a different idea of the problem that they had to face together. It looks more like it was more like immature people shouting past each other for a year (not saying it was that, but it looks that way). Moral of the story: tact, grace, and diplomacy are important. So is speaking one’s truth, but there is a tactful time, place, and manner. And also, no matter how brilliant someone is, if they can’t develop these traits, they end up rocking the boat a lot.

jacquesm 3y ago
Spot on.

Uptrenda 3y ago

How do I filter news of this.

jibal 3y ago

why are so many people so keen on this prepper?

oraphalous 3y ago

Fascinating! Apparently central to the complete farce going on at OpenAI - is this paper by one of the board members: https://cset.georgetown.edu/publication/decoding-intentions/ https://cset.georgetown.edu/publication/decoding-intentions/ It's about - my favourite topic: Costly Signaling! How curious - that such a topic would be the possible catalyst for one of the most extraordinary collapses of a leading tech company. Specifically - the paper is about using costly signalling as a means to align (or demonstrate algnment) various kinds of AI interested entities (governments, private corporations, etc) with the public good. The gist of costly signalling - to try and convince others you really mean what you say - you use a signal that is very expensive to you in some respect. You don't just ask a girl to marry you, you buy a big diamond ring! The idea being - cheaters are much less likely to suffer such expense. Apparently the troubles at OpenAI escalated when one of the board members - Helen Toner - published this paper. It is critical of OpenAI, and Sam Altman was pissed at the reputational damage to the company and wanted her removed. The board instead removed him. The gist of the paper's criticisms is that while OpenAI has invested in some costly signals to indicate its alignment with AI safety, overall, it judges those signals were ultimately rather weak (or cheap). Now here is what I find fascinating about all this: up until reading this paper I had found the actions of the OpenAI board completely baffling, but now suddenly their actions make a kind of insane sense. They are just taking their thinking on costly signalling to its logical conclusion. By putting the entire fate of the company and its amazing market position at risk - they are sending THE COSTLIEST SIGNAL possible, relatively speaking: willingness to suffer self-annihilation. Academics truly are wondrous people... that they can lose themselves in a system of thought so deeply, in a way regular people can't. I can't help but have a genuine, sublime appreciation for this, even while thinking they are some of the silliest people on this planet. Here's where I feel they went wrong. Costly signals by and large should be without explicit intention. If you are consciously sending various signals that are costly - you are probably a weirdo. Systems of costly signalling work because they are implicit, shared and in many respects, innate. That's why even insects can engage in costly signalling. But these folk see costly signals as an explicit activity to be engaged in as part of explicit public policy - and unsurprisingly, see it riddled with ambiguity. Of course it would be - individual agents can't just make signals up, and expect the system to understand them. Semiotics biatch.... But rather than reflect on this they double down on signalling as an explicit policy choice. How do they propose to reduce ambiguity? Even costlier signals! It's no wonder then they see it as entirely rational to accept self destruction as a possibility. That's how they escape the horrid existential dread of being doubted by the other. In biology though, no creatures survived in the long-run to reproduce where they invested in costly signals that that didn't confer at least as much, if not more benefit to them in excess of what they paid in the first place. Those that ignore this basic cost-benefit analysis in their signalling will suffer the ignomony of being perceived as ABSOLUTE NUTTERS. Which is exactly how the world is thinking about the OpenAI board. The world doesn't see a group of highly safety aligned AI leaders. The world sees a bunch of disfunctional crazy people.

alex_young 3y ago

This is the most amazing part of the article imo: Ms. Toner disagreed. The board’s mission is to ensure that the company creates artificial intelligence that “benefits all of humanity,” and if the company was destroyed, she said, that could be consistent with its mission. In the board’s view, OpenAI would be stronger without Mr. Altman. Corporate suicidal ideation.

jacquesm 3y ago
Par for the course for EA and the nine dwarfs of Eschatology.
dkjaudyeqooe 3y ago
If your mission is to not harm humanity and you think your charity is going to harm humanity then its destruction fulfils that mission.

GreedClarifies 3y ago

Sounds like Sam was attempting to fix the board membership. OpenAI had grown massively since some of the board members were installed. Some of them were simply not the caliber of people that one would have running such a prestigious institution, especially not with the weight they had due to the board being depopulated. Sam realized this and maybe was attempting to address the issue. Some of the members (ahem, Helen, Tasha and to a lesser extent Adam) liked their positions and struck first, probably convincing poor Ilya that this was about AI safety. Being lightweights they did not do any pre-work or planning, they just plowed ahead. They didn't think through that Sam and Greg have added tremendous value to the company and that the company would favor them far over the board that added zero value. They didn't think through that tech in general would see rain makers and value creators being cut loose and side with them instead of figureheads. They didn't think that partners and customers, who dealt with Sam and Greg daily would be find the move disconcerting (at a minimum). They didn't even think through who would be the next CEO. Maybe they didn't think it through since they didn't care. There was only upside for them since Sam was going to get rid of them sooner or later. They didn't see that having been on the OpenAI board was an honor and enormous career boost. Or maybe their ambition was so great that nothing mattered but controlling OpenAI. Further, they thought that if they slandered Sam that he would be cowed and that they would retain their power. I wonder how many times they had pulled this stunt in the past and it worked?

zoogeny 3y ago

> Mr. Sutskever’s frustration with Mr. Altman echoed what had happened in 2021 when another senior A.I. scientist left OpenAI to form the company Anthropic. That scientist and other researchers went to the board to try to push Mr. Altman out. So Altman faced another similar challenge to his authority and prevailed. I recall hearing that Anthropic started because the people who had left were unhappy with OpenAIs track record on AI safety. > In the email, Mr. Altman said that he had reprimanded Ms. Toner for the paper and that it was dangerous to the company, > Mr. Altman complained that the research paper seemed to criticize OpenAI’s efforts to keep its A.I. technologies safe > Senior OpenAI leaders [...] later discussed whether Ms. Toner should be removed That paints a pretty bleak picture that isn't favorable to Altman. Two times he was challenged about OpenAIs safety and both times he worked to purge those who opposed him. I can't tell if this is a contention between accelerationism and deccelerationism or if it is a contention between safety and recklessness. Is Altman ignoring the warnings his employees/peers are giving him and retaliating against them? Or is he facing insubordination? I wish OpenAI would split neatly into two. But based on the heightened emotions caused by the insane amount of PR I only see two outcomes. Altman returns and OpenAI stays unified vs. Altman stays at MS and OpenAI is obliterated. I am guessing Altman is hoping that senior management will choose a unified OpenAI at all costs, including ignoring the red flags above. He has engineered a situation where the only way OpenAI remains unified is if he returns.

jiggawatts 3y ago
> I recall hearing that Anthropic started because the people who had left were unhappy with OpenAIs track record on AI safety. Anthropic announced Claude 2.1 just recently, to which HN members notably responded by saying that their models are "overly restricted", to the point of uselessness. E.g.: https://news.ycombinator.com/item?id=38366134 https://news.ycombinator.com/item?id=38366134 and: https://news.ycombinator.com/item?id=38368072 https://news.ycombinator.com/item?id=38368072 etc.: https://news.ycombinator.com/item?id=38370590 https://news.ycombinator.com/item?id=38370590 It seems that those people left Open AI because they're insane and think that AI safety means to completely lobotomize the models to the point of uselessness. That last bit is not hyperbole: right here on HN many people were complaining that the Claude model is useless to them because it refuses to obey orders and keeps interjecting some absurd refusal such as "killing a process is too violent" instead of providing Python code for this harmless activity. It's hard to put into words exactly, but I feel like that there's a disconnect in the minds of these people and the reality on the ground, and those false axioms or assumptions when driven to their logical conclusion lead to nonsense. I suspect that Sam Altman doesn't share these delusions, and is more sane and commercially minded. Models like GPT3, 4, and the upcoming v5 have no risk of "escaping" and taking over humanity. None. Zero. Zip. Similarly, they're not much more (in practice) than a fancy search engine. Search engines let their users search for whatever they want, including hateful content, violent content, racist, sexist, or whatever. So why lobotomise the AI? That's undermining their utility, making them much less useful! An here's the thing: their potential utility is enormous. But not if they're dumbed down to the point of uselessness! Microsoft is throwing 10+ billion dollars at OpenAI and Sam Altman precisely because of this potential utility. I can see this straightforward commercial mindset of utility => $$$ being incompatible with what is rapidly turning into an anti-useful-AI mindset that is bordering on insane religious zealotry.

didibus 3y ago

> Ms. Toner disagreed. The board’s mission is to ensure that the company creates artificial intelligence that “benefits all of humanity,” and if the company was destroyed, she said, that could be consistent with its mission We're in true sci-fi land, when we're discussing it might be best to destroy SkyNet before it's too late

jacquesm 3y ago

I'm beginning to strongly associate Effective Altruism with out-of-control naivety.

coffeeshopgoth 3y ago

Having served on boards, the one thing I have not seen that would be really key (and maybe I just have not been lucky in my searches), is something that answers the question of what is the language around releasing information regarding the company to the press or in a research capacity? Typically there are rules that are set up to give the board notice of anything going out to the public, be it an article or research paper, that allows for a heads up or gives time to discuss the implications of such a thing. While whistleblowers are necessary, was there a need to be a sort of whistleblower in this case? Was there adequate board discussion around the subject and the paper before the release? If not, nobody needs a rogue board member like that and it was definitely not in the interest of the company - she is the one at fault here. If that process did happen, she definitely did the right thing and shame on the board for not getting out in front of it.