16 ms·

The Inside Story of Microsoft's Partnership with OpenAI

240 points by jyunwai 3y ago

gwern 3y ago
Further confirmation of what I said 9 days ago (https://news.ycombinator.com/item?id=38373572 https://news.ycombinator.com/item?id=38373572) based on the NYT (https://www.nytimes.com/2023/11/21/technology/openai-altman-board-fight.html https://www.nytimes.com/2023/11/21/technology/openai-altman-...) followed by WSJ (https://www.wsj.com/tech/ai/altman-firing-openai-520a3a8c https://www.wsj.com/tech/ai/altman-firing-openai-520a3a8c): the firing was over Sam Altman's attempt to takeover the board by firing one of the holdouts (Helen Toner), which would secure him a majority indefinitely (especially after packing the board further by filling vacancies), thereby neutering all remaining governance controls over Altman. In particular, in this piece, Altman acknowledges on the record that he was trying to fire Toner and went about it in a 'ham-fisted way'. Excerpts: "...Some members of the OpenAI board had found Altman an unnervingly slippery operator. For example, earlier this fall he’d confronted one member, Helen Toner, a director at the Center for Security and Emerging Technology, at Georgetown University, for co-writing a paper that seemingly criticized OpenAI for “stoking the flames of AI hype.” Toner had defended herself (though she later apologized to the board for not anticipating how the paper might be perceived). Altman began approaching other board members, individually, about replacing her. When these members compared notes about the conversations, some felt that Altman had misrepresented them as supporting Toner’s removal. “He’d play them off against each other by lying about what other people thought”, the person familiar with the board’s discussions told me. “Things like that had been happening for years.” (A person familiar with Altman’s perspective said that he acknowledges having been “ham-fisted in the way he tried to get a board member removed”, but that he hadn’t attempted to manipulate the board.) ...His tactical skills were so feared that, when 4 members of the board---Toner, D’Angelo, Sutskever, and Tasha McCauley---began discussing his removal, they were determined to guarantee that he would be caught by surprise. “It was clear that, as soon as Sam knew, he’d do anything he could to undermine the board”, the person familiar with those discussions said...Two people familiar with the board’s thinking say that the members felt bound to silence by confidentiality constraints...But whenever anyone asked for examples of Altman not being “consistently candid in his communications”, as the board had initially complained, its members kept mum, refusing even to cite Altman’s campaign against Toner. ...The dismissed board members, meanwhile, insist that their actions were wise. “There will be a full and independent investigation, and rather than putting a bunch of Sam’s cronies on the board we ended up with new people who can stand up to him”, the person familiar with the board’s discussions told me. “Sam is very powerful, he’s persuasive, he’s good at getting his way, and now he’s on notice that people are watching.” Toner told me, “The board’s focus throughout was to fulfill our obligation to OpenAI’s mission.” (Altman has told others that he welcomes the investigation---in part to help him understand why this drama occurred, and what he could have done differently to prevent it.) Some A.I. watchdogs aren’t particularly comfortable with the outcome. Margaret Mitchell, the chief ethics scientist at Hugging Face, an open-source A.I. platform, told me, “The board was literally doing its job when it fired Sam. His return will have a chilling effect. We’re going to see a lot less of people speaking out within their companies, because they’ll think they’ll get fired---and the people at the top will be even more unaccountable.” Altman, for his part, is ready to discuss other things. “I think we just move on to good governance and good board members and we’ll do this independent review, which I’m super excited about”, he told me. “I just want everybody to move on here and be happy. And we’ll get back to work on the mission”."
- omeze 3y ago
  Yea, this turned out to be the simplest explanation… try to undermine board, get fired, win anyway (lol) Lesson here is… the next time a startup CEO gets fired they should try the social media spam gambit…? Depends on how much $ is on the line but worked at least once!
  - ianmcgowan 3y ago
    I think the lesson is that after firing your CEO and 80% of the organization threatens to quit, then maybe don't do that. Though if MSFT wasn't there for a soft landing ($), not sure the number would be as high as that.
    - logicchains 3y ago
      The lesson is if you're going to fire your CEO for alleged unethical behaviour, make sure you're willing to actually argue why you fired him, to convince people it was a reasonable decision.
      - starttoaster 3y ago
        Agreed. The board's public note about all this basically read as, "Sam is not able to do his job." They made this public note during a time of generally high praise for OpenAI's success, so nobody believed it. I hate to side in either direction here, but the board really showed incompetence with public relations here. Having said that, I'm sure there's some poor PR person at OpenAI sweating over exactly this, and to them I say, it's not your fault, every single person in the board should have been involved in that public notice, and everyone apparently agreed this was a sufficient explanation for a bunch of people with no internal context.
        TillE 3y ago
        I really think the board and the whole organization would've come out of this just fine (minus a handful of resignations) if they'd just outlined their reasons to employees. Of course you can't keep a CEO who you can't trust. Really bizarre that they didn't do that.
SpaceManNabs 3y ago
[flagged]
- breadwinner 3y ago
  Name a successful CEO who most people say is "nice". Steve Jobs? No. Larry Ellison? No. Bill Gates? No. Nice guys don't become CEOs. Most CEOs are slimy.
  - SpaceManNabs 3y ago
    So? He is still a slimy person. Btw, Lisa Su and Rose Marcario, but both are women so maybe you are right in that nice guys can't be ceos. ;)
  - kjreact 3y ago
    I haven’t heard too many negative comments towards Warren Buffett. He also has a pretty down-to-earth persona. But in general I tend to agree that nice guys don’t succeed in business.
  - tennisflyi 3y ago
    Cook? Nadella?
  - pfannkuchen 3y ago
    I haven’t really heard of Jobs or Gates being manipulative within the company, just ruthless. Curious if anyone has comparable manipulation stories?
- deleted 3y ago
  [deleted]
xianshou 3y ago
Now that this and other articles have established that the board's actions were based not on a single event but rather on general, long-term, underspecified concerns, no one captures the essence of what happened more perfectly than Matt Levine: "Well, sure, but [the board not trusting Altman] is a fight about AI safety. It’s just a metaphorical fight about AI safety. I am sorry, I have made this joke before, but events keep sharpening it. The OpenAI board looked at Sam Altman and thought 'this guy is smarter than us, he can outmaneuver us in a pinch, and it makes us nervous. He’s done nothing wrong so far, but we can’t be sure what he’ll do next as his capabilities expand. We do not fully trust him, we cannot fully control him, and we do not have a model of how his mind works that we fully understand. Therefore we have to shut him down before he grows too powerful.' I’m sorry! That is exactly the AI misalignment worry! If you spend your time managing AIs that are growing exponentially smarter, you might worry about losing control of them, and if you spend your time managing Sam Altman you might worry about losing control of him, and if you spend your time managing both of them you might get confused about which is which. Maybe Sam Altman will turn the old board members into paper clips." (Link: https://www.bloomberg.com/opinion/articles/2023-11-29/the-robots-will-insider-trade https://www.bloomberg.com/opinion/articles/2023-11-29/the-ro...)
- TeMPOraL 3y ago
  That's pure gold, and I'm even more impressed at Levine finding time and will to understand enough about AI x-risk arguments to not just speak the language, but make a joke in it.
  - foobarian 3y ago
    Makes me wonder, has anyone actually physically seen Sam A. lately? :-)
- SkyMarshal 3y ago
  That's hilarious, though it also demonstrates the AI Safety problem in another way - If true, Sam made a stupid mistake in assuming he could lie/fudge/exaggerate each board member's position to the others and that they wouldn't eventually talk amongst themselves and compare notes. That suggests he's either really not that smart, or he's overconfident and sloppy. An AGI, or especially an ASI, would presumably avoid such easily-detectable mistakes. Things just start happening and seemingly disconnected events steer reality in a particular direction, and there is no way to concretely link them or discern their true provenance. This is already exactly the problem with the most sophisticated disinformation campaigns run by the FSB/IRA/etc. It appears random, grassroots, and uncoordinated, and is difficult or impossible to discern its true provenance. But it all operates across a shared a set of objectives - increase social divisions, decrease trust in democracy and democratic institutions, and/or increase general irrationality and emotional response among the public. If humans can already accomplish that, imagine what an ASI could do.
acl777 3y ago
https://archive.ph/s2CdK https://archive.ph/s2CdK
denton-scratch 3y ago
The general consensus (in the places I've read) is that the old board were incompetent blunderers. TFA seems to agree, without saying so outright. I don't see how you can build an $80 billion operation, with a board full of blunderers. I get that it was those blunderers AND sam, until about a year ago; and I get that a year earlier still, they had several experienced businessmen on the (then larger) board. None of the "blunderers" has spoken yet, AFAIK; I doubt they've taken the cloth and become Trappists, so I assume they're keeping their powder dry, and waiting to see how the situation shakes out.
- rvba 3y ago
  CEO and employees are the ones that build the company. In many places the role of the board is just to fire / hire the CEO. In some orgs they have more roles: getting financing, customers, audit, agreeing on certain things (e.g. investments, or contracts). Sometimes the board also has more prerogatives. But in general the role is a more ceremonial role that once per year chooses the audit firm (one of the big 4 companies.. so not very hard), maybe makes some obligatory resolutions, like confirming the audit result. Their biggest power is to remove the CEO and choose the CEO's compensation. Getting a seat on a board of a public company is like free money. Most boards dont meet often. Typical boards in companies represent the shareholders (aka remove the CEO if they want to do something crazy), while in non profits they are usually there to find financing or networking. So yes, you can get very far with a bad board, because most boards are quite inactive. The members read some documents, maybe sometimes request some data - and that's all.
  - denton-scratch 3y ago
    Thanks. Yes, I guess I knew that about boards, subliminally. > So yes, you can get very far with a bad board That I didn't know. I'm inclined to believe it, and it seems at first glance to clear-up in my mind quite a few things about government in general that keep surprising me.
  - yannyu 3y ago
    This seems like an overly simplistic take and reads like someone who hasn't worked closely with board members or been on the board of a company with a lot of money behind it.
upwardbound 3y ago
Some members of the OpenAI board had found Altman an unnervingly slippery operator. For example, earlier this fall he’d confronted one member, Helen Toner, a director at the Center for Security and Emerging Technology, at Georgetown University, for co-writing a paper that seemingly criticized OpenAI for “stoking the flames of AI hype.” Toner had defended herself (though she later apologized to the board for not anticipating how the paper might be perceived). Altman began approaching other board members, individually, about replacing her. When these members compared notes about the conversations, some felt that Altman had misrepresented them as supporting Toner’s removal. “He’d play them off against each other by lying about what other people thought,” the person familiar with the board’s discussions told me. “Things like that had been happening for years.”
- kranke155 3y ago
  Sounds like a manipulative individual.
  - m_fayer 3y ago
    Sounds like a CEO of a hyped tech company.
    - upwardbound 3y ago
      Yes. And that type of person is not usually the type of trustworthy or selfless person whom we as a species critically need, in a role like OpenAI leader that has final say over the ethical value system of the artificial superintelligence which OpenAI is building towards. This concern (that a profit-maximizing CEO like Sam is not the type of person who should decide the ethical fate of humanity) was and is the whole point of having a non-profit as the owner of the for-profit tech company, so that ethically-driven people who value human well-being over growth can be in charge of the non-profit board and exercise final say over the AI's ethical constraints. The fact that Microsoft now has a board observer seat seems highly corrupting, and contrary to the board's fiduciary purpose. This is discussed here: https://news.ycombinator.com/item?id=38471990 https://news.ycombinator.com/item?id=38471990 Basically, Microsoft now has an enormous, and indecent, insider information advantage (allowing them to conduct software development industry market front-running, which would be illegal but it will be impossible to prove that the source of their ideas is the "forbidden fruit" of what they hear in board meetings). Details may be found in that link.
stainablesteel 3y ago
it honestly seems like the board isn't a good fit, and they tend to have crazy mental breakdowns that caused the entire company to lose stability i agree sam is too smart for them, they need to replace the board
- gumballindie 3y ago
  There are more mental health issues around that company than breakdown. The constant bombardment of schizofrenic doomerism is a good indicator of what the issues may be. Altman on the other hand is just a bro wanting to make money by inflating the capabilities of a chat bot through fomo fud and an army of incel spammers.
  - rat9988 3y ago
    I'm not why we should see him as a "bro".
norir 3y ago
Reading this article, I kept think about James Micken's 2018 usenix security talk https://www.youtube.com/watch?v=ajGX7odA87k https://www.youtube.com/watch?v=ajGX7odA87k. He warns quite effectively against widely deploying technology that we don't understand. Kevin Scott, on the other hand, seems to have exactly the opposite take. I suppose there is a reason though why one of them is a successful businessman and the other is an academic.
- dekhn 3y ago
  I don't think ML is substantially different from other technologies that we have deployed that we didn't completely understand: early atomic weapons, early molecular biology, cars, and more. In each case, by deploying a bit with caution, we learned important lessons that allowed us to continue. Rather than sticking our head in the ground and remaining ignorant. In each case there were people calling for the deployments to stop. Of all those, I think only atomic weapons ever represented an existential risk to humanity- and we figured that out within a decade or to and set up political structures to prevent it (although, ideally, we'd have worldwide disarmment). Unless somebody can make some sort of extremely convincing argument that by deploying this tech we are taking an existential risk, rather than purely banal ones, it doesn't seem reasonble to prevent deployment.
  - ethanbond 3y ago
    What? All of those inventions you listed were mechanistically understood quite well. Atomic weapons wouldn’t have been possible without a completely sound understanding of their mechanics. A better example is perhaps pharmaceuticals, many of which we do not actually understand the mechanisms of action. But note those go through extremely rigorous testing for exactly that reason.
    - dekhn 3y ago
      The inventions were partly understood mechanistically- just like ML. What we didn't understand was the consequences of deployment (such as massive radiation leaks due to testing, unpredictable yields, the risks of global thermonuclear war, and the ongoing cost of maintaining a fleet of nuclear weapons). Similar issues with molecular biology- we knew how to clone a gene from one organism to another, but that doesn't mean we really knew all the things going on (side effects) or what the large-scale implications are (hence the Asilomar agreement). Even cars- while engines were understood mechanistically, it took quite some time for people to appreciate why automobile safety glass was necessary. See the experiences learned during testing nuclear weapons- we tested them because we didn't understand them mechanistcally, at least not fully enough to predict many effects.
mrcwinn 3y ago
I don’t have any reason to trust Sam Altman much. To me, he seems polished enough to actually be slimy. He’s well liked by others but wealthy tech is a particularly insular group. But my god, in what world should the future of “AI safety” be in the hands of a small board of directors that works in a completely opaque manner and with uncertain motivations or capabilities? Surely humanity’s fate is not meant to be guarded by the Quora dude.
- Kinrany 3y ago
  Four is better than one through.
  - tsunamifury 3y ago
    I dunno... seems like its an AvP situation -- whoever wins, we lose.
    - deleted 3y ago
      [deleted]
sarchertech 3y ago
The article implies that the Mira Murati and the rest of the C suite were the ones responsible for getting people to sign the letter threatening to quit unless the board resigned. And there seems to be some direct involvement from Microsoft (even before they announced they would hire employees who quit). This sounds a lot different from the narrative that it was a grass roots show of support for Sam Altman. If the CEO, the rest of the execs, and the investors in the for-profit were pushing employees to sign, doing so wasn’t a sign of loyalty to Altman at all.
- mcast 3y ago
  It was strange seeing how many people with genuine intentions of advancing AI for good were blindly threatening to join MSFT, a company with much less genuine reputation. I think OpenAI's board is even more controversial than ever now.
  - epgui 3y ago
    To be fair, Microsoft does seem to have repaired a lot of its bad reputation in the last decade or so. Has it not? At least that was my impression. The word Microsoft used to be borderline toxic.
    - vintagedave 3y ago
      I found this article, 'VSCode is Designed to Fracture'[0], insightful for its look at the current Microsoft. Its thesis is that VSCode is meant to leverage open source and popularity to increase closed source vendor lock-in to Microsoft. It cites recent examples of VSCode extensions being closed source, so that while VSCode seems open, the critical portions are not. It's worth reading the full article. 0: https://ghuntley.com/fracture/ https://ghuntley.com/fracture/
      - epgui 3y ago
        Thanks for sharing!
- 3y ago
6gvONxR4sf7o 3y ago
> Altman began approaching other board members, individually, about replacing [Toner]. When these members compared notes about the conversations, some felt that Altman had misrepresented them as supporting Toner’s removal. “He’d play them off against each other by lying about what other people thought,” the person familiar with the board’s discussions told me. “Things like that had been happening for years.” > ... [Altman's] tactical skills were so feared that, when four members of the board—Toner, D’Angelo, Sutskever, and Tasha McCauley—began discussing his removal, they were determined to guarantee that he would be caught by surprise. “It was clear that, as soon as Sam knew, he’d do anything he could to undermine the board,” the person familiar with those discussions said. > The person familiar with [Altman's] perspective said that he and the board had engaged in “very normal and healthy boardroom debate,” but that some board members were unversed in business norms and daunted by their responsibilities. > [Re: the board's silence after firing Altman] Two people familiar with the board’s thinking say that the members felt bound to silence by confidentiality constraints. This seems to be the most concrete information about what he did since it happened. Sounds like he lied for years and justified it as "business norms."
- 3cats-in-a-coat 3y ago
  I haven't heard anything yet that presents Helen Toner as someone who was useful on the board. She'd write dramatic papers about the dangers of AI which put her own company (I mean whose board she was on) in a bad light, and was unreceptive to feedback from Altman about the optics of the entire situation. Maybe she thought she's doing the right thing, I'm sure all those people think they're doing their earnest version of "the right thing". Yet the company she praised, Anthropic, is clearly dysfunctional, with users complaining all over the Internet that Claude refuses to perform basic tasks out of misguided "safety concerns" ("I can't help you kill Linux processes, as this is unethical"), making their subscriptions useless. That's the Helen Toner model. See, people who deal with security often like to put the spotlight on themselves by proclaiming grave danger is imminent and then position themselves as the experts who can save us from this danger. This is such a common pattern and sometimes it's not even consciously done, but extremely toxic regardless. Everyone wants to be important and regarded, so they lean on behaviors that achieve that outcome. Sam Altman is your typical startup entrepreneur sleezeball. And yes, here he was trying to manufacture consent within a board that was largely complacent through "divide and conquer". The details of his behavior can be nauseating, but still... I can't deny on the total he did make countless correct leadership decisions at OpenAI, and getting Helen Toner off the board... for all we know so far... feels like one of those correct leadership decisions. So I can't feel too bad about it. The only thing I feel bad about it is Ilya Sutskever. I hope he and Altman can find a productive and dignified way to continue their partnership.
renewiltord 3y ago
Ultimately, now we can get back to using a useful tool instead of having people assign numerical values to unlikely events and use metaphorical references to the fact that they can't Align the Saltman.

240 points by jyunwai 3y ago

Further confirmation of what I said 9 days ago (https://news.ycombinator.com/item?id=38373572 https://news.ycombinator.com/item?id=38373572) based on the NYT (https://www.nytimes.com/2023/11/21/technology/openai-altman-board-fight.html https://www.nytimes.com/2023/11/21/technology/openai-altman-...) followed by WSJ (https://www.wsj.com/tech/ai/altman-firing-openai-520a3a8c https://www.wsj.com/tech/ai/altman-firing-openai-520a3a8c): the firing was over Sam Altman's attempt to takeover the board by firing one of the holdouts (Helen Toner), which would secure him a majority indefinitely (especially after packing the board further by filling vacancies), thereby neutering all remaining governance controls over Altman. In particular, in this piece, Altman acknowledges on the record that he was trying to fire Toner and went about it in a 'ham-fisted way'. Excerpts: "...Some members of the OpenAI board had found Altman an unnervingly slippery operator. For example, earlier this fall he’d confronted one member, Helen Toner, a director at the Center for Security and Emerging Technology, at Georgetown University, for co-writing a paper that seemingly criticized OpenAI for “stoking the flames of AI hype.” Toner had defended herself (though she later apologized to the board for not anticipating how the paper might be perceived). Altman began approaching other board members, individually, about replacing her. When these members compared notes about the conversations, some felt that Altman had misrepresented them as supporting Toner’s removal. “He’d play them off against each other by lying about what other people thought”, the person familiar with the board’s discussions told me. “Things like that had been happening for years.” (A person familiar with Altman’s perspective said that he acknowledges having been “ham-fisted in the way he tried to get a board member removed”, but that he hadn’t attempted to manipulate the board.) ...His tactical skills were so feared that, when 4 members of the board---Toner, D’Angelo, Sutskever, and Tasha McCauley---began discussing his removal, they were determined to guarantee that he would be caught by surprise. “It was clear that, as soon as Sam knew, he’d do anything he could to undermine the board”, the person familiar with those discussions said...Two people familiar with the board’s thinking say that the members felt bound to silence by confidentiality constraints...But whenever anyone asked for examples of Altman not being “consistently candid in his communications”, as the board had initially complained, its members kept mum, refusing even to cite Altman’s campaign against Toner. ...The dismissed board members, meanwhile, insist that their actions were wise. “There will be a full and independent investigation, and rather than putting a bunch of Sam’s cronies on the board we ended up with new people who can stand up to him”, the person familiar with the board’s discussions told me. “Sam is very powerful, he’s persuasive, he’s good at getting his way, and now he’s on notice that people are watching.” Toner told me, “The board’s focus throughout was to fulfill our obligation to OpenAI’s mission.” (Altman has told others that he welcomes the investigation---in part to help him understand why this drama occurred, and what he could have done differently to prevent it.) Some A.I. watchdogs aren’t particularly comfortable with the outcome. Margaret Mitchell, the chief ethics scientist at Hugging Face, an open-source A.I. platform, told me, “The board was literally doing its job when it fired Sam. His return will have a chilling effect. We’re going to see a lot less of people speaking out within their companies, because they’ll think they’ll get fired---and the people at the top will be even more unaccountable.” Altman, for his part, is ready to discuss other things. “I think we just move on to good governance and good board members and we’ll do this independent review, which I’m super excited about”, he told me. “I just want everybody to move on here and be happy. And we’ll get back to work on the mission”."

omeze 3y ago
Yea, this turned out to be the simplest explanation… try to undermine board, get fired, win anyway (lol) Lesson here is… the next time a startup CEO gets fired they should try the social media spam gambit…? Depends on how much $ is on the line but worked at least once!
- ianmcgowan 3y ago
  I think the lesson is that after firing your CEO and 80% of the organization threatens to quit, then maybe don't do that. Though if MSFT wasn't there for a soft landing ($), not sure the number would be as high as that.
  - logicchains 3y ago
    The lesson is if you're going to fire your CEO for alleged unethical behaviour, make sure you're willing to actually argue why you fired him, to convince people it was a reasonable decision.
    - starttoaster 3y ago
      Agreed. The board's public note about all this basically read as, "Sam is not able to do his job." They made this public note during a time of generally high praise for OpenAI's success, so nobody believed it. I hate to side in either direction here, but the board really showed incompetence with public relations here. Having said that, I'm sure there's some poor PR person at OpenAI sweating over exactly this, and to them I say, it's not your fault, every single person in the board should have been involved in that public notice, and everyone apparently agreed this was a sufficient explanation for a bunch of people with no internal context.
      - TillE 3y ago
        I really think the board and the whole organization would've come out of this just fine (minus a handful of resignations) if they'd just outlined their reasons to employees. Of course you can't keep a CEO who you can't trust. Really bizarre that they didn't do that.

SpaceManNabs 3y ago

[flagged]

breadwinner 3y ago
Name a successful CEO who most people say is "nice". Steve Jobs? No. Larry Ellison? No. Bill Gates? No. Nice guys don't become CEOs. Most CEOs are slimy.
- SpaceManNabs 3y ago
  So? He is still a slimy person. Btw, Lisa Su and Rose Marcario, but both are women so maybe you are right in that nice guys can't be ceos. ;)
- kjreact 3y ago
  I haven’t heard too many negative comments towards Warren Buffett. He also has a pretty down-to-earth persona. But in general I tend to agree that nice guys don’t succeed in business.
- tennisflyi 3y ago
  Cook? Nadella?
- pfannkuchen 3y ago
  I haven’t really heard of Jobs or Gates being manipulative within the company, just ruthless. Curious if anyone has comparable manipulation stories?
deleted 3y ago
[deleted]

xianshou 3y ago

Now that this and other articles have established that the board's actions were based not on a single event but rather on general, long-term, underspecified concerns, no one captures the essence of what happened more perfectly than Matt Levine: "Well, sure, but [the board not trusting Altman] is a fight about AI safety. It’s just a metaphorical fight about AI safety. I am sorry, I have made this joke before, but events keep sharpening it. The OpenAI board looked at Sam Altman and thought 'this guy is smarter than us, he can outmaneuver us in a pinch, and it makes us nervous. He’s done nothing wrong so far, but we can’t be sure what he’ll do next as his capabilities expand. We do not fully trust him, we cannot fully control him, and we do not have a model of how his mind works that we fully understand. Therefore we have to shut him down before he grows too powerful.' I’m sorry! That is exactly the AI misalignment worry! If you spend your time managing AIs that are growing exponentially smarter, you might worry about losing control of them, and if you spend your time managing Sam Altman you might worry about losing control of him, and if you spend your time managing both of them you might get confused about which is which. Maybe Sam Altman will turn the old board members into paper clips." (Link: https://www.bloomberg.com/opinion/articles/2023-11-29/the-robots-will-insider-trade https://www.bloomberg.com/opinion/articles/2023-11-29/the-ro...)

TeMPOraL 3y ago
That's pure gold, and I'm even more impressed at Levine finding time and will to understand enough about AI x-risk arguments to not just speak the language, but make a joke in it.
- foobarian 3y ago
  Makes me wonder, has anyone actually physically seen Sam A. lately? :-)
SkyMarshal 3y ago
That's hilarious, though it also demonstrates the AI Safety problem in another way - If true, Sam made a stupid mistake in assuming he could lie/fudge/exaggerate each board member's position to the others and that they wouldn't eventually talk amongst themselves and compare notes. That suggests he's either really not that smart, or he's overconfident and sloppy. An AGI, or especially an ASI, would presumably avoid such easily-detectable mistakes. Things just start happening and seemingly disconnected events steer reality in a particular direction, and there is no way to concretely link them or discern their true provenance. This is already exactly the problem with the most sophisticated disinformation campaigns run by the FSB/IRA/etc. It appears random, grassroots, and uncoordinated, and is difficult or impossible to discern its true provenance. But it all operates across a shared a set of objectives - increase social divisions, decrease trust in democracy and democratic institutions, and/or increase general irrationality and emotional response among the public. If humans can already accomplish that, imagine what an ASI could do.

acl777 3y ago

https://archive.ph/s2CdK https://archive.ph/s2CdK

denton-scratch 3y ago

The general consensus (in the places I've read) is that the old board were incompetent blunderers. TFA seems to agree, without saying so outright. I don't see how you can build an $80 billion operation, with a board full of blunderers. I get that it was those blunderers AND sam, until about a year ago; and I get that a year earlier still, they had several experienced businessmen on the (then larger) board. None of the "blunderers" has spoken yet, AFAIK; I doubt they've taken the cloth and become Trappists, so I assume they're keeping their powder dry, and waiting to see how the situation shakes out.

rvba 3y ago
CEO and employees are the ones that build the company. In many places the role of the board is just to fire / hire the CEO. In some orgs they have more roles: getting financing, customers, audit, agreeing on certain things (e.g. investments, or contracts). Sometimes the board also has more prerogatives. But in general the role is a more ceremonial role that once per year chooses the audit firm (one of the big 4 companies.. so not very hard), maybe makes some obligatory resolutions, like confirming the audit result. Their biggest power is to remove the CEO and choose the CEO's compensation. Getting a seat on a board of a public company is like free money. Most boards dont meet often. Typical boards in companies represent the shareholders (aka remove the CEO if they want to do something crazy), while in non profits they are usually there to find financing or networking. So yes, you can get very far with a bad board, because most boards are quite inactive. The members read some documents, maybe sometimes request some data - and that's all.
- denton-scratch 3y ago
  Thanks. Yes, I guess I knew that about boards, subliminally. > So yes, you can get very far with a bad board That I didn't know. I'm inclined to believe it, and it seems at first glance to clear-up in my mind quite a few things about government in general that keep surprising me.
- yannyu 3y ago
  This seems like an overly simplistic take and reads like someone who hasn't worked closely with board members or been on the board of a company with a lot of money behind it.

upwardbound 3y ago

Some members of the OpenAI board had found Altman an unnervingly slippery operator. For example, earlier this fall he’d confronted one member, Helen Toner, a director at the Center for Security and Emerging Technology, at Georgetown University, for co-writing a paper that seemingly criticized OpenAI for “stoking the flames of AI hype.” Toner had defended herself (though she later apologized to the board for not anticipating how the paper might be perceived). Altman began approaching other board members, individually, about replacing her. When these members compared notes about the conversations, some felt that Altman had misrepresented them as supporting Toner’s removal. “He’d play them off against each other by lying about what other people thought,” the person familiar with the board’s discussions told me. “Things like that had been happening for years.”

kranke155 3y ago
Sounds like a manipulative individual.
- m_fayer 3y ago
  Sounds like a CEO of a hyped tech company.
  - upwardbound 3y ago
    Yes. And that type of person is not usually the type of trustworthy or selfless person whom we as a species critically need, in a role like OpenAI leader that has final say over the ethical value system of the artificial superintelligence which OpenAI is building towards. This concern (that a profit-maximizing CEO like Sam is not the type of person who should decide the ethical fate of humanity) was and is the whole point of having a non-profit as the owner of the for-profit tech company, so that ethically-driven people who value human well-being over growth can be in charge of the non-profit board and exercise final say over the AI's ethical constraints. The fact that Microsoft now has a board observer seat seems highly corrupting, and contrary to the board's fiduciary purpose. This is discussed here: https://news.ycombinator.com/item?id=38471990 https://news.ycombinator.com/item?id=38471990 Basically, Microsoft now has an enormous, and indecent, insider information advantage (allowing them to conduct software development industry market front-running, which would be illegal but it will be impossible to prove that the source of their ideas is the "forbidden fruit" of what they hear in board meetings). Details may be found in that link.

stainablesteel 3y ago

it honestly seems like the board isn't a good fit, and they tend to have crazy mental breakdowns that caused the entire company to lose stability i agree sam is too smart for them, they need to replace the board

gumballindie 3y ago
There are more mental health issues around that company than breakdown. The constant bombardment of schizofrenic doomerism is a good indicator of what the issues may be. Altman on the other hand is just a bro wanting to make money by inflating the capabilities of a chat bot through fomo fud and an army of incel spammers.
- rat9988 3y ago
  I'm not why we should see him as a "bro".

norir 3y ago

Reading this article, I kept think about James Micken's 2018 usenix security talk https://www.youtube.com/watch?v=ajGX7odA87k https://www.youtube.com/watch?v=ajGX7odA87k. He warns quite effectively against widely deploying technology that we don't understand. Kevin Scott, on the other hand, seems to have exactly the opposite take. I suppose there is a reason though why one of them is a successful businessman and the other is an academic.

dekhn 3y ago
I don't think ML is substantially different from other technologies that we have deployed that we didn't completely understand: early atomic weapons, early molecular biology, cars, and more. In each case, by deploying a bit with caution, we learned important lessons that allowed us to continue. Rather than sticking our head in the ground and remaining ignorant. In each case there were people calling for the deployments to stop. Of all those, I think only atomic weapons ever represented an existential risk to humanity- and we figured that out within a decade or to and set up political structures to prevent it (although, ideally, we'd have worldwide disarmment). Unless somebody can make some sort of extremely convincing argument that by deploying this tech we are taking an existential risk, rather than purely banal ones, it doesn't seem reasonble to prevent deployment.
- ethanbond 3y ago
  What? All of those inventions you listed were mechanistically understood quite well. Atomic weapons wouldn’t have been possible without a completely sound understanding of their mechanics. A better example is perhaps pharmaceuticals, many of which we do not actually understand the mechanisms of action. But note those go through extremely rigorous testing for exactly that reason.
  - dekhn 3y ago
    The inventions were partly understood mechanistically- just like ML. What we didn't understand was the consequences of deployment (such as massive radiation leaks due to testing, unpredictable yields, the risks of global thermonuclear war, and the ongoing cost of maintaining a fleet of nuclear weapons). Similar issues with molecular biology- we knew how to clone a gene from one organism to another, but that doesn't mean we really knew all the things going on (side effects) or what the large-scale implications are (hence the Asilomar agreement). Even cars- while engines were understood mechanistically, it took quite some time for people to appreciate why automobile safety glass was necessary. See the experiences learned during testing nuclear weapons- we tested them because we didn't understand them mechanistcally, at least not fully enough to predict many effects.

mrcwinn 3y ago

I don’t have any reason to trust Sam Altman much. To me, he seems polished enough to actually be slimy. He’s well liked by others but wealthy tech is a particularly insular group. But my god, in what world should the future of “AI safety” be in the hands of a small board of directors that works in a completely opaque manner and with uncertain motivations or capabilities? Surely humanity’s fate is not meant to be guarded by the Quora dude.

Kinrany 3y ago
Four is better than one through.
- tsunamifury 3y ago
  I dunno... seems like its an AvP situation -- whoever wins, we lose.
  - deleted 3y ago
    [deleted]

sarchertech 3y ago

The article implies that the Mira Murati and the rest of the C suite were the ones responsible for getting people to sign the letter threatening to quit unless the board resigned. And there seems to be some direct involvement from Microsoft (even before they announced they would hire employees who quit). This sounds a lot different from the narrative that it was a grass roots show of support for Sam Altman. If the CEO, the rest of the execs, and the investors in the for-profit were pushing employees to sign, doing so wasn’t a sign of loyalty to Altman at all.

mcast 3y ago
It was strange seeing how many people with genuine intentions of advancing AI for good were blindly threatening to join MSFT, a company with much less genuine reputation. I think OpenAI's board is even more controversial than ever now.
- epgui 3y ago
  To be fair, Microsoft does seem to have repaired a lot of its bad reputation in the last decade or so. Has it not? At least that was my impression. The word Microsoft used to be borderline toxic.
  - vintagedave 3y ago
    I found this article, 'VSCode is Designed to Fracture'[0], insightful for its look at the current Microsoft. Its thesis is that VSCode is meant to leverage open source and popularity to increase closed source vendor lock-in to Microsoft. It cites recent examples of VSCode extensions being closed source, so that while VSCode seems open, the critical portions are not. It's worth reading the full article. 0: https://ghuntley.com/fracture/ https://ghuntley.com/fracture/
    - epgui 3y ago
      Thanks for sharing!
3y ago

6gvONxR4sf7o 3y ago

> Altman began approaching other board members, individually, about replacing [Toner]. When these members compared notes about the conversations, some felt that Altman had misrepresented them as supporting Toner’s removal. “He’d play them off against each other by lying about what other people thought,” the person familiar with the board’s discussions told me. “Things like that had been happening for years.” > ... [Altman's] tactical skills were so feared that, when four members of the board—Toner, D’Angelo, Sutskever, and Tasha McCauley—began discussing his removal, they were determined to guarantee that he would be caught by surprise. “It was clear that, as soon as Sam knew, he’d do anything he could to undermine the board,” the person familiar with those discussions said. > The person familiar with [Altman's] perspective said that he and the board had engaged in “very normal and healthy boardroom debate,” but that some board members were unversed in business norms and daunted by their responsibilities. > [Re: the board's silence after firing Altman] Two people familiar with the board’s thinking say that the members felt bound to silence by confidentiality constraints. This seems to be the most concrete information about what he did since it happened. Sounds like he lied for years and justified it as "business norms."

3cats-in-a-coat 3y ago
I haven't heard anything yet that presents Helen Toner as someone who was useful on the board. She'd write dramatic papers about the dangers of AI which put her own company (I mean whose board she was on) in a bad light, and was unreceptive to feedback from Altman about the optics of the entire situation. Maybe she thought she's doing the right thing, I'm sure all those people think they're doing their earnest version of "the right thing". Yet the company she praised, Anthropic, is clearly dysfunctional, with users complaining all over the Internet that Claude refuses to perform basic tasks out of misguided "safety concerns" ("I can't help you kill Linux processes, as this is unethical"), making their subscriptions useless. That's the Helen Toner model. See, people who deal with security often like to put the spotlight on themselves by proclaiming grave danger is imminent and then position themselves as the experts who can save us from this danger. This is such a common pattern and sometimes it's not even consciously done, but extremely toxic regardless. Everyone wants to be important and regarded, so they lean on behaviors that achieve that outcome. Sam Altman is your typical startup entrepreneur sleezeball. And yes, here he was trying to manufacture consent within a board that was largely complacent through "divide and conquer". The details of his behavior can be nauseating, but still... I can't deny on the total he did make countless correct leadership decisions at OpenAI, and getting Helen Toner off the board... for all we know so far... feels like one of those correct leadership decisions. So I can't feel too bad about it. The only thing I feel bad about it is Ilya Sutskever. I hope he and Altman can find a productive and dignified way to continue their partnership.

renewiltord 3y ago

Ultimately, now we can get back to using a useful tool instead of having people assign numerical values to unlikely events and use metaphorical references to the fact that they can't Align the Saltman.