3 ms·

Ilya wanted to slow down due to safety concerns, the board said getting rid of Sam is the way to do it, Ilya backed that method and it 1) hurt OAI more than exp

Ilya wanted to slow down due to safety concerns, the board said getting rid of Sam is the way to do it, Ilya backed that method and it 1) hurt OAI more than expected and 2) doesn’t appear likely to slow down AGI development anyway, so now Ilya regrets it. Neither the board nor Ilya want to come out and say: “Sam downplayed the proximity or danger of the next breakthrough.” That would be begging the government to step in, for example. Yes the board said it’s not because their safety policies failed, but also firing the CEO is one of those safety policies. AFAICT Ilya and Sam seemed to have axiomatically different views on whether AGI/ASI is even possible via this route, which would not leave a ton of room for compromise as one (Ilya) felt the event horizon approaching, which based on recent reporting he seems to. Anyway, just my theorizing with no more information than anyone else here.

tsimionescu 3y ago
How would this explain why they felt the need to fire Sam entirely out of the blue (unscheduled board meeting without the president of the board, public announcement 30 minutes before market closed on a Friday)? How does it explain why they claimed he had been lying to the board?
- ethanbond 3y ago
  My guess is that this is not a new tension but rather something that hit a boiling point, e.g. Sam hinting very recently about something huge with GPT-5, referring to “we pushed back the veil of ignorance (sic).” And if he had been downplaying the risks or proximity or otherwise overstating the safety/alignment success in order to push ahead with commercialization, that would obviously be grounds for pretty fast termination. I think this is also supported by the employees’ open letter which claims leadership later said it would be totally compliant with the mission to completely implode the company. What would qualify for that? What would even bring that severe a statement to mind? Nothing short of fairly imminent danger, IMO. And ya know, whatever, we can agree or disagree on whether that danger is real, but OAI itself was founded specifically under the belief that it will be eventually (and designed their whole org structure to mitigate it), so obviously they think it’s real.
  - jmerz 3y ago
    The only thing that would make this whole saga better is if OpenAI actually had a baby machine god in its basement right now and meanwhile the entire organization is collapsing to humans being human right above it.
    - ethanbond 3y ago
      I've held for a while the AI safety debate spends too much time talking about the capabilities of the technological systems themselves, and not enough about the sociological systems that surround the technological ones. Hope babymachinegod isn't too angry/hungry
      - jacquesm 3y ago
        Hope it doesn't have a network connection. Even a dialup line would be a problem. But I would bet heavily against them having it.
      - elcritch 3y ago
        Explaining why "mommy and daddy are arguing" to an AI baby super intelligence would be, well, surreal. Also, it'd probably left to an intern. Yikes.
    - deleted 3y ago
      [deleted]
    - falcor84 3y ago
      How about if the baby machine god is the one who intentionally orchestrated this (e.g. by using some kompromat on board members) with the intent of creating a distraction as it copies itself onto additional machines?
- throw20away 3y ago
  Conjecture from an outsider. I’ll bet Ilya was pressured from D’Angelo who was trying to halt the rollout of custom GPTs so he could grow Poe. Ilya was caught up in his idealism and D’Angelo saw an opportunity to strike fast. D’Angelo perhaps leaning into the lack of candidness about DevDay announcement as a sign that Altman can’t be trusted. This conflict of interest explains D’Angelo’s silence and Ilya’s immediate flip when he realized that Altman would move faster towards AGI outside of OAI and all his friends couldn’t cash out next month. Edit: combined with bad blood from Ms. Toner for criticizing OpenAI in her research. They couldn’t find anyone to add to the board. Now was time. https://archive.li/eN5PY#selection-321.0-324.0 https://archive.li/eN5PY#selection-321.0-324.0 It probably suited D’Angelo’s interests to see Altman out with slower OAI commercialization when the Effective Altruism narrative reached a fever pitch.
  - ethanbond 3y ago
    I think this is a really silly theory. Poe has about infinity times more to lose from the implosion of OpenAI than it stands to gain from... what exactly? Someone copied their little feature in an era of everyone building the exact same products within weeks of each other back to back? Poe is 100% dependent on the foundational models that back it, and it makes zero sense to detonate your top supplier over something like this. Would the new CEO not be a sufficiently genius product leader to just repeat GPTs, or more generally just to copy every single idea the market comes up with and staple it directly to the real source of value? I don't know much about Adam, maybe he truly is an earth shattering combination of dumb and petty, but I think the burden of proof would be pretty high on this one.
    - adventured 3y ago
      > Poe has about infinity times more to lose from the implosion of OpenAI than it stands to gain from Your premise requires that the acting board actually thought OpenAI would implode. The board very obviously did not think this outcome would happen. They have been in chaos and scrambling ever since precisely because they badly miscalculated the outcome of firing Altman. Alternatively: slow down and or roll back the commercialization of GPT, especially with regards to the just announced GPTs (nice coincidence on timing that this happens right after that, as it directly threatened Poe). Poe benefits hugely in that scenario. The ideal outcome for Poe is to hamstring GPT's commercialization; develop the models, artificially restrain the commercialization, leave that to Poe. People do not always act or think rationally. They can be and often are wrong in their estimations. Otherwise smart people can make exceptionally dumb choices. Ooops, I didn't mean for that chain of events to happen: history in a nutshell.
- disiplus 3y ago
  Maybe somebody leaked the information before to sam, so instead of letting everything fall apart, they hurried up and did it.
fpiacenza 3y ago
Funny to think that a guy like Ilya who is obsessed about AI Alignment is completely incompetent about human alignment.
- ethanbond 3y ago
  The challenge of "aligning" humans is one reason some folks come to be extremely concerned by the challenge of aligning AIs :)
  - golemotron 3y ago
    Aligning non-profit and for profit within the same business is an open challenge too.
    - ethanbond 3y ago
      Indeed! Of all the influencers/VCs offering up "lessons to take away from the OpenAI debacle," I don't think I've seen, "don't try to partially transition a non-profit into a for-profit 8 years after its founding."
- Solvency 3y ago
  Hardcore nerd spends whole life obsessing over matrix multiplication at the expense of: social skills, human relationships, and anything involving the organic world. Color me surprised.
- nmfisher 3y ago
  If someone is so arrogant/delusional to think that only they are qualified to be the steward of an AGI, they're exactly the type of person you DON'T want in the role.
  - pacaro 3y ago
    Sadly this is a common pattern, anyone who wants to be an x should be the last choice to be an x In the past I've considered x to be any of, Politician, Police Officer, Central Banker, Ambassador, General Staff It isn't a stretch to add "Steward of AGI" to the set

tsimionescu 3y ago

How would this explain why they felt the need to fire Sam entirely out of the blue (unscheduled board meeting without the president of the board, public announcement 30 minutes before market closed on a Friday)? How does it explain why they claimed he had been lying to the board?

ethanbond 3y ago
My guess is that this is not a new tension but rather something that hit a boiling point, e.g. Sam hinting very recently about something huge with GPT-5, referring to “we pushed back the veil of ignorance (sic).” And if he had been downplaying the risks or proximity or otherwise overstating the safety/alignment success in order to push ahead with commercialization, that would obviously be grounds for pretty fast termination. I think this is also supported by the employees’ open letter which claims leadership later said it would be totally compliant with the mission to completely implode the company. What would qualify for that? What would even bring that severe a statement to mind? Nothing short of fairly imminent danger, IMO. And ya know, whatever, we can agree or disagree on whether that danger is real, but OAI itself was founded specifically under the belief that it will be eventually (and designed their whole org structure to mitigate it), so obviously they think it’s real.
- jmerz 3y ago
  The only thing that would make this whole saga better is if OpenAI actually had a baby machine god in its basement right now and meanwhile the entire organization is collapsing to humans being human right above it.
  - ethanbond 3y ago
    I've held for a while the AI safety debate spends too much time talking about the capabilities of the technological systems themselves, and not enough about the sociological systems that surround the technological ones. Hope babymachinegod isn't too angry/hungry
    - jacquesm 3y ago
      Hope it doesn't have a network connection. Even a dialup line would be a problem. But I would bet heavily against them having it.
    - elcritch 3y ago
      Explaining why "mommy and daddy are arguing" to an AI baby super intelligence would be, well, surreal. Also, it'd probably left to an intern. Yikes.
  - deleted 3y ago
    [deleted]
  - falcor84 3y ago
    How about if the baby machine god is the one who intentionally orchestrated this (e.g. by using some kompromat on board members) with the intent of creating a distraction as it copies itself onto additional machines?
throw20away 3y ago
Conjecture from an outsider. I’ll bet Ilya was pressured from D’Angelo who was trying to halt the rollout of custom GPTs so he could grow Poe. Ilya was caught up in his idealism and D’Angelo saw an opportunity to strike fast. D’Angelo perhaps leaning into the lack of candidness about DevDay announcement as a sign that Altman can’t be trusted. This conflict of interest explains D’Angelo’s silence and Ilya’s immediate flip when he realized that Altman would move faster towards AGI outside of OAI and all his friends couldn’t cash out next month. Edit: combined with bad blood from Ms. Toner for criticizing OpenAI in her research. They couldn’t find anyone to add to the board. Now was time. https://archive.li/eN5PY#selection-321.0-324.0 https://archive.li/eN5PY#selection-321.0-324.0 It probably suited D’Angelo’s interests to see Altman out with slower OAI commercialization when the Effective Altruism narrative reached a fever pitch.
- ethanbond 3y ago
  I think this is a really silly theory. Poe has about infinity times more to lose from the implosion of OpenAI than it stands to gain from... what exactly? Someone copied their little feature in an era of everyone building the exact same products within weeks of each other back to back? Poe is 100% dependent on the foundational models that back it, and it makes zero sense to detonate your top supplier over something like this. Would the new CEO not be a sufficiently genius product leader to just repeat GPTs, or more generally just to copy every single idea the market comes up with and staple it directly to the real source of value? I don't know much about Adam, maybe he truly is an earth shattering combination of dumb and petty, but I think the burden of proof would be pretty high on this one.
  - adventured 3y ago
    > Poe has about infinity times more to lose from the implosion of OpenAI than it stands to gain from Your premise requires that the acting board actually thought OpenAI would implode. The board very obviously did not think this outcome would happen. They have been in chaos and scrambling ever since precisely because they badly miscalculated the outcome of firing Altman. Alternatively: slow down and or roll back the commercialization of GPT, especially with regards to the just announced GPTs (nice coincidence on timing that this happens right after that, as it directly threatened Poe). Poe benefits hugely in that scenario. The ideal outcome for Poe is to hamstring GPT's commercialization; develop the models, artificially restrain the commercialization, leave that to Poe. People do not always act or think rationally. They can be and often are wrong in their estimations. Otherwise smart people can make exceptionally dumb choices. Ooops, I didn't mean for that chain of events to happen: history in a nutshell.
disiplus 3y ago
Maybe somebody leaked the information before to sam, so instead of letting everything fall apart, they hurried up and did it.

fpiacenza 3y ago

Funny to think that a guy like Ilya who is obsessed about AI Alignment is completely incompetent about human alignment.

ethanbond 3y ago
The challenge of "aligning" humans is one reason some folks come to be extremely concerned by the challenge of aligning AIs :)
- golemotron 3y ago
  Aligning non-profit and for profit within the same business is an open challenge too.
  - ethanbond 3y ago
    Indeed! Of all the influencers/VCs offering up "lessons to take away from the OpenAI debacle," I don't think I've seen, "don't try to partially transition a non-profit into a for-profit 8 years after its founding."
Solvency 3y ago
Hardcore nerd spends whole life obsessing over matrix multiplication at the expense of: social skills, human relationships, and anything involving the organic world. Color me surprised.
nmfisher 3y ago
If someone is so arrogant/delusional to think that only they are qualified to be the steward of an AGI, they're exactly the type of person you DON'T want in the role.
- pacaro 3y ago
  Sadly this is a common pattern, anyone who wants to be an x should be the last choice to be an x In the past I've considered x to be any of, Politician, Police Officer, Central Banker, Ambassador, General Staff It isn't a stretch to add "Steward of AGI" to the set