8 ms·

There can exist an inherent delusion within elements of a company, that if left unchallenged, can persist. An agreement for instance, can seem airtight because

by boh 3y ago

There can exist an inherent delusion within elements of a company, that if left unchallenged, can persist. An agreement for instance, can seem airtight because it's never challenged, but falls apart in court. The OpenAI fallacy was that non-profit principals were guiding the success of the firm, and when the board decided to test that theory, it broke the whole delusion. Had it not fully challenged Altman, the board could've kept the delusion intact long enough to potentially pressure Altman to limit his side-projects or be less profit minded, since Altman would have an interest to keep the delusion intact as well. Now the cat is out of the bag, and people no longer believe that a non-profit who can act at will is a trusted vehicle for the future.

jacquesm 3y ago
Yes, indeed and that's the real loss here: any chance of governing this properly got blown up by incompetence.
- hef19898 3y ago
  Of we ignore the risks and threats of AI for a second, this whole story is actually incredibly funny. So much childish stupidity on display on all sides is just hilarious. Makes what the world would look like if, say, the Manhattan Project would have been managed the same way. Well, a younger me working at OpenAI would resign latest after my collegues stage a coup againstvthe board out of, in my view, a personality cult. Propably would have resigned after the third CEO was announced. Older me would wait for a new gig to be ligned up to resign, with beginning after CEO number 2 the latest. The cyckes get faster so. It took FTX a little bit longer from hottest start up to enter the trajectory of crash and burn, OpenAI did faster. I just hope this helps ro cool down the ML sold as AI hype a notch.
  - jacquesm 3y ago
    The scary thing is that these incompetents are supposedly the ones to look out for the interests of humanity. It would be funny if it weren't so tragic. Not that I had any illusions about this being a fig leaf in the first place.
    - stingraycharles 3y ago
      Perhaps they were put in that position precisely because of their incompetence, not despite of it.
      - jacquesm 3y ago
        I wouldn't rule that out. Normally you'd expect a bit more wisdom rather than only smarts on a board. And some of those really shouldn't be there at all (conflicts of interest, lack of experience).
  - jibe 3y ago
    Of we ignore the risks and threats of AI for a second [..] just hope this helps ro cool down the ML sold as AI hype If it is just ML sold as AI hype, are you really worried about the threat of AI?
    - hef19898 3y ago
      It can be both, a hype and a danger. I don't worry much about AGI by now (I stopped insulting Alexa so, just to be sure). The danger of generative AI is that it disrupts all kinds of things: arts, writers, journalism, propaganda... That threat already exists, the tech being no longer being hyped might allow us to properly adress that problem.
      - jacquesm 3y ago
        > I stopped insulting Alexa so, just to be sure Priceless. The modern version of Pascal's wager.
  - anonymouskimmer 3y ago
    > Makes what the world would look like if, say, the Manhattan Project would have been managed the same way. It was not possible for a war-time government crash project to have been managed the same way. During WW2 the existential fear was an embodied threat currently happening. No one was even thinking about a potential for profits or even any additional products aside from an atomic bomb. And if anyone had ideas on how to pursue that bomb that seemed like a decent idea, they would have been funded to pursue them. And this is not even mentioning the fact that security was tight. I'm sure there were scientists who disagreed with how the Manhattan project was being managed. I'm also sure they kept working on it despite those disagreements.
    - Apocryphon 3y ago
      That's what happened to the German program though https://en.wikipedia.org/wiki/German_nuclear_weapons_program https://en.wikipedia.org/wiki/German_nuclear_weapons_program
      - anonymouskimmer 3y ago
        Well, yes, but they were the existential threat. Hey, maybe this means the AGIs will fight amongst themselves and thus give us the time to outwit them. :D
        jowea 3y ago
        Actual scifi plot.
    - hooande 3y ago
      For real. It's like, did you see Oppenheimer? There's a reason they put the military in charge of that.
- postmodest 3y ago
  Ignoring "Don't be Ted Faro" to pursue a profit motive is indeed a form of incompetence.
- zer00eyz 3y ago
  > any chance of governing this properly got blown up by incompetence No one knows why the board did this. No one is talking about that part. Yet every one is on twitter talking shit about the situation. I have worked with a lot of PhD's and some of them can be, "disconnected" from anything that isn't their research. This looks a lot like that, disconnected from what average people would do, almost childlike (not ish, like). Maybe this isn't the group of people who should be responsible for "alignment".
  - kmlevitt 3y ago
    The Fact still nobody knows why they did it is part of the problem now though. They have already clarified it was not for any financial reason, security reason, or privacy/safety reason, so that rules out all the important ones that spring to anyone’s minds. And they refuse to elaborate why in writing despite being asked to repeatedly. Any reason good enough to fire him is good enough to share with the interim CEO and the rest of the company, if not the entire world. If they can’t even do that much, you can’t blame employees for losing faith in their leadership. They couldn’t even tell SAM ALTMAN why, and he was the one getting fired!
    - denton-scratch 3y ago
      > The Fact still nobody knows why they did it is part of the problem now though. The fact that Altman and Brockman were hired so quickly by Microsoft gives a clue: it takes time to hire someone. For one thing, they need time to decide. These guys were hired by Microsoft between close-of-business on Friday and start-of-business on Monday. My supposition is that this hiring was in the pipeline a few weeks ago. The board of OpenAI found out on Thursday, and went ballistic, understandably (lack of candidness). My guess is there's more shenanigans to uncover - I suspect that Altman gave Microsoft an offer they couldn't refuse, and that OpenAI was already screwed by Thursday. So realizing that OpenAI was done for, they figured "we might as well blow it all up".
      - mediaman 3y ago
        The problem with this analysis is the premise: that it "takes time to hire someone." This is not an interview process for hiring a junior dev at FAANG. If you're Sam & Greg, and Satya gives you an offer to run your own operation with essentially unlimited funding and the ability to bring over your team, then you can decide immediately. There is no real lower bound of how fast it could happen. Why would they have been able to decide so quickly? Probably because they prioritize the ability to bring over the entire team as fast as possible, and even though they could raise a lot of money in a new company, that still takes time, and they view it as critically important to hire over the new team as fast as possible (within days) that they accept whatever downsides there may be to being a subsidiary of Microsoft. This is what happens when principles see opportunity and are unencumbered by bureaucratic checks. They can move very fast.
- slavik81 3y ago
  > that's the real loss here: any chance of governing this properly got blown up by incompetence If this incident is representative, I'm not sure there was ever a possibility of good governance.
- bart_spoon 3y ago
  Was it due to incompetence though? The way it has played out has made me feel it was always doomed. It is apparent that those concerned with AI safety were gravely concerned with the direction the company was taking, and were losing power rapidly. This move by the board may have simply done in one weekend what was going to happen anyways over the coming months/years anyways.
bartread 3y ago
> pressure Altman to limit his side-projects People keep talking about this. That was never going to happen. Look at Sam Altman's career: he's all about startups and building companies. Moreover, I can't imagine he would have agreed to sign any kind of contract with OpenAI that required exclusivity. Know who you're hiring; know why you're hiring them. His "side-projects" could have been hugely beneficial to them over the long term.
- itsoktocry 3y ago
  >His "side-projects" could have been hugely beneficial to them over the long term. How can you make a claim like this when, right or wrong, Sam's independence is literally, currently, tanking the company? How could allowing Sam to do what he wants benefit OpenAI, the non-profit entity?
  - brookst 3y ago
    > How could allowing Sam to do what he wants benefit OpenAI, the non-profit entity? Let's take personalities out of it and see if it makes more sense: How could a new supply of highly optimized, lower-cost AI hardware benefit OpenAI?
  - bartread 3y ago
    > Sam's independence is literally, currently, tanking the company? Honestly, I think they did that to themselves.
    - hef19898 3y ago
      And of course Sam is totally not involved in any of this, right?
      - bartread 3y ago
        In trashing the company's value? No, I'm not entirely sure it's fair to blame that one on him. I don't know the guy or have an opinion on him but, based on what I've seen since Friday, I don't think he's done that much to contribute to this particular mess. The company was literally on cloud nine this time last week and, if Friday hadn't happened, it still would be.
bnralt 3y ago
> Now the cat is out of the bag, and people no longer believe that a non-profit who can act at will is a trusted vehicle for the future. And maybe it’s not. The big mistake people make is hearing non-profit and think it means there’s a greater amount of morality. It’s the same mistake as assuming everyone who is religious is therefore more moral (worth pointing out that religions are nonprofits as well). Most hospitals are nonprofits, yet they still make substantial profits and overcharge customers. People are still people, and still have motives; they don't suddenly become more moral when they join a non-prof board. In many ways, removing a motive that has the most direct connection to quantifiable results (profit) can actually make things worse. Anyone who has seen how nonprofits work know how dysfunctional they can be.
- maksimur 3y ago
  > Most hospitals are nonprofits, yet they still make substantial profits and overcharge customers. Are you talking about American hospitals?
  - deaddodo 3y ago
    There are private hospitals all over the world. I would daresay, they're more common than public ones, from a global perspective. In addition, public hospitals still charge for their services, it's just who pays the bill that changes, in some nations (the government as the insuring body vs a private insuring body or the individual).
    - swagempire 3y ago
      Its about incentives though.
    - sangnoir 3y ago
      > There are private hospitals all over the world. I would daresay, they're more common than public ones, from a global perspective. Outside of the US, private hospitals tend to be overtly for-profit. Price-gauging "non-profit" hospitals are mostly an American phenomenon.
davesque 3y ago
Calling it a delusion seems too provocative. Another way to say it is that principles take agreement and trust to follow. The board seems to have been so enamored with its principles that it completely lost sight of the trust required to uphold them.
hooande 3y ago
This is one of the most insightful comments I've seen on this whole situation.

by boh 3y ago

jacquesm 3y ago

Yes, indeed and that's the real loss here: any chance of governing this properly got blown up by incompetence.

hef19898 3y ago
Of we ignore the risks and threats of AI for a second, this whole story is actually incredibly funny. So much childish stupidity on display on all sides is just hilarious. Makes what the world would look like if, say, the Manhattan Project would have been managed the same way. Well, a younger me working at OpenAI would resign latest after my collegues stage a coup againstvthe board out of, in my view, a personality cult. Propably would have resigned after the third CEO was announced. Older me would wait for a new gig to be ligned up to resign, with beginning after CEO number 2 the latest. The cyckes get faster so. It took FTX a little bit longer from hottest start up to enter the trajectory of crash and burn, OpenAI did faster. I just hope this helps ro cool down the ML sold as AI hype a notch.
- jacquesm 3y ago
  The scary thing is that these incompetents are supposedly the ones to look out for the interests of humanity. It would be funny if it weren't so tragic. Not that I had any illusions about this being a fig leaf in the first place.
  - stingraycharles 3y ago
    Perhaps they were put in that position precisely because of their incompetence, not despite of it.
    - jacquesm 3y ago
      I wouldn't rule that out. Normally you'd expect a bit more wisdom rather than only smarts on a board. And some of those really shouldn't be there at all (conflicts of interest, lack of experience).
- jibe 3y ago
  Of we ignore the risks and threats of AI for a second [..] just hope this helps ro cool down the ML sold as AI hype If it is just ML sold as AI hype, are you really worried about the threat of AI?
  - hef19898 3y ago
    It can be both, a hype and a danger. I don't worry much about AGI by now (I stopped insulting Alexa so, just to be sure). The danger of generative AI is that it disrupts all kinds of things: arts, writers, journalism, propaganda... That threat already exists, the tech being no longer being hyped might allow us to properly adress that problem.
    - jacquesm 3y ago
      > I stopped insulting Alexa so, just to be sure Priceless. The modern version of Pascal's wager.
- anonymouskimmer 3y ago
  > Makes what the world would look like if, say, the Manhattan Project would have been managed the same way. It was not possible for a war-time government crash project to have been managed the same way. During WW2 the existential fear was an embodied threat currently happening. No one was even thinking about a potential for profits or even any additional products aside from an atomic bomb. And if anyone had ideas on how to pursue that bomb that seemed like a decent idea, they would have been funded to pursue them. And this is not even mentioning the fact that security was tight. I'm sure there were scientists who disagreed with how the Manhattan project was being managed. I'm also sure they kept working on it despite those disagreements.
  - Apocryphon 3y ago
    That's what happened to the German program though https://en.wikipedia.org/wiki/German_nuclear_weapons_program https://en.wikipedia.org/wiki/German_nuclear_weapons_program
    - anonymouskimmer 3y ago
      Well, yes, but they were the existential threat. Hey, maybe this means the AGIs will fight amongst themselves and thus give us the time to outwit them. :D
      - jowea 3y ago
        Actual scifi plot.
  - hooande 3y ago
    For real. It's like, did you see Oppenheimer? There's a reason they put the military in charge of that.
postmodest 3y ago
Ignoring "Don't be Ted Faro" to pursue a profit motive is indeed a form of incompetence.
zer00eyz 3y ago
> any chance of governing this properly got blown up by incompetence No one knows why the board did this. No one is talking about that part. Yet every one is on twitter talking shit about the situation. I have worked with a lot of PhD's and some of them can be, "disconnected" from anything that isn't their research. This looks a lot like that, disconnected from what average people would do, almost childlike (not ish, like). Maybe this isn't the group of people who should be responsible for "alignment".
- kmlevitt 3y ago
  The Fact still nobody knows why they did it is part of the problem now though. They have already clarified it was not for any financial reason, security reason, or privacy/safety reason, so that rules out all the important ones that spring to anyone’s minds. And they refuse to elaborate why in writing despite being asked to repeatedly. Any reason good enough to fire him is good enough to share with the interim CEO and the rest of the company, if not the entire world. If they can’t even do that much, you can’t blame employees for losing faith in their leadership. They couldn’t even tell SAM ALTMAN why, and he was the one getting fired!
  - denton-scratch 3y ago
    > The Fact still nobody knows why they did it is part of the problem now though. The fact that Altman and Brockman were hired so quickly by Microsoft gives a clue: it takes time to hire someone. For one thing, they need time to decide. These guys were hired by Microsoft between close-of-business on Friday and start-of-business on Monday. My supposition is that this hiring was in the pipeline a few weeks ago. The board of OpenAI found out on Thursday, and went ballistic, understandably (lack of candidness). My guess is there's more shenanigans to uncover - I suspect that Altman gave Microsoft an offer they couldn't refuse, and that OpenAI was already screwed by Thursday. So realizing that OpenAI was done for, they figured "we might as well blow it all up".
    - mediaman 3y ago
      The problem with this analysis is the premise: that it "takes time to hire someone." This is not an interview process for hiring a junior dev at FAANG. If you're Sam & Greg, and Satya gives you an offer to run your own operation with essentially unlimited funding and the ability to bring over your team, then you can decide immediately. There is no real lower bound of how fast it could happen. Why would they have been able to decide so quickly? Probably because they prioritize the ability to bring over the entire team as fast as possible, and even though they could raise a lot of money in a new company, that still takes time, and they view it as critically important to hire over the new team as fast as possible (within days) that they accept whatever downsides there may be to being a subsidiary of Microsoft. This is what happens when principles see opportunity and are unencumbered by bureaucratic checks. They can move very fast.
slavik81 3y ago
> that's the real loss here: any chance of governing this properly got blown up by incompetence If this incident is representative, I'm not sure there was ever a possibility of good governance.
bart_spoon 3y ago
Was it due to incompetence though? The way it has played out has made me feel it was always doomed. It is apparent that those concerned with AI safety were gravely concerned with the direction the company was taking, and were losing power rapidly. This move by the board may have simply done in one weekend what was going to happen anyways over the coming months/years anyways.

bartread 3y ago

> pressure Altman to limit his side-projects People keep talking about this. That was never going to happen. Look at Sam Altman's career: he's all about startups and building companies. Moreover, I can't imagine he would have agreed to sign any kind of contract with OpenAI that required exclusivity. Know who you're hiring; know why you're hiring them. His "side-projects" could have been hugely beneficial to them over the long term.

itsoktocry 3y ago
>His "side-projects" could have been hugely beneficial to them over the long term. How can you make a claim like this when, right or wrong, Sam's independence is literally, currently, tanking the company? How could allowing Sam to do what he wants benefit OpenAI, the non-profit entity?
- brookst 3y ago
  > How could allowing Sam to do what he wants benefit OpenAI, the non-profit entity? Let's take personalities out of it and see if it makes more sense: How could a new supply of highly optimized, lower-cost AI hardware benefit OpenAI?
- bartread 3y ago
  > Sam's independence is literally, currently, tanking the company? Honestly, I think they did that to themselves.
  - hef19898 3y ago
    And of course Sam is totally not involved in any of this, right?
    - bartread 3y ago
      In trashing the company's value? No, I'm not entirely sure it's fair to blame that one on him. I don't know the guy or have an opinion on him but, based on what I've seen since Friday, I don't think he's done that much to contribute to this particular mess. The company was literally on cloud nine this time last week and, if Friday hadn't happened, it still would be.

bnralt 3y ago

> Now the cat is out of the bag, and people no longer believe that a non-profit who can act at will is a trusted vehicle for the future. And maybe it’s not. The big mistake people make is hearing non-profit and think it means there’s a greater amount of morality. It’s the same mistake as assuming everyone who is religious is therefore more moral (worth pointing out that religions are nonprofits as well). Most hospitals are nonprofits, yet they still make substantial profits and overcharge customers. People are still people, and still have motives; they don't suddenly become more moral when they join a non-prof board. In many ways, removing a motive that has the most direct connection to quantifiable results (profit) can actually make things worse. Anyone who has seen how nonprofits work know how dysfunctional they can be.

maksimur 3y ago
> Most hospitals are nonprofits, yet they still make substantial profits and overcharge customers. Are you talking about American hospitals?
- deaddodo 3y ago
  There are private hospitals all over the world. I would daresay, they're more common than public ones, from a global perspective. In addition, public hospitals still charge for their services, it's just who pays the bill that changes, in some nations (the government as the insuring body vs a private insuring body or the individual).
  - swagempire 3y ago
    Its about incentives though.
  - sangnoir 3y ago
    > There are private hospitals all over the world. I would daresay, they're more common than public ones, from a global perspective. Outside of the US, private hospitals tend to be overtly for-profit. Price-gauging "non-profit" hospitals are mostly an American phenomenon.

davesque 3y ago

Calling it a delusion seems too provocative. Another way to say it is that principles take agreement and trust to follow. The board seems to have been so enamored with its principles that it completely lost sight of the trust required to uphold them.

hooande 3y ago

This is one of the most insightful comments I've seen on this whole situation.