3 ms·

OpenAI's oversight structure is plainly stupid by any measure. Their CEO and core "partners" liquidated the board and reformed it within a few days of his firin

by woopsn 3y ago

OpenAI's oversight structure is plainly stupid by any measure. Their CEO and core "partners" liquidated the board and reformed it within a few days of his firing. The buck stops with Sam. The only thing Helen Toner screwed up was being an associate professor with net worth < $500,000,000.

0xDEAFBEAD 3y ago
I've made this point before, but the ratio of "people criticizing OpenAI's legal structure" to "people proposing a better legal structure" in these discussions is remarkably high. Not one critic I've seen has stated what the best legal structure for OpenAI should actually have been, in a world where OpenAI really is stewarding a technology that could be an existential threat to humans. I really wish they would do so, because then we'd have a shot at improving OpenAI's governance. But so far it's been crickets. BTW, I think one of the big lessons of this episode is that legal structure matters much less than you'd think. On paper, OpenAI is supposed to follow its charter https://openai.com/charter https://openai.com/charter and the board had the power to fire Sam. In practice, Sam was kinda sorta violating the charter, but he's a charismatic leader who was about to bring in a big payday for the employees (https://www.cnbc.com/2023/11/30/openai-tender-offer-on-track-despite-leadership-fracas-sources.html https://www.cnbc.com/2023/11/30/openai-tender-offer-on-track...), and that turned out to matter more than the org's legal structure.
- woopsn 3y ago
  In that world there is no alternative, we should shut them down and break their stuff.
  - 0xDEAFBEAD 3y ago
    Public opinion polls show significant support for regulating AI companies, e.g. here's one survey: https://www.linkedin.com/pulse/poll-shows-british-public-supports-moratorium-smarter-than-human-qvbpc https://www.linkedin.com/pulse/poll-shows-british-public-sup... Here's a group pushing for regulation and looking for volunteers: https://pauseai.info/ https://pauseai.info/ I would hesitate to physically break stuff -- sounds potentially unethical and could harm momentum for regulation
    - woopsn 3y ago
      I'm not advocating anything be broken. But if they're really doing what they say, and what I assumed you referred to by existential risk -- developing technologies "akin to the hydrogen bomb", that stand to "capture the light cone of all future value in the universe", etc. -- then of course that has to happen.
      - 0xDEAFBEAD 3y ago
        I encourage you to volunteer.
- theGnuMe 3y ago
  Capitalism ftw.
  - 0xDEAFBEAD 3y ago
    I generally agree: https://news.ycombinator.com/item?id=38471297 https://news.ycombinator.com/item?id=38471297 However I make an exception for existentially dangerous technology
- lacker 3y ago
  Okay, I'll take a shot. I think the legal structure is very important, but the important part is not the vague wording of the charter. "Violating the charter" is always fuzzy and debatable. "Getting the votes of most board members" is not. So the important part is who controls the board. Instead of a structure where the board members get to choose other board members, I would suggest a structure where different outside groups get fixed numbers of board members. So, for example, it could have been (remembering that the initial board was just Sam Altman and Elon Musk): * One board member chosen by Sam Altman * One board member chosen by Elon Musk * One board member chosen by OpenPhil * One board member chosen by Microsoft * One board member chosen by some VC investor The precise composition is something they would argue about during funding, maybe some of these groups would get more or less seats. The root "governance" problem is that the board churned a lot because so many people had conflicts, or other reasons that I don't know as an outsider. The remaining board was unrepresentative of stakeholders (ie, no Elon Musk, nobody independently trusted by Microsoft) and probably not as experienced as you'd like. The problem with this sort of structure is that it gives everyone a lot of incentives to play politics with the composition of the board itself. Which happened a lot here - Reid Hoffman exiting and not really getting replaced by a similar person, the debate between Sam and Helen about her presence on the board, probably other cases. I bet they end up with something similar, with different stakeholders represented. Give Bret Taylor some time to cook.
  - 0xDEAFBEAD 3y ago
    Thanks for the answer, interesting points!

by woopsn 3y ago

0xDEAFBEAD 3y ago

I've made this point before, but the ratio of "people criticizing OpenAI's legal structure" to "people proposing a better legal structure" in these discussions is remarkably high. Not one critic I've seen has stated what the best legal structure for OpenAI should actually have been, in a world where OpenAI really is stewarding a technology that could be an existential threat to humans. I really wish they would do so, because then we'd have a shot at improving OpenAI's governance. But so far it's been crickets. BTW, I think one of the big lessons of this episode is that legal structure matters much less than you'd think. On paper, OpenAI is supposed to follow its charter https://openai.com/charter https://openai.com/charter and the board had the power to fire Sam. In practice, Sam was kinda sorta violating the charter, but he's a charismatic leader who was about to bring in a big payday for the employees (https://www.cnbc.com/2023/11/30/openai-tender-offer-on-track-despite-leadership-fracas-sources.html https://www.cnbc.com/2023/11/30/openai-tender-offer-on-track...), and that turned out to matter more than the org's legal structure.

woopsn 3y ago
In that world there is no alternative, we should shut them down and break their stuff.
- 0xDEAFBEAD 3y ago
  Public opinion polls show significant support for regulating AI companies, e.g. here's one survey: https://www.linkedin.com/pulse/poll-shows-british-public-supports-moratorium-smarter-than-human-qvbpc https://www.linkedin.com/pulse/poll-shows-british-public-sup... Here's a group pushing for regulation and looking for volunteers: https://pauseai.info/ https://pauseai.info/ I would hesitate to physically break stuff -- sounds potentially unethical and could harm momentum for regulation
  - woopsn 3y ago
    I'm not advocating anything be broken. But if they're really doing what they say, and what I assumed you referred to by existential risk -- developing technologies "akin to the hydrogen bomb", that stand to "capture the light cone of all future value in the universe", etc. -- then of course that has to happen.
    - 0xDEAFBEAD 3y ago
      I encourage you to volunteer.
theGnuMe 3y ago
Capitalism ftw.
- 0xDEAFBEAD 3y ago
  I generally agree: https://news.ycombinator.com/item?id=38471297 https://news.ycombinator.com/item?id=38471297 However I make an exception for existentially dangerous technology
lacker 3y ago
Okay, I'll take a shot. I think the legal structure is very important, but the important part is not the vague wording of the charter. "Violating the charter" is always fuzzy and debatable. "Getting the votes of most board members" is not. So the important part is who controls the board. Instead of a structure where the board members get to choose other board members, I would suggest a structure where different outside groups get fixed numbers of board members. So, for example, it could have been (remembering that the initial board was just Sam Altman and Elon Musk): * One board member chosen by Sam Altman * One board member chosen by Elon Musk * One board member chosen by OpenPhil * One board member chosen by Microsoft * One board member chosen by some VC investor The precise composition is something they would argue about during funding, maybe some of these groups would get more or less seats. The root "governance" problem is that the board churned a lot because so many people had conflicts, or other reasons that I don't know as an outsider. The remaining board was unrepresentative of stakeholders (ie, no Elon Musk, nobody independently trusted by Microsoft) and probably not as experienced as you'd like. The problem with this sort of structure is that it gives everyone a lot of incentives to play politics with the composition of the board itself. Which happened a lot here - Reid Hoffman exiting and not really getting replaced by a similar person, the debate between Sam and Helen about her presence on the board, probably other cases. I bet they end up with something similar, with different stakeholders represented. Give Bret Taylor some time to cook.
- 0xDEAFBEAD 3y ago
  Thanks for the answer, interesting points!