4 ms·
That's a bit naive, to put it mildly. It presumes that nobody else would be able to replicate the effort and that the parties that are able to replicate it woul
by jacquesm 3y ago
That's a bit naive, to put it mildly. It presumes that nobody else would be able to replicate the effort and that the parties that are able to replicate it would also destroy theirs after proving that it could be done. Fat chance.
- barrkel 3y agoIndeed. The only way to retain control of the leading ship in the race is to keep it together as you steer it. If the ship disintegrates, then you're no longer in control of the leading ship, and someone else will win the race.
- 0xDEAFBEAD 3y agoThe charter states that OpenAI should allow another company to win the race under certain circumstances: >We are concerned about late-stage AGI development becoming a competitive race without time for adequate safety precautions. Therefore, if a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop competing with and start assisting this project. We will work out specifics in case-by-case agreements, but a typical triggering condition might be “a better-than-even chance of success in the next two years.” https://openai.com/charter https://openai.com/charter That's why Sam pressuring Helen to shut up about Anthropic is so concerning.
- jacquesm 3y ago> we commit to stop competing with and start assisting this project That bit I never understood. It doesn't even have a qualification as to who is stewarding the project as if any AI should be assisted no matter who ends up owning it. The whole thing is just so utterly disconnected from the real world.
- 0xDEAFBEAD 3y agoIf you believe that aligned AGI will create a post-scarcity society, there's no reason for its creator to hoard the benefits for themselves. The main thing is to ensure that post-scarcity society actually comes into being. Edit: Note also the "value-aligned, safety-conscious" caveat.
- jacquesm 3y agoI don't believe that. What I believe is that the first party to create an AGI will likely end up being nationalized and that that AI will be weaponized the next day to the exclusion of the rest of the world.
- comp_throw7 3y agoThat's fine, but the OpenAI founders (including Sam Altman![1]) explicitly had in mind "AI kills everyone" as one of the main problems they were attempting to tackle when they started the organization. (Obviously starting OpenAI as a solution to this problem was a terrible mistake, but given that they did so, having a board charged with pulling the plug is better than not having that.) [1] https://blog.samaltman.com/machine-intelligence-part-1 https://blog.samaltman.com/machine-intelligence-part-1
- jacquesm 3y agoThat whole charter is bullshit. You don't invite Microsoft to the party if you care about ethics and you don't disable your fail-safe by giving them full access to all of your stuff. Hence my position that the charter was a fig leaf designed to throw regulators off.
- 0xDEAFBEAD 3y agoElsewhere in this thread you stated: "if OpenAI was 'the good guys' then any chance of the good guys getting a head start has just been blown out of the water" So basically we have two cases here: 1. The charter is a deceptive fig leaf, OpenAI aren't the good guys. In which case blowing up OpenAI looks reasonable 2. The charter is not a fig leaf, Helen is doing the right thing by upholding it I'd say upholding the charter is the right thing in intermediate situations as well. Basically, OpenAI are the good guys, in part, to the degree to which the board (and other stakeholders) actually uphold the charter instead of being purely profit-driven.
- 3y ago
- wnoise 3y agoDid you miss the part where it says "value-aligned"? One's that aren't value-aligned are not subject to that clause.
- ytoawwhra92 3y agoThe actions of other organisations are not in the scope of the board's mission. The actions of the company the board controls are in that scope. "The board’s mission is to ensure that the company creates artificial intelligence that “benefits all of humanity,”" We can't control other's actions, but we can control our own. If we feel that our actions are diverging from our own sense of what we ought to be doing we can change our actions, regardless of how others are behaving.
- jacquesm 3y agoThat is absolutely true. But you do operate on a larger stage and sticking your head in the sand and pretending the rest of the world doesn't exist is also not going to work. Altman appears to me to be reckless, ms. Toner appears to be naive. Both should not be near this kind of power.
- nicce 3y agoI would not say that it is fair comparison in this case. People do the best with the control they have. In this case, if your company fails to provide for greater good, then it is best to destroy it. And it also shows some example to others. Whatever others do, you have no power. I does not matter if you pretent or not. Comparison works only if you could do something about it.
- jacquesm 3y agoYes, but then maybe you should realize that you are out of your depth and abstain. Because clear as day this kind of power is actively being sought by many operators and if OpenAI was 'the good guys' then any chance of the good guys getting a head start has just been blown out of the water. If she agreed with the Microsoft deal but not with what followed that too would be hopelessly naive. Microsoft + ethics?
- nicce 3y ago> If she agreed with the Microsoft deal but not with what followed that too would be hopelessly naive. Microsoft + ethics? I guess the main problem here is that nobody knows properly what this deal contains and how it was made. I believe CEO has power to do such deal alone?