4 ms·

> (which depend, by the way, on accelerating the world toward those hypothetical "concerning" scenarios as fast as possible) Yes, this dynamic is exactly the o

> (which depend, by the way, on accelerating the world toward those hypothetical "concerning" scenarios as fast as possible) Yes, this dynamic is exactly the one that anyone who's concerned about AI is concerned about. I don't know why you state this as if it's evidence against the concerns lol. Someone being concerned about the incentives of a situation doesn't de facto make them immune to those incentives, obviously. The implication that someone who's concerned about an arms race dynamic could simply opt out of the system that produces that dynamic is simply confused about what arms race dynamics are. The entire point is that it's a trap, and it's a trap even if you know it's a trap, and even if you don't like that it's a trap. There's nothing dishonest or hypocritical about being in the trap: it is literally a trap –– that is what it does and why it is bad! I'm confused by these comments that imply people believe Dario et al are "pure-of-heart Chosen Ones destined to lead us around the great filter." Who? I've never seen it. And any AI-doomer is probably of the opinion that the entire question of Dario's or anyone else's personal moral character is 99% irrelevant. Because, again, it's a trap. The dynamics at play are so much larger than whether someone irks people for their lecturing tone. I would much rather give my money to Dario, who seems like a generally good person, versus Sama, who seems like a complete snake, but I'm under no illusions that doing so changes the fundamental dynamics that are steering us to AI doom. I doubt anyone does. And yes, obviously they are angling toward being the winner who takes it all. That is literally the trap. If you believe what they believe, yelling "let's cooperate!" while hurdling towards the finish line and tripping your competitors is the only reasonable thing to do. That is the problem.

senordevnyc 5mo ago
This is an excellent comment, and I agree. I do think that there’s also evidence that Altman’s behavior can also be explained as a person who is naturally manipulative also being stuck in the trap and responding to incentives. But not necessarily a snake just in it for himself. The thing I keep coming back to about Altman: he doesn’t have any equity in OpenAI. And he definitely could have if he’d wanted. It’s hard for me to square that with the idea of him being greedy and self-interested.
caconym_ 5mo ago
> I don't know why you state this as if it's evidence against the concerns lol. Someone being concerned about the incentives of a situation doesn't de facto make them immune to those incentives, obviously. I think you're reading some subtext into my comment that I didn't intend. Knowing myself, I assume the scare quotes there are just a bit of casual irony re: the insanely high stakes here. The word "concerns" as used by previous commenters doesn't seem equal to the context. > The implication that someone who's concerned about an arms race dynamic could simply opt out of the system that produces that dynamic is simply confused about what arms race dynamics are. You can, in fact, opt out. You can opt out and do your damndest to stop what's happening, throw every cent you have at it, bend any ear that will listen, make use of the fact that your voice (as Anthropic leadership) has some meaningful weight. If you really believe that we are heading down a path that's likely to end poorly for most or all of humanity, and you are the kind of person who's inclined to favor saving billions of lives over saving your own skin when the stakes are still relatively distant, abstract, and generally unclear, opting out is obviously on the table as a grand gesture that burns your position in the race to show just how fucking serious you are. The sense of inevitability your comment shares with many others does not seem well founded---we have, for instance, not had a global nuclear war yet. Leaders in the 20th and 21st centuries have shown remarkable restraint. If today's political and tech leaders are unable to think beyond this inevitability, for whatever reason, the worst outcomes essentially become a self-fulfilling prophecy to the extent that reality bears them out. --- But yes, these people are acting the way they are for obvious reasons, obviously. My previous comment is reacting to the general disagreement over whether Anthropic actually believes what they say about safety, etc., or whether it's a marketing gimmick. The purpose of my comment is to explain that "it's hard not to be cynical" about actions taken by very rich and powerful people that are claimed to be in everybody's best interests but are indistinguishable from the actions they would take to maximize their future power and wealth. I think everyone ought to agree with that statement. It's not a value judgment; it's simply an observation of how it feels to be on a plane whose pilot appears to be robbing the passengers (including you) at gunpoint and is conspicuously wearing the only parachute on board.
- SubiculumCode 5mo ago
  It's not just America. The main secret is out of the bag. If it wasn't Anthropic, it would be another company/nation state. Sure they could obtain, and with not.money or leverage, complain about data centers at local rallies, or they can be in the game, and hopefully steer it. It's going to happen with or without any one company or country. The secret it out, and it's unstoppable without complete societal breakdown..So either you advocate for the end of civilization, or you hope that you can help steer the emergence of super intelligence into something not wholly terrible. Personally, I don't see much hope, even if there wasn't such a thing as AI. The power to destroy is always easier than the power to create, and as our power grows, the differential grows, until at some point, containment is no longer possible

senordevnyc 5mo ago

This is an excellent comment, and I agree. I do think that there’s also evidence that Altman’s behavior can also be explained as a person who is naturally manipulative also being stuck in the trap and responding to incentives. But not necessarily a snake just in it for himself. The thing I keep coming back to about Altman: he doesn’t have any equity in OpenAI. And he definitely could have if he’d wanted. It’s hard for me to square that with the idea of him being greedy and self-interested.

caconym_ 5mo ago

> I don't know why you state this as if it's evidence against the concerns lol. Someone being concerned about the incentives of a situation doesn't de facto make them immune to those incentives, obviously. I think you're reading some subtext into my comment that I didn't intend. Knowing myself, I assume the scare quotes there are just a bit of casual irony re: the insanely high stakes here. The word "concerns" as used by previous commenters doesn't seem equal to the context. > The implication that someone who's concerned about an arms race dynamic could simply opt out of the system that produces that dynamic is simply confused about what arms race dynamics are. You can, in fact, opt out. You can opt out and do your damndest to stop what's happening, throw every cent you have at it, bend any ear that will listen, make use of the fact that your voice (as Anthropic leadership) has some meaningful weight. If you really believe that we are heading down a path that's likely to end poorly for most or all of humanity, and you are the kind of person who's inclined to favor saving billions of lives over saving your own skin when the stakes are still relatively distant, abstract, and generally unclear, opting out is obviously on the table as a grand gesture that burns your position in the race to show just how fucking serious you are. The sense of inevitability your comment shares with many others does not seem well founded---we have, for instance, not had a global nuclear war yet. Leaders in the 20th and 21st centuries have shown remarkable restraint. If today's political and tech leaders are unable to think beyond this inevitability, for whatever reason, the worst outcomes essentially become a self-fulfilling prophecy to the extent that reality bears them out. --- But yes, these people are acting the way they are for obvious reasons, obviously. My previous comment is reacting to the general disagreement over whether Anthropic actually believes what they say about safety, etc., or whether it's a marketing gimmick. The purpose of my comment is to explain that "it's hard not to be cynical" about actions taken by very rich and powerful people that are claimed to be in everybody's best interests but are indistinguishable from the actions they would take to maximize their future power and wealth. I think everyone ought to agree with that statement. It's not a value judgment; it's simply an observation of how it feels to be on a plane whose pilot appears to be robbing the passengers (including you) at gunpoint and is conspicuously wearing the only parachute on board.

SubiculumCode 5mo ago
It's not just America. The main secret is out of the bag. If it wasn't Anthropic, it would be another company/nation state. Sure they could obtain, and with not.money or leverage, complain about data centers at local rallies, or they can be in the game, and hopefully steer it. It's going to happen with or without any one company or country. The secret it out, and it's unstoppable without complete societal breakdown..So either you advocate for the end of civilization, or you hope that you can help steer the emergence of super intelligence into something not wholly terrible. Personally, I don't see much hope, even if there wasn't such a thing as AI. The power to destroy is always easier than the power to create, and as our power grows, the differential grows, until at some point, containment is no longer possible