4 ms·

Sabotaging your own company is not enough. You'd have to sabotage every other AI lab too. If you assume that's impossible, attempting to be the first to produce

by mrob 12d ago

Sabotaging your own company is not enough. You'd have to sabotage every other AI lab too. If you assume that's impossible, attempting to be the first to produce ASI is a rational alternative, even if you think it will most likely kill you. At least that way you have some control over the P(doom).

deleted 12d ago
[deleted]
goatlover 12d ago
This sounds like a rationalization for gambling with everyone's lives, assuming they really believe such a thing.
- mrob 12d ago
  What's the alternative? There's no realistic way to globally coordinate and enforce a ban. A direct action campaign against AI labs would only motivate increased security until sabotage becomes impossible. Starting a nuclear war would probably be enough to stop dangerous AI research, and that's only likely to kill about 50% of humans instead 100%, so it's a great improvement, but how is an individual supposed to start a nuclear war? Of course it would be better if everybody stopped, but it's a prisoner's dilemma scenario with no possibility of future rounds, so the rational play is to defect.
  - MentalM 12d ago
    > There's no realistic way to globally coordinate and enforce a ban. I don't see what is unrealistic about that. The major players already have strong incentives to freeze AI advancements at the the current state, while the technology is highly efficient for totalitarian surveillance systems that can eliminate any threat to their power in the buds, but still not so advanced that it could create crises that might endanger their power. So I think the situation where several largest countries declare AI a deadly threat for humanity and start fighting it development much like they fight ISIS to be quite plausible. This isn't a prisoner's dilemma; the distribution of risks and rewards is entirely different. Ban AI research, give away advanced models, bomb a couple of Russian datacenters where they launching those models unsafely, and then AI development will stop in the entire world.
bigstrat2003 12d ago
> If you assume that's impossible, attempting to be the first to produce ASI is a rational alternative, even if you think it will most likely kill you. That is completely irrational.
- mrob 12d ago
  1% chance of survival beats 0%.
- pixl97 12d ago
  Humans aren't exactly rational

by mrob 12d ago

deleted 12d ago

[deleted]

goatlover 12d ago

This sounds like a rationalization for gambling with everyone's lives, assuming they really believe such a thing.

mrob 12d ago
What's the alternative? There's no realistic way to globally coordinate and enforce a ban. A direct action campaign against AI labs would only motivate increased security until sabotage becomes impossible. Starting a nuclear war would probably be enough to stop dangerous AI research, and that's only likely to kill about 50% of humans instead 100%, so it's a great improvement, but how is an individual supposed to start a nuclear war? Of course it would be better if everybody stopped, but it's a prisoner's dilemma scenario with no possibility of future rounds, so the rational play is to defect.
- MentalM 12d ago
  > There's no realistic way to globally coordinate and enforce a ban. I don't see what is unrealistic about that. The major players already have strong incentives to freeze AI advancements at the the current state, while the technology is highly efficient for totalitarian surveillance systems that can eliminate any threat to their power in the buds, but still not so advanced that it could create crises that might endanger their power. So I think the situation where several largest countries declare AI a deadly threat for humanity and start fighting it development much like they fight ISIS to be quite plausible. This isn't a prisoner's dilemma; the distribution of risks and rewards is entirely different. Ban AI research, give away advanced models, bomb a couple of Russian datacenters where they launching those models unsafely, and then AI development will stop in the entire world.

bigstrat2003 12d ago

> If you assume that's impossible, attempting to be the first to produce ASI is a rational alternative, even if you think it will most likely kill you. That is completely irrational.

mrob 12d ago
1% chance of survival beats 0%.
pixl97 12d ago
Humans aren't exactly rational