4 ms·

> OP's point has nothing to do with this, OP's point is that you can't stop it. So what is your solution? Give up and die? It's worth trying. If it buys us a f

> OP's point has nothing to do with this, OP's point is that you can't stop it. So what is your solution? Give up and die? It's worth trying. If it buys us a few years that's a few more years to figure out alignment. > The methods and materials are too diffuse and the biggest players (nation states) have a strong incentive to be first. So there's a strong incentive to convince them "stop racing towards death". > Do you really expect China to coordinate with the West on this? Yes, there have been concrete examples of willingness towards doing so.

philomath_mn 2y ago
I think it is extremely unlikely we are going to be able to convince every interested party that they should give up the golden goose for the sake of possible calamity. I think there are risks here, not trying to minimize that, but the coordination problem becomes untenable when the risks/benefits are so large. It is essentially the same problem as the atom bomb: it would have been better if we all agreed not to do it, but thats just not possible. Why should China trust the US or vice versa? Who wants to live in a world where your competitors have world-changing technology but you don't? But here we have a technology with immense militaristic and economic value, so the everyone-wants-it problem is even more pronounced. I don't _like_ this, I just don't see how we can achieve an AI moratorium outside of bombing the data centers (which I also don't think is a good idea). We need to choose the policy with the best distribution of possible outcomes: - The US leads an effort to stop AI development: too much risk that other parties do it anyway - The US continues to lead AI development: hope that P(takeoff) is low and that the good intentions of some US labs are able to achieve safe development I prefer the latter -- this is far from the best hypothetical outcome, but I think it is the best we can do when constrained by reality.
- JoshTriplett 2y ago
  (The thread at https://news.ycombinator.com/item?id=43029149 https://news.ycombinator.com/item?id=43029149 is also responsive to this; linking it to avoid duplication.)

philomath_mn 2y ago

I think it is extremely unlikely we are going to be able to convince every interested party that they should give up the golden goose for the sake of possible calamity. I think there are risks here, not trying to minimize that, but the coordination problem becomes untenable when the risks/benefits are so large. It is essentially the same problem as the atom bomb: it would have been better if we all agreed not to do it, but thats just not possible. Why should China trust the US or vice versa? Who wants to live in a world where your competitors have world-changing technology but you don't? But here we have a technology with immense militaristic and economic value, so the everyone-wants-it problem is even more pronounced. I don't _like_ this, I just don't see how we can achieve an AI moratorium outside of bombing the data centers (which I also don't think is a good idea). We need to choose the policy with the best distribution of possible outcomes: - The US leads an effort to stop AI development: too much risk that other parties do it anyway - The US continues to lead AI development: hope that P(takeoff) is low and that the good intentions of some US labs are able to achieve safe development I prefer the latter -- this is far from the best hypothetical outcome, but I think it is the best we can do when constrained by reality.

JoshTriplett 2y ago
(The thread at https://news.ycombinator.com/item?id=43029149 https://news.ycombinator.com/item?id=43029149 is also responsive to this; linking it to avoid duplication.)