3 ms·

This is what AI safety activist Connor Leahy calls offense-defense asymmetry. Imagine if a nuke could be created with household materials. We got lucky that was

This is what AI safety activist Connor Leahy calls offense-defense asymmetry. Imagine if a nuke could be created with household materials. We got lucky that wasn't the case, but we didn't know that it wasn't the case before studying physics. If we weren't so lucky that physics happened to turn out that way, we already wouldn't exist, because the offense would be so much stronger than defense, we wouldn't stand a chance even with the best surveillance systems. A profitable prediction markets trader who goes by the alias semiotic rivalry posted this prediction on his X account recently: > Prediction: in 1-2 years, almost everyone will agree that open sourcing the best AIs is totally crazy as the AIs become capable of autonomous hacking of most humans and institutions, serious terrorism and creation of biological weapons, etc. And I mostly agree with him. As much as it's awful to have such a concentration of capabilities in the hands of a few governments or corporations, as far as I can tell there are no easy answers here, only a matter of trade offs and different risks.

salawat 2mo ago
Keep in mind the world is run by hierarchs. Of course the solution will look hierarchical in nature. The problem is fundamentally speaking, the hierarchism is a self perpetuating problem, because we still have to solve who amongst us can be trusted with the top of the hierarchy, which hierarchism states by definition "most of us can't be". It's like encryption, it's excellent at turning every other hard problem into a key management problem.
- energy123 2mo ago
  Non-hierarchical structures (like anarchy) are plausible only when defence is stronger than offence, because the individual/clan/nation can defend against any offence from any other entity, while also being incapable of attacking any other entity due to weak offence relative to the other entity's defence. But reality doesn't owe us this luxury, unfortunately. Offense increases entropy. It's easy to destroy. Attack surfaces are numerous. Humans are fragile. It's hard to defend.

salawat 2mo ago

Keep in mind the world is run by hierarchs. Of course the solution will look hierarchical in nature. The problem is fundamentally speaking, the hierarchism is a self perpetuating problem, because we still have to solve who amongst us can be trusted with the top of the hierarchy, which hierarchism states by definition "most of us can't be". It's like encryption, it's excellent at turning every other hard problem into a key management problem.

energy123 2mo ago
Non-hierarchical structures (like anarchy) are plausible only when defence is stronger than offence, because the individual/clan/nation can defend against any offence from any other entity, while also being incapable of attacking any other entity due to weak offence relative to the other entity's defence. But reality doesn't owe us this luxury, unfortunately. Offense increases entropy. It's easy to destroy. Attack surfaces are numerous. Humans are fragile. It's hard to defend.