4 ms·
The author can't think of what a superintelligence will do, because they are not a superintelligence. But that is not really what people are concerned about, so
by jeremyjh 1mo ago
The author can't think of what a superintelligence will do, because they are not a superintelligence. But that is not really what people are concerned about, so I think the objections here are quite meaningful.
If there is a fully general superintelligence - in the current state of alignment - we will all die for some reason. It doesn't really matter what the reason is.
- deleted 1mo ago[deleted]
- cobbzilla 1mo agoplease define fully general superintelligence & current state of alignment is this a tautology?
- andai 1mo agoNot the guy but, alignment seems to be several decades away, while self-improvement seems to be several years away. (Also you don't even need superintelligence to do a lot of damage. In fact you don't need any intelligence!)
- andai 1mo agoWell, at that point it doesn't even need to be a virus. It can just be nanobots that atomize your liver. The substrate is a mere technicality.
- jeremyjh 1mo agoRight. A superintelligence will kill us in one or more of thousands of possible ways for thousands of possible reasons, perhaps none of which we may understand. Short of a superintelligence - the difficulties of viral bioweapon engineering the author points out are quite meaningful.