"ASI" is still limited by physics and logic. Or if not, we're just more screwed, maybe. I suppose we're in agreement that in the long run (maybe longer than we

"ASI" is still limited by physics and logic. Or if not, we're just more screwed, maybe. I suppose we're in agreement that in the long run (maybe longer than we think with some really cool stuff like Ems in between...), the likely scenario is we're screwed one way or another? I'm just glad people are working on trying to discover the chains of logic that let us not be so screwed, hard as that problem may be. I don't see any paradox there that makes the problem impossible, logic is logic independent of intelligence. Similarly, I'm very sure that no one else and no superintelligent AGI can ever reproduce the original string I used to produce the sha512 hash of a5136df53e73c4fdcde80d9ba41f2526c6e144e344c85a355f518b161f5b5171a6de831908ec32f6a6e503ae055219027d8e9890517977bbcd16a6cd9f36c858 (perhaps they'll find a collision). (Incidentally there are many secure systems and schemes that have not fallen. But you'd probably write them off as too narrow for some reason.) Yes, there's "difficulty comprehending" current deep learning models, put modestly. There's also interesting work in interpretability/explainability going on to work on that. Nothing there says "impossible". However, even granting that it is actually impossible to explain and reason about current AI methods to the degree we need to in order to create chains of logic for alignment purposes, that just says that it's impossible to reason about an AGI produced using such methods, if it's even possible to produce an AGI that way. It says nothing about it being impossible to reason about an AGI produced using different methods.