Sure, sure probably. But there might be some interesting test, where something that is really AGI, and fails something like a "recognizing yourself in a mirror"

Sure, sure probably. But there might be some interesting test, where something that is really AGI, and fails something like a "recognizing yourself in a mirror" test (not that specific test obviously, but you see what I'm getting at--some sort of trick)...that works for intelligence + consciousness but not just intelligence. No idea what that could be, but it's possible it could exist. Again I think we need to separate out that consciousness doesn't imply soul. So we have a few things: soul, consciousness, intelligence, and we are really only interested here in constructing the latter in an AGI. Just thinking about a nice analogy just now: may be a glib oversimplification but one way to characterize soul, consciousness, intelligence is as: memory, present awareness, imagination. What is a soul if not a memory of all experiences of that soul? What is consciousness if not awareness of the present? What is intelligence if not some sort of sophisticated imagination that is useful/aligns with some model (reality, or another)? Just an idea Another interesting thing is: if your reasoning is fast enough, it doesn't have to be abstract. You can probably get the same speedup that abstraction/generalization provides to categories of observations with enough parallel processing of specific observations. Obviously from that view then abstraction is like a hack for humans to overcome neural speed limits/memory limits, but it may not be necessary for AGI: I think this is one reason why AGI (or at least its inner workings) may remain incomprehensible to us.