I'm not sure that we can ever 'understand' intelligence in humans, let alone something as fluffy as consciousness simply because the systems are so vast, interc

I'm not sure that we can ever 'understand' intelligence in humans, let alone something as fluffy as consciousness simply because the systems are so vast, interconnected, and inseparable. It seems more and more apparent that the networks and structures that cause 'us' to emerge are so deeply non-hierarchical and interconnected that labeling parts in a way where we can say this-does-that and that-does-this is not a meaningful idea. It's not like a car engine where a component does one thing, has an input from this system, and produces an output to this system. It's more like a weather system where small deviation in one part of the system can have small and large changes everywhere else, which in turn cause changes of their own, and on and on. I suspect that AGI will first come about as a mishmash of 'expert systems' with some currently incomprehensible glue allowing them all to communicate effectively. I further guess that it's development will be incremental in nature - taking tiny pieces of things that work and putting them together with novel techniques that also worked somewhere else until eventually you get something that thinks back at you.