> The theory seems to rest on the idea that because AGI has a perfectly observable mechanical substrate, it will be possible to analyze and understand the funda

> The theory seems to rest on the idea that because AGI has a perfectly observable mechanical substrate, it will be possible to analyze and understand the fundamental causes of intelligence and thus improve upon it - a relatively simple matter of a software upgrade. I've never thought of it this way, that AGI will necessarily develop because it beyond some point picks up understanding about itself and then intentionally improves. I rather see it this way: the development of AGI is in the beginning rather unintentional. It will just be the result of a system dynamic that is more stable than other dynamics. Evolution isn't intentional either, is it? It's coincidence and randomness, trial and error, and dynamically more stable beings outlive other less stable beings. Stability appears to me to somewhat correlate with complexity. More complex beings having more ability to adapt and thus be able to survive in a greater set of conditions. As soon as the complexity of a beings information processing - which is mostly represented through number of brain cells and links (in biology) and number of artificial neurons (in AI) - goes into a dynamic of growth (like a positive feedback loop), I'd say it's pretty much on the pathway to AGI. Provided, the being isn't physically limited to a degree of complexity that's just too low.