My issue is less with the strategy as NN optimisaiton, and more as the OP says, a road to general intelligence. There's an a priori "limit" on the form of any

My issue is less with the strategy as NN optimisaiton, and more as the OP says, a road to general intelligence. There's an a priori "limit" on the form of any time/space complexity "intelligence" can have: it cannot be exponential. We know that if you exhaust all infinite number of ways of describing a system by facts (/data), then you don't need intelligence. Whatever intelligence is, it's a strategy animals use for competent action in the face of their bodies not being able to exponentially expend energy to solve every trivial problem. Exponential complexity is a killer to the project of building intelligence in every respect. The universe, a priori, cannot require an exponential space complexity description of it to be parsable: since that would require exponentially many universes. Exponential scaling, for me (via ordinary conceptual analysis), seems a hallmark of any system not using intelligence. Exp. scaling is "the shortcut" to solving the problem without intelligence. It's the hallmark of the system not building O(log_{large number}) rich representations of the world.