That implies we have hit or at least know the exact capacity ceiling for, in particular, recurrent/dynamic networks (I do agree that with static networks it see

That implies we have hit or at least know the exact capacity ceiling for, in particular, recurrent/dynamic networks (I do agree that with static networks it seems more clear cut). While it will not be general AI, I'd assume, things like the neutral Turing machine, pointer networks, and many other open research areas leave room to speculate that we have not yet "found" that ceiling.