A neural network capable of making generic inferences about the output of another neural network, in real time, just needs to be able to run a simulation of tha

A neural network capable of making generic inferences about the output of another neural network, in real time, just needs to be able to run a simulation of that other network. While we do not currently have the resources to do it with a human brain, it is in principle quite doable. The critical difference between this and the Halting problem is that solving the Halting problem by running a simulation would require simulating in finite time what another system does in infinite time. This requires simulating in finite time what the other system also does in finite time. Which requires a better system, but not an impossibly better system. Moving on, you are over-estimating the requirements of the system that I describe. Today, with humans, a system like Waze can provide warnings about the road ahead which provides a useful assist to human drivers. This with a very small fraction of humans using the program, and even fewer actively registering hazards like "object on road". And yet, it is useful. Automated driving systems can participate in a similar system. Only they will be more likely to provide information, and their information can be more detailed. Such as what lane the foreign object is in. It doesn't take a lot of data about what is out of view ahead to improve driving by a lot. But humans are built to only pay attention to one thing at that speeds. Automated systems can integrate information from multiple. Which means that they can be better. Yes, what they are doing is simply feature extraction/input output mapping/information synthesis - but in principle nothing more is needed to drive better than is possible for humans. At the moment, humans are better. But it is not impossible that computers can become better. It is, in fact, inevitable.