> In contrast, a neural component is a (typically over-)parameterized, differentiable blackbox function that does not have an a priori specification. > The hol

> In contrast, a neural component is a (typically over-)parameterized, differentiable blackbox function that does not have an a priori specification. > The holy grail for research on neurosym- bolic programming would be a general learning algorithm that could efficiently generate arbitrary combinations of neural components and symbolic code. This is one area where the authors may be missing the point of the parallel distributed processing approach. The authors treat ANN’s as blind guesses from massive training data, but they forget that in most cases the choice of training data is the a priori specification. Additionally, an ANN can be comprised of multitudes of types of artificial neurons. While some learning models are designed to discover relationships between complex data, other learning models are designed for simple data. The simpler the training data, the easier to assign a meaning to the output. Simple back-propagation was followed by the development of deep learning and complex network architectures. The explosion in interest in deep learning is, in part, due to the realization that complex neural architectures can be comprised of a variety of neural components in a variety of ways and that the resulting neural architectures would almost certainly avail us with solutions that span the entirety of possible problems to solve. That is to say, deterministic behavior (“symbolic program synthesis”) could be implemented by an ANN, if the ANN had the required minimum complexity. Since the minimum complexity of such an ANN is currently incomprehensibly high, due to deep learning being in its infancy, there has been a heavy focus on striking results with low complexity. In my view, instead of trying to bring back the zombie of symbolic programming for AI, the authors’ time would be better spent on contributing novel, highly complex neural architectures. For example, some biological neurons in the visual cortex have been shown to be responsive to single individuals faces. If there is one neuron per face, then there could be thousands or more of face neurons in the visual cortex. Where normal deep learning would focus on parameterized feature recognition for face detection, a highly complex neural architecture would focus on multitudes of small models that are trained with minute data sets of individuals per model. A standard deep learning “face detection” model could then feed the training of the “face neurons” Layering other types of neural architecture around and within that multitude of models, along with some other neural architecture that generates new “face neurons” to be included in the multitude, would provide both the robustness of deep learning and the determinism of symbolic programming. Each “face neuron” would have an a priori specification of which particular face data was used in its training. One could even imagine further granularity such as “smile neurons”, “chin neurons”, “pimple neurons”, and so on. That is neuro-glial programming, a subset of Parallel Distributed Processing, not neuro-symbolic programming. https://www.igi-global.com/chapter/artificial-neuroglial-networks/10243 https://www.igi-global.com/chapter/artificial-neuroglial-net...