3 ms·

More is different. Even when things share the same base units they can have completely different qualitative properties at scale based on the structure and envi

More is different. Even when things share the same base units they can have completely different qualitative properties at scale based on the structure and environments. You and the sun are both just leptons and baryons, but that's not very helpful. In this case, these models have vastly different architectures, learn language is vastly different format, and are given massive corpora of words written from millions of random perspectives instead of receiving words from one perspective via targeted communications. On what basis is there to expect such a system to magically develop the expressive intentional characteristics of human language when we can't even find another animal who seems to have this ability?

DavidSJ 5y ago
From my reading of where neuroscience is at the moment, it is a tenable hypothesis that the posterior neocortex (a major functional component of human cognition) is essentially a statistical inference machine. There are a number of facts suggestive of this viewpoint: For one, the apparent generality of the algorithms, resulting in a great deal of neuroplasticity and data-agnosticism. For example, you can wire visual input into auditory cortex and that auditory cortex will learn to see. [1] For two, the similarity of Gabor filters learned automatically by statistical inference algorithms and the response properties of visual neurons. [2] For three, the fantastic success, relative to other techniques at least, of such algorithms on natural language tasks such as machine translation. Even its deficits are suggestive, such as the fact that the neocortex, like most statistical inference algorithms, can't easily do one-shot learning — for that you need a hippocampus. Present-day machines obviously lack many abilities which humans have. Until we have a fully-functional AGI, it's hard to know for sure that there isn't a major missing piece which is nothing like the algorithms now under development. However, I'm not aware of any other algorithmic hypotheses of how the mind works which seem to so naturally fit the neuroscientific facts. [1] Sur et al (1988). Experimentally Induced Visual Projections into Auditory Thalamus and Cortex. https://doi.org/10.1126/science.2462279 https://doi.org/10.1126/science.2462279 [2] Olshausen and Field (1996). Emergence of simple-cell receptive field properties by learning a sparse code for natural images. https://doi.org/10.1038/381607a0 https://doi.org/10.1038/381607a0

DavidSJ 5y ago

From my reading of where neuroscience is at the moment, it is a tenable hypothesis that the posterior neocortex (a major functional component of human cognition) is essentially a statistical inference machine. There are a number of facts suggestive of this viewpoint: For one, the apparent generality of the algorithms, resulting in a great deal of neuroplasticity and data-agnosticism. For example, you can wire visual input into auditory cortex and that auditory cortex will learn to see. [1] For two, the similarity of Gabor filters learned automatically by statistical inference algorithms and the response properties of visual neurons. [2] For three, the fantastic success, relative to other techniques at least, of such algorithms on natural language tasks such as machine translation. Even its deficits are suggestive, such as the fact that the neocortex, like most statistical inference algorithms, can't easily do one-shot learning — for that you need a hippocampus. Present-day machines obviously lack many abilities which humans have. Until we have a fully-functional AGI, it's hard to know for sure that there isn't a major missing piece which is nothing like the algorithms now under development. However, I'm not aware of any other algorithmic hypotheses of how the mind works which seem to so naturally fit the neuroscientific facts. [1] Sur et al (1988). Experimentally Induced Visual Projections into Auditory Thalamus and Cortex. https://doi.org/10.1126/science.2462279 https://doi.org/10.1126/science.2462279 [2] Olshausen and Field (1996). Emergence of simple-cell receptive field properties by learning a sparse code for natural images. https://doi.org/10.1038/381607a0 https://doi.org/10.1038/381607a0