3 ms·

TBH the idea that humans learn anything on a blank slate pure learning architecture is messed up. Our brains are literally evolved to interpret causality into t

TBH the idea that humans learn anything on a blank slate pure learning architecture is messed up. Our brains are literally evolved to interpret causality into the world. It isn't randomly just "learned" any more than walking is.

andyxor 6y ago
exactly, besides ignoring the innate structures, heuristics and biases hardcoded via evolution, the whole notion of "learning" became highly intertwined with reinforcement kind of learning, i.e trial & error, stimulus and response behaviorist terms popularized by Pavlov and Skinner a century ago, which is just one type in a large repertoire of adaptation mechanisms. Memory in these models is used as afterfact, or some side utility for complex iterative routines based on calculus of function optimization. While in living organisms memory and its "hardcoded" shortcuts allow to cut through the search space quickly as in a large database index. Speaking in database terms we have something like "materialized views" on acquired and genetically inherited knowledge, built from compressed and hierarchically organized sensory data and prior related actions and associations, including causal links. Causality is just a way to associate items in the memory graph. Error correction doesn't play as much role in storing and retrieving information and pattern recognition, as current machine learning models may lead you to believe. Instead, something akin to self-organized clustering is going on, with new info embedded in the existing "concept" graph via associations and generalizations, through simple LINK and JOIN mechanisms on massive scale.[1] The formation of this graph in long term memory is tightly coupled with sleep cycles and memory consolidation, while short term memory serves as a kind of cache. Knowledge is organized hierarchically starting from principal components [2] of sensory data from e.g. visual receptive fields, and increasing in level of abstraction via "chunking", connecting objects A and B to form a new object C via JOIN mechanism, or associating objects A and B via LINK mechanism. Both LINK and JOIN outputs are "persisted" to memory via Hebbian plasticity. All knowledge including causal links are expressed via this simple mechanism. Generating a prediction given a new sensory signal is just LINKing the signal with existing cluster by similarity. Navigation in this abstract space is facilitated via coordinate system similar or perhaps identical to the role hippocampal place & grid cells play in spatial navigation. Similarity between objects is determined as similarity between their "embeddings" in this abstract concept space. It's possible that innate structures are genetically pre-wired in this graph which represent high level "schemas", such as innate language grammar which distinguishes e.g. verb from noun, visual object grammar which distinguishes "up" from "down", etc. It is also possible these are embodied, i.e. connected to some representation of motor and sensory embeddings. And serve to bootstrap the graph structure for subsequent knowledge acquisition. I.e. no blank slate. The information is passed, stored and retrieved via several (analogue) means both in point-to-point and broadcast communication, with electromagnetic oscillations playing primary role in synchronization in neural assemblies, facilitating e.g. speech segmentation (or boundary detection in general), and coupling an input signal "embedding" to existing knowledge embeddings in short term memory; while neural plasticity/LTP/STDP as storage mechanisms on single neuron level. [1] See Leslie Valiant "neuroidal" model and his book https://www.amazon.com/Circuits-Mind-Leslie-G-Valiant/dp/0195126688 https://www.amazon.com/Circuits-Mind-Leslie-G-Valiant/dp/019... [2] See Oja Rule http://www.scholarpedia.org/article/Oja_learning_rule http://www.scholarpedia.org/article/Oja_learning_rule and Olshausen & Field classic work on sparse coding http://www.scholarpedia.org/article/Sparse_coding http://www.scholarpedia.org/article/Sparse_coding

andyxor 6y ago

exactly, besides ignoring the innate structures, heuristics and biases hardcoded via evolution, the whole notion of "learning" became highly intertwined with reinforcement kind of learning, i.e trial & error, stimulus and response behaviorist terms popularized by Pavlov and Skinner a century ago, which is just one type in a large repertoire of adaptation mechanisms. Memory in these models is used as afterfact, or some side utility for complex iterative routines based on calculus of function optimization. While in living organisms memory and its "hardcoded" shortcuts allow to cut through the search space quickly as in a large database index. Speaking in database terms we have something like "materialized views" on acquired and genetically inherited knowledge, built from compressed and hierarchically organized sensory data and prior related actions and associations, including causal links. Causality is just a way to associate items in the memory graph. Error correction doesn't play as much role in storing and retrieving information and pattern recognition, as current machine learning models may lead you to believe. Instead, something akin to self-organized clustering is going on, with new info embedded in the existing "concept" graph via associations and generalizations, through simple LINK and JOIN mechanisms on massive scale.[1] The formation of this graph in long term memory is tightly coupled with sleep cycles and memory consolidation, while short term memory serves as a kind of cache. Knowledge is organized hierarchically starting from principal components [2] of sensory data from e.g. visual receptive fields, and increasing in level of abstraction via "chunking", connecting objects A and B to form a new object C via JOIN mechanism, or associating objects A and B via LINK mechanism. Both LINK and JOIN outputs are "persisted" to memory via Hebbian plasticity. All knowledge including causal links are expressed via this simple mechanism. Generating a prediction given a new sensory signal is just LINKing the signal with existing cluster by similarity. Navigation in this abstract space is facilitated via coordinate system similar or perhaps identical to the role hippocampal place & grid cells play in spatial navigation. Similarity between objects is determined as similarity between their "embeddings" in this abstract concept space. It's possible that innate structures are genetically pre-wired in this graph which represent high level "schemas", such as innate language grammar which distinguishes e.g. verb from noun, visual object grammar which distinguishes "up" from "down", etc. It is also possible these are embodied, i.e. connected to some representation of motor and sensory embeddings. And serve to bootstrap the graph structure for subsequent knowledge acquisition. I.e. no blank slate. The information is passed, stored and retrieved via several (analogue) means both in point-to-point and broadcast communication, with electromagnetic oscillations playing primary role in synchronization in neural assemblies, facilitating e.g. speech segmentation (or boundary detection in general), and coupling an input signal "embedding" to existing knowledge embeddings in short term memory; while neural plasticity/LTP/STDP as storage mechanisms on single neuron level. [1] See Leslie Valiant "neuroidal" model and his book https://www.amazon.com/Circuits-Mind-Leslie-G-Valiant/dp/0195126688 https://www.amazon.com/Circuits-Mind-Leslie-G-Valiant/dp/019... [2] See Oja Rule http://www.scholarpedia.org/article/Oja_learning_rule http://www.scholarpedia.org/article/Oja_learning_rule and Olshausen & Field classic work on sparse coding http://www.scholarpedia.org/article/Sparse_coding http://www.scholarpedia.org/article/Sparse_coding