3 ms·
Yours is a good characterisation of Hebbian learning. Backpropagation in ANN & Hebbian learning in NN analagously strengthen correlations between inputs & succ
by deepnet 10y ago
Yours is a good characterisation of Hebbian learning.
Backpropagation in ANN & Hebbian learning in NN analagously strengthen correlations between inputs & succesful outputs - learning useful associations.
Stanley & Riskonen's NEAT evolves neural structure
http://nn.cs.utexas.edu/?neat http://nn.cs.utexas.edu/?neat
Schmidhuber & Togelius have nets that prune connections to evolve / learn structure.
- danielmorozoff 10y agoThank you for the references. I was not as much interested in mechanism to reduce the training speed by optimizing the structure in order to learn more efficiently. NEAT aims to 'minimize topologies and grow incrementally.' Rather my interest was in something similar to Q-learning mechanism or RL-LSTM that could learn structural sub-domains within a network, and reimplement them in other portions during training (maybe paralleling NEAT) or during instances of poor output probability.
- deepnet 10y agoThere is some evidence that large deep nets evolve using convergent strategies and solutions across modalities - at least for words and pictures. Low level specific edge & feature detectors combine into parts of things, these then combine into thing detectors and ultimately vectors representing objects and encoding their relationships. The final layer classification ( the training objective ) is a low dimensional slice of a rich conceptually navigable semantic vector space. That these top layer 'thought vectors' occur even for limited classification objectives implies the structure comes from the datas contextual relationships and the net learns how things relate to better classify what things are.