2 ms·

Thank you for the references. I was not as much interested in mechanism to reduce the training speed by optimizing the structure in order to learn more efficie

Thank you for the references. I was not as much interested in mechanism to reduce the training speed by optimizing the structure in order to learn more efficiently. NEAT aims to 'minimize topologies and grow incrementally.' Rather my interest was in something similar to Q-learning mechanism or RL-LSTM that could learn structural sub-domains within a network, and reimplement them in other portions during training (maybe paralleling NEAT) or during instances of poor output probability.

deepnet 10y ago
There is some evidence that large deep nets evolve using convergent strategies and solutions across modalities - at least for words and pictures. Low level specific edge & feature detectors combine into parts of things, these then combine into thing detectors and ultimately vectors representing objects and encoding their relationships. The final layer classification ( the training objective ) is a low dimensional slice of a rich conceptually navigable semantic vector space. That these top layer 'thought vectors' occur even for limited classification objectives implies the structure comes from the datas contextual relationships and the net learns how things relate to better classify what things are.