3 ms·
Understanding the underlying structure is key. And the significance of structure -- the ability to see it, understand it, and re-create it -- has been recognize
by espeed 7y ago
Understanding the underlying structure is key. And the significance of structure -- the ability to see it, understand it, and re-create it -- has been recognized and pointed out by Feynman [1] and some of the world's best programmers [2]. Using data-structure representations that best match reality's underlying structure may be key too. With this in mind I had a thought -- a "dopey idea" in the Steve Jobs to Jony Ive sense of the word [3] -- the brain doesn't do backprop [4] -- at least not directly (as far as we know) -- but neural nets using backprop/gradient-descent work pretty well. Why so?
What if the underlying structure these nets are modeling isn't the brain but something else...maybe something more fundamental...like the curvature of spacetime, a path integral representation of information, or some other crude representation of an underlying quantum structure. And if so, what if learning is just the brain optimizing itself in accordance with this structure, and has this idea been explored before? Evidently it has. The idea of the brain as a quantum information processor has been explored to some degree in holonomic brain theory [5].
[1] "What I cannot create, I do not understand" https://en.wikiquote.org/wiki/Richard_Feynman https://en.wikiquote.org/wiki/Richard_Feynman
[2] Programmers on Data Structures https://news.ycombinator.com/item?id=20047607 https://news.ycombinator.com/item?id=20047607
[3] Jony Ive talking about Steve Jobs' Dopey Ideas [video] https://www.youtube.com/watch?v=4WNYYCX3Tg0 https://www.youtube.com/watch?v=4WNYYCX3Tg0
[4] Artificial intelligence pioneer says we need to start over https://www.axios.com/artificial-intelligence-pioneer-says-we-need-to-start-over-1513305524-f619efbd-9db0-4947-a9b2-7a4c310a28fe.html https://www.axios.com/artificial-intelligence-pioneer-says-w...
[5] Holonomic Brain Theory https://en.wikipedia.org/wiki/Holonomic_brain_theory https://en.wikipedia.org/wiki/Holonomic_brain_theory
- bawana 7y agothe principle of backprop can be accomplished in structural changes to the synapse. Although that would be analogous to changing the activation function in each 'neuron' of a layer in a convnet-we have just 'factored it out' into a sequence of weights used in the conv layers. Up regulating and down regulating the number of receptors can change how each synapse contributes to a biological net.