3 ms·
So... I don't understand. >An action potential goes from the cell body down the axon to the axon terminals to another cell's dendrites. How do you figure that
by salawat 6y ago
So... I don't understand.
>An action potential goes from the cell body down the axon to the axon terminals to another cell's dendrites.
How do you figure that doesn't allow backprop?
A neuronal bit is a loop of neurons. Information absolutely can back- propagate. If it couldn't, how does anyone think it'd be at all possible to learn how to get better at anything?
Neuron fires dendrite to axon, secondary neuron fires dendrite to Axon, Axon branches back to previous neuron's dendrites, rinse, repeat, or add more intervening neurons... Trying to disinclude backprop based on the morphology of a single neuron is... Kinda missing the point.
It's all about the level of connection between neurons and how long or whether a signal returns unmodified to the progenitor that effects the stability of the encoded information or behavior. At least to the best I've been able to plausibly model it. Haven't exactly figured out how to shove a bunch of measuring sticks in there to confirm or deny, but I just can't how a uniderectional action potential forwarding element implies lack of backprop in a graph of connections fully capable of developing cycles.
- visarga 6y ago> How do you figure that doesn't allow backprop? When you backprop through a linear layer (a matrix W), you need to multiply with W.transpose which is impossible if connections are not bidirectional. > Information absolutely can back- propagate. If it couldn't, how does anyone think it'd be at all possible to learn how to get better at anything? Local error aggregation can have a similar effect with backprop but you can run each layer in parallel, you don't need to wait for the signal to reach the loss function and then gradients to come all the way back. An interesting read: Decoupled Neural Interfaces using Synthetic Gradients (DeepMind, 2017) http://proceedings.mlr.press/v70/jaderberg17a/jaderberg17a.pdf http://proceedings.mlr.press/v70/jaderberg17a/jaderberg17a.p...
- salawat 6y agoYou're conflating the implementation with the principle. There is no matrix math with neurons; quite the opposite, we posit the existence of the matrix math from the behavior we've observed with neural systems governed by a sigmoid function. The equations we've derived are secondary to the initial implementation. Just as you tweak the error factor in backprop, so too do weights between intersecting neuron networks adjust until thought and intention falls into line with eventual perception and execution. The map is not the territory.
- visarga 6y ago> There is no matrix math with neurons Are we talking about the brain or neural nets biological plausible neural nets?
- mr_toad 6y ago> If it couldn't, how does anyone think it'd be at all possible to learn how to get better at anything? Presumably part of the feedback loop (at least for things like motor skills and rote memorisation) is external to the brain. Our brain causes us to act, which alters our perceptions, which causes the brain to adjust.
- salawat 6y agoAnnnd... What exactly encodes that perception thing? The brain! There's nothing magicabout sense data except that it's how we're used to tweaking things. It is still, in the end, just more neurons telling other neurons what's off. Sure, some are hooked up to photoreceptors. I could swap those out with about anything else and still get learning. Unfortunately, the practical benefit of mastery of stuff you imagine tends to be less useful than it feels like it should be IRL.
- YeGoblynQueenne 6y ago>> If it couldn't, how does anyone think it'd be at all possible to learn how to get better at anything? Personally, when I've come to the point where I'm thinking to myself "that must be it, what else can it be?", I am at the point where I need to do more work to answer the latter part of the question.