16 ms·
Predictive coding has been unified with backpropagation
- AbrahamParangi 6y agoThis is an overly strong claim for the paper (which is good!) backing it. If anyone is interested in the reader's digest version of the original paper check out https://www.youtube.com/watch?v=LB4B5FYvtdI https://www.youtube.com/watch?v=LB4B5FYvtdI
- visarga 6y agonice video
- blueyes 6y agoI'm glad people are talking about this, and the similarity between predictive coding and the action of biological neurons is interesting. But we shouldn't fetishize predictive coding. There's a wider discussion going on, and several theories as to how back propagation might work in the brain. https://www.cell.com/trends/cognitive-sciences/fulltext/S1364-66131930012-9 https://www.cell.com/trends/cognitive-sciences/fulltext/S136... https://www.nature.com/articles/s41583-020-0277-3 https://www.nature.com/articles/s41583-020-0277-3
- andyxor 6y agothere is no evidence of back-propagation in the brain. See Professor Edmund T. Rolls books on biologically plausible neural networks: "Brain Computations: What and How" (2020) https://www.amazon.com/gp/product/0198871104 https://www.amazon.com/gp/product/0198871104 "Cerebral Cortex: Principles of Operation" (2018) https://www.oxcns.org/b12text.html https://www.oxcns.org/b12text.html "Neural Networks and Brain Function" (1997) https://www.oxcns.org/b3_text.html https://www.oxcns.org/b3_text.html
- ShamelessC 6y ago"There is just one problem: [biological neural networks] are physically incapable of running the backpropagation algorithm." From the linked article.
- blueyes 6y agoI read that sentence. The article is not the only source of truth in brain function, and its author may be too certain about the brain. In any case, there will always be dissimilarities between biological neurons and computations on silicon, which probably shouldn't be called neurons, in order to avoid confusion.
- ShamelessC 6y agoI agree. Really don't appreciate the level at which researchers are willing to make these comparisons right now. They're moving fast and publishing things. It's probably far too late to change the name for computational neural nets - but I agree. Something like a "differentiable learning graph" or something would be better.
- xzvf 6y agoAt scale, Evolutionary Strategies (ES) are a very good approximation of the gradient as well. Don’t recommend to jump just yet to conclusions and unifications.
- jnwatson 6y agoThe author's point is that predictive coding is a plausible mechanism by which biological neurons work. ES are not. ANNs have deviated widely from their biological inspiration, most notably in the way that information flows, since backpropagation requires two way flow and biological axons are one-directional. If predictive coding and backpropagation are shown to have similar power, then there's a rough idea that the way that ANNs work isn't too far from how brains work (with lots and lots of caveats).
- whimsicalism 6y ago> If predictive coding and backpropagation are shown to have similar power, then there's a rough idea that the way that ANNs work isn't too far from how brains work (with lots and lots of caveats). So many caveats that I don't even really think that is a true statement.
- nmca 6y agoInteresting discussion on the ICLR openreview, resulting in a reject: https://openreview.net/forum?id=PdauS7wZBfC https://openreview.net/forum?id=PdauS7wZBfC
- justicezyx 6y agoCopied from this URL, the final review comments that 1) summarized the other reviews, 2) describes the rational for rejection: ``` This paper extends recent work (Whittington & Bogacz, 2017, Neural computation, 29(5), 1229-1262) by showing that predictive coding (Rao & Ballard, 1999, Nature neuroscience 2(1), 79-87) as an implementation of backpropagation can be extended to arbitrary network structures. Specifically, the original paper by Whittington & Bogacz (2017) demonstrated that for MLPs, predictive coding converges to backpropagation using local learning rules. These results were important/interesting as predictive coding has been shown to match a number of experimental results in neuroscience and locality is an important feature of biologically plausible learning algorithms. The reviews were mixed. Three out of four reviews were above threshold for acceptance, but two of those were just above. Meanwhile, the fourth review gave a score of clear reject. There was general agreement that the paper was interesting and technically valid. But, the central criticisms of the paper were: Lack of biological plausibility The reviewers pointed to a few biologically implausible components to this work. For example, the algorithm uses local learning rules in the same sense that backpropagation does, i.e., if we assume that there exist feedback pathways with symmetric weights to feedforward pathways then the algorithm is local. Similarly, it is assumed that there paired error neurons, which is biologically questionable. Speed of convergence The reviewers noted that this model requires many more iterations to converge on the correct errors, and questioned the utility of a model that involves this much additional computational overhead. The authors included some new text regarding biological plausibility and speed of convergence. They also included some new results to address some of the other concerns. However, there is still a core concern about the importance of this work relative to the original Whittington & Bogacz (2017) paper. It is nice to see those original results extended to arbitrary graphs, but is that enough of a major contribution for acceptance at ICLR? Given that there are still major issues related to (1) in the model, it is not clear that this extension to arbitrary graphs is a major contribution for neuroscience. And, given the issues related to (2) above, it is not clear that this contribution is important for ML. Altogether, given these considerations, and the high bar for acceptance at ICLR, a "reject" decision was recommended. However, the AC notes that this was a borderline case. ``` The core reason is that the proposed model lacks biological plausibility. Or, if ignoring this weakness, the model is then computationally more intensive. I HAVE NOT read the paper, but the review seems mostly based "feeling"; i.e., the reviewers feel that this work is not above the bar. Note that I am not criticizing the reviewers here, in my past review career of maybe in the range of 100+ papers, which I did until 6 years ago, most of them are junks. For the ones that are truly good work, which checks all the boxes: new result, hard problem, solid validation, it was easy to accept. For yet a few other papers, which all seem to fall into the feeling category, everything looks right, but it was always on a borderline. And the review results can vary substantially based on the reviewers' own backgrounds.
- jdonaldson 6y agoYeah, I don't like this title. Coding for backprop is worth getting excited about, but please don't assume it supersedes all forms of "predictive coding". Plenty of predictive learning techniques do just fine without it, including our own brains. In keeping with the No-Free-Lunch theorem, it's also highly desirable in general to have a variety of approaches at hand for solving certain predictive coding problems. Yes, this makes ML (as a field) cumbersome, but it also prevents us from painting ourselves into a corner.
- nerdponx 6y agoIs this "coding for backprop", or "coding for the same results as backprop"?
- hctaw 6y agoI don't know enough about biology or ML to know if what I'm posting below is totally wrong, but here goes. "Backprop" == "Feedback" of a non-linear dynamical system. Feedback is mathematical description of the behavior of systems, not a literal one. I don't know of BNNs are incapable of backprop anymore than an RLC filter is incapable of "feedback" when analyzing the ODE of the latter tells you that there's a feedback path (which is what, physically? The return path for charge?) So what makes BNN incapable of feedback? Are they mechanically and electrically insulated from eachother? How do they share information, and what is the return path? Other than that I wish more unification was done on ML algorithms and dynamical systems, just in general. There's too much crossover to ignore.
- khawkins 6y ago> Other than that I wish more unification was done on ML algorithms and dynamical systems, just in general. There's too much crossover to ignore. Check out this work, "Deep relaxation: partial differential equations for optimizing deep neural networks" by Pratik Chaudhari, Adam Oberman, Stanley Osher, Stefano Soatto & Guillaume Carlier. https://link.springer.com/article/10.1007/s40687-018-0148-y https://link.springer.com/article/10.1007/s40687-018-0148-y
- nerdponx 6y agoThe article says this: > The backpropagation algorithm requires information to flow forward and backward along the network. But biological neurons are one-directional. An action potential goes from the cell body down the axon to the axon terminals to another cell's dendrites. An axon potential never travels backward from a cell's terminals to its body. The point of the research here is that backpropagation turns out not to be necessary to fit a neural network, and that it can be approximated with predictive coding, which does not require end-to-end backwards information flow.
- salawat 6y agoSo... I don't understand. >An action potential goes from the cell body down the axon to the axon terminals to another cell's dendrites. How do you figure that doesn't allow backprop? A neuronal bit is a loop of neurons. Information absolutely can back- propagate. If it couldn't, how does anyone think it'd be at all possible to learn how to get better at anything? Neuron fires dendrite to axon, secondary neuron fires dendrite to Axon, Axon branches back to previous neuron's dendrites, rinse, repeat, or add more intervening neurons... Trying to disinclude backprop based on the morphology of a single neuron is... Kinda missing the point. It's all about the level of connection between neurons and how long or whether a signal returns unmodified to the progenitor that effects the stability of the encoded information or behavior. At least to the best I've been able to plausibly model it. Haven't exactly figured out how to shove a bunch of measuring sticks in there to confirm or deny, but I just can't how a uniderectional action potential forwarding element implies lack of backprop in a graph of connections fully capable of developing cycles.
- cs702 6y agoEDIT: Before you read my comment below, please see https://news.ycombinator.com/item?id=26702815 https://news.ycombinator.com/item?id=26702815 and https://openreview.net/forum?id=PdauS7wZBfC https://openreview.net/forum?id=PdauS7wZBfC for a different view. -- If the results hold, they seem significant enough to me that I'd go as far as saying the authors of the paper would end up getting an important award at some point, not just for unifying the fields of biological and artificial intelligence, but also for making it trivial to train models in a fully distributed manner, with all learning done locally -- if the results hold. Here's the paper: "Predictive Coding Approximates Backprop along Arbitrary Computation Graphs" https://arxiv.org/abs/2006.04182 https://arxiv.org/abs/2006.04182 I'm making my way through it right now.
- JackFr 6y agoMy background is as an interested amateur, but > also for making it trivial to train models in a fully distributed manner, with all learning done locally seems like a really huge development. At the same time I remain pretty skeptical of claims of unifying the fields of biological and artificial intelligence. I think the recent tremendous successes in AI & ML lead to an unjustified over confidence that we are close to understanding the way biological systems must work.
- himinlomax 6y agoIndeed, it's worth mentioning we still have absolutely no idea how memory works.
- andyxor 6y agowe know a lot about memory, but most AI researchers are simply ignorant in neuroscience or cognitive psychology and stick with their comfort zone. Saying "we have no idea" is just being lazy.
- TaupeRanger 6y agoNo. We really have no idea what is going on. We only know some basic psychology about it (holding 7 things in short term, etc.) If we knew something about implementation, we could implement human-like memory.
- fouric 6y ago> Predictive coding is the idea that BNNs generate a mental model of their environment and then transmit only the information that deviates from this model. Predictive coding considers error and surprise to be the same thing. This reminds me of a Slate Star Codex article on Friston[1]. [1] https://slatestarcodex.com/2018/03/04/god-help-us-lets-try-to-understand-friston-on-free-energy/ https://slatestarcodex.com/2018/03/04/god-help-us-lets-try-t...
- adamnemecek 6y agoI think that this sort of forward backward thing is a very general idea. There’s a one to many relationship called the adjoint, and a many to one relationship called the norm. I wrote something about this here https://github.com/adamnemecek/adjoint https://github.com/adamnemecek/adjoint
- tsmithe 6y agoIn fact, the compositional structure underlying that of predictive coding [0,1] is abstractly the same as that underlying backprop [2]. (Disclaimer: [0,1] are my own papers; I'm working on a more precise and extensive version of [1] right now!) [0] https://arxiv.org/abs/2006.01631 https://arxiv.org/abs/2006.01631 [1] https://arxiv.org/abs/2101.10483 https://arxiv.org/abs/2101.10483 [2] https://arxiv.org/abs/1711.10455 https://arxiv.org/abs/1711.10455
- eli_gottlieb 6y agoHurry and publish before I have manuscripts ready applying these results.
- tsmithe 6y agoHey, Eli :-) I'm working on it; I'll send you an e-mail. Things quickly turned out to be more general than I realized last year.
- eli_gottlieb 6y agoThat would make sense. The whole ACT/categorical cybernetics community has been working out how massively general optics are :-).
- selimthegrim 6y agoWhat were you going to say about Young tableaux?
- 6y ago
- klmadfejno 6y ago> Predictive coding is the idea that BNNs generate a mental model of their environment and then transmit only the information that deviates from this model. Predictive coding considers error and surprise to be the same thing. Hebbian theory is specific mathematical formulation of predictive coding. This is an excellent, concise explanation. It sounds intuitive as something that could work. Would love to try and dabble with this. Any resources?
- ilaksh 6y agoDoes anyone know of a simple code example that demonstrates the original predictive coding concept from 1999? Ideally applied to some type of simple image/video problem. I thought I saw a Matlab explanation of that 99 paper but have not found it again.
- phreeza 6y agoThis was already shown for MLPs some years ago, and it is not really that surprising that it applies to many other architectures. Note that while learning can take place locally, it does still require an upward and downward stream of information flow, which is not supported by the neuroanatomy in all cases. So while it is an interesting avenue of research, I don't think it's anywhere near as revolutionary as this blog post makes it out to be.
- 0lmer 6y agoBut does predictive coding perceived as a valid theory for cortical neurons functioning? There was a paper from 2017 drawing similar conclusions about backprop approximation with Spike-Timing-Dependent Plasticity: https://arxiv.org/abs/1711.04214 https://arxiv.org/abs/1711.04214 Looks more grounded to current models of neuronal functioning. Nevertheless, it changed nothing in the field of deep learning since then.
- andyxor 6y agoas long as the model requires delta rule, or 'teacher signal' based error correction it is not biologically plausible.
- jwmullally 6y agoSome general background on STDP for the thread: Biological neurons don't just emit constant 0...1 float values, they communicate using time sensitive bursts of voltage known as "spike trains". Spiking Neural Networks (SNN) are a closer aproximation of natural networks than typical ML ANNs. [0] gives a quick overview. Spike-Timing-Dependant-Plasticity is a local learning rule experimentally observed in biological neurons. It's a form of Hebbian learning, aka "Neurons that fire together wire together." Summary from [1]. The top graph gives a clear picture of how the rule works. > With STDP, repeated presynaptic spike arrival a few milliseconds before postsynaptic action potentials leads in many synapse types to Long-Term Potentiation (LTP) of the synapses, whereas repeated spike arrival after postsynaptic spikes leads to Long-Term Depression (LTD) of the same synapse. --- [0]: https://towardsdatascience.com/deep-learning-versus-biological-neurons-floating-point-numbers-spikes-and-neurotransmitters-6eebfa3390e9 https://towardsdatascience.com/deep-learning-versus-biologic... [1]: http://www.scholarpedia.org/article/Spike-timing_dependent_plasticity http://www.scholarpedia.org/article/Spike-timing_dependent_p...
- laplacesdemon48 6y ago1) How similar is this to creating a resnet? [1]. What are some key differences and similarities? 2) Has a CNN version of this been implemented in PyTorch? [1] https://arxiv.org/pdf/1512.03385.pdf https://arxiv.org/pdf/1512.03385.pdf (Figure 2)
- lukeplato 6y agoThe paper was posted on HN a few months ago: https://news.ycombinator.com/item?id=24693609 https://news.ycombinator.com/item?id=24693609
- Animats 6y agoIs this approach "more local", in the sense that you could build hardware where local units got work done with less communication? That would have potential. It's feasible to build ICs with a few million simple compute units if they don't have to talk to each other or to memory much. GPUs are a few hundred or a few thousand parallel units that talk to memory a lot.