4 ms·
Predictive Coding Approximates Backprop Along Arbitrary Computation Graphs (2020)
- amelius 5y agoIf backprop is not needed, would this finding make automatic-differentiation functionality obsolete in DL frameworks, allowing these frameworks to become much simpler? Or is there still some constant factor that makes backprop favorable?
- coolness 5y agoNeeds [2020] in the title. Interesting work nevertheless.
- deleted 5y ago[deleted]
- yewenjie 5y agoAstralCodexTen (formerly SlateStarCodex) has discussed this here - https://astralcodexten.substack.com/p/link-unifying-predictive-coding-with https://astralcodexten.substack.com/p/link-unifying-predicti... He mostly points to this post in LessWrong - https://www.lesswrong.com/posts/JZZENevaLzLLeC3zn/predictive-coding-has-been-unified-with-backpropagation https://www.lesswrong.com/posts/JZZENevaLzLLeC3zn/predictive...
- obstbraende 5y agohere's a link with reviewer comments https://openreview.net/forum?id=PdauS7wZBfC https://openreview.net/forum?id=PdauS7wZBfC (praise to openreview!)
- pishpash 5y agoThe decision reasoning is super helpful to put things in context. The arbitrary binary decision to "accept" vs "reject" especially for the snooty "high bar for acceptance at ICLR" is laughable in a world of free information access.
- pishpash 5y agoAlso, most recent state of progress: Predictive Coding Can Do Exact Backpropagation on Any Neural Network (2021) https://arxiv.org/abs/2103.04689 https://arxiv.org/abs/2103.04689
- l33tman 5y agoThis (and its follow up papers) have already been discussed multiple times here. Don't have the links handy though..