5 ms·

Predictive Coding Approximates Backprop Along Arbitrary Computation Graphs

92 points by lukeplato 6y ago

deleted 6y ago
[deleted]
p1esk 6y ago
For those asking about Imagenet results, this method is 100x more computationally intensive than a regular backprop. The code seems legit at first glance (in Pytorch, even though they pretty much implemented everything from scratch). The main result is the local Hebbian-like learning converges to exactly the same gradients as the ones produced with backprop.
- SubiculumCode 6y ago
  "100x more computationally expensive" but also, intrinsically more parallelizable than back prop. From the paper: "In terms of computational cost, one inference iteration in the predictive coding network is about as costly as a backprop backwards pass. Thus, due to using 100-200 iterations for full convergence, our algorithm is substantially more expensive than backprop which limits the scalability of our method. However, this serial cost is misleading when talking about highly parallel neural architectures. In the brain, neurons cannot wait for a sequential forward and backward sweep. By phrasing our algorithm as a global descent, our algorithm is fully parallel across layers. There is no waiting and no phases to be coordinated. Each neuron need only respond to its local driving inputs and downwards error signals. We believe that this local and parallelizable property of our algorithm may engender the possibility of substantially more efficient implementations on neuromorphic hardware"
  - p1esk 6y ago
    Yes, I'm not concerned with the computational cost. Reading the paper more carefully, I can't see any obvious weaknesses. The main idea is based on [1]: a brain can learn exactly the same things as what can be learned with backprop. How come this is not more widely known? This sounds like a huge breakthrough to me. https://www.mitpressjournals.org/doi/full/10.1162/NECO_a_00949 https://www.mitpressjournals.org/doi/full/10.1162/NECO_a_009...
    - orbifold 6y ago
      They don't address the fundamental issue of BPTT in an RNN model. To my mind this is a much larger block to biological plausibility than how to track the error locally. Besides this has been addressed in various forms in earlier work: - Hochreiter and Schmidhuber present a truncated version of BPTT in the original LSTM paper that tracks part of the error locally and forward in time (https://www.bioinf.jku.at/publications/older/2604.pdf https://www.bioinf.jku.at/publications/older/2604.pdf) - Bellec et al. https://www.nature.com/articles/s41467-020-17236-y https://www.nature.com/articles/s41467-020-17236-y present a similar algorithm that tracks the error with something they call eligibility traces. Despite the title of the paper the method is also applicable to non-spiking RNNs. - Bohnstigl et al. present another solution: https://arxiv.org/pdf/2007.12723.pdf https://arxiv.org/pdf/2007.12723.pdf
  - nl 6y ago
    This maybe true, but on existing hardware isn't super useful. Even on the potential neuromophic hardware it's "only" parallel with the number of layers. That's useful but not as parallel as current approaches.
    - p1esk 6y ago
      What do you mean? Looking at the equations, I don't see why this local learning rule can't be parallel with the number of neurons, so it's vastly more parallel, but requires multiple iterations per neuron (100x in the paper).
  - thesz 6y ago
    >In the brain, neurons cannot wait for a sequential forward and backward sweep. I think that neurons hate when we animate them. Being a little more serious, is it possible to perform gradual back propagation? I.e., for iteration 1 train only last layer, for iteration 2 change two last layers, for iteration 4 train three last layers, etc. I also have to say that in my spare time I am investigating a way to train networks on the whole set as opposed to the single sample or batch back propagation and one result I've got so far is not quite uninteresting, in my opinion: https://github.com/thesz/higgs-logistic-regression https://github.com/thesz/higgs-logistic-regression The approximation of gradients is still use of gradients. It still computes updates which are need to be shared and still has some bottleneck on scalability.
Mumps 6y ago
For those curious on the topic of Predictive Coding, I'd highly recommend reading two of they key source papers on it: 1. James CR Whittington and Rafal Bogacz. An approximation of the error backpropagation algorithmin a predictive coding network with local hebbian synaptic plasticity.Neural computation, 29(5):1229–1262, 2017. 2. Rafal Bogacz. A tutorial on the free-energy framework for modelling perception and learning.Journalof mathematical psychology, 76:198–211, 2017. (source: grad student working on PC networks. happy to chat if anyone wants)
- igorkraw 6y ago
  Can we discuss via mail? You can find my contact in my profile. I started out with neuromorphic research before I switched to something else and this paper basically was my original research plan (but I never would have arrived at actually deriving this on the old trajectory I was on I think) so I'd love to go into the nitty gritty details
- l33tman 6y ago
  Yeah I'd be happy to chat some time, also been doing some PC-related work! BTW the PC-stuff goes back to the 90's (at least), Rao and Ballard 1999 is usually referred to. The 2017 papers quoted above are specifically about comparing with backprop gradients.
jojo_was_a_man 6y ago
Can this apply also to Spiking neural network?(SNN)
jandrewrogers 6y ago
It is an interesting result, I worked on the design of efficient computational engines for predictive coding type models almost two decades ago. Intuitively it seemed like this should be the case but it was never demonstrated. Naive implementations were slow even at the time because obvious ways of prediction error minimization for these computational structures is fairly pathological in multiple dimensions. Somewhere in my dusty research archives is a new mechanism for doing close to optimal prediction error minimization at scale at a much higher efficiency such that it became practical. This might motivate me to revive that work; few people were interested at the time.

92 points by lukeplato 6y ago

deleted 6y ago

[deleted]

For those asking about Imagenet results, this method is 100x more computationally intensive than a regular backprop. The code seems legit at first glance (in Pytorch, even though they pretty much implemented everything from scratch). The main result is the local Hebbian-like learning converges to exactly the same gradients as the ones produced with backprop.

SubiculumCode 6y ago
"100x more computationally expensive" but also, intrinsically more parallelizable than back prop. From the paper: "In terms of computational cost, one inference iteration in the predictive coding network is about as costly as a backprop backwards pass. Thus, due to using 100-200 iterations for full convergence, our algorithm is substantially more expensive than backprop which limits the scalability of our method. However, this serial cost is misleading when talking about highly parallel neural architectures. In the brain, neurons cannot wait for a sequential forward and backward sweep. By phrasing our algorithm as a global descent, our algorithm is fully parallel across layers. There is no waiting and no phases to be coordinated. Each neuron need only respond to its local driving inputs and downwards error signals. We believe that this local and parallelizable property of our algorithm may engender the possibility of substantially more efficient implementations on neuromorphic hardware"
- p1esk 6y ago
  Yes, I'm not concerned with the computational cost. Reading the paper more carefully, I can't see any obvious weaknesses. The main idea is based on [1]: a brain can learn exactly the same things as what can be learned with backprop. How come this is not more widely known? This sounds like a huge breakthrough to me. https://www.mitpressjournals.org/doi/full/10.1162/NECO_a_00949 https://www.mitpressjournals.org/doi/full/10.1162/NECO_a_009...
  - orbifold 6y ago
    They don't address the fundamental issue of BPTT in an RNN model. To my mind this is a much larger block to biological plausibility than how to track the error locally. Besides this has been addressed in various forms in earlier work: - Hochreiter and Schmidhuber present a truncated version of BPTT in the original LSTM paper that tracks part of the error locally and forward in time (https://www.bioinf.jku.at/publications/older/2604.pdf https://www.bioinf.jku.at/publications/older/2604.pdf) - Bellec et al. https://www.nature.com/articles/s41467-020-17236-y https://www.nature.com/articles/s41467-020-17236-y present a similar algorithm that tracks the error with something they call eligibility traces. Despite the title of the paper the method is also applicable to non-spiking RNNs. - Bohnstigl et al. present another solution: https://arxiv.org/pdf/2007.12723.pdf https://arxiv.org/pdf/2007.12723.pdf
- nl 6y ago
  This maybe true, but on existing hardware isn't super useful. Even on the potential neuromophic hardware it's "only" parallel with the number of layers. That's useful but not as parallel as current approaches.
  - p1esk 6y ago
    What do you mean? Looking at the equations, I don't see why this local learning rule can't be parallel with the number of neurons, so it's vastly more parallel, but requires multiple iterations per neuron (100x in the paper).
- thesz 6y ago
  >In the brain, neurons cannot wait for a sequential forward and backward sweep. I think that neurons hate when we animate them. Being a little more serious, is it possible to perform gradual back propagation? I.e., for iteration 1 train only last layer, for iteration 2 change two last layers, for iteration 4 train three last layers, etc. I also have to say that in my spare time I am investigating a way to train networks on the whole set as opposed to the single sample or batch back propagation and one result I've got so far is not quite uninteresting, in my opinion: https://github.com/thesz/higgs-logistic-regression https://github.com/thesz/higgs-logistic-regression The approximation of gradients is still use of gradients. It still computes updates which are need to be shared and still has some bottleneck on scalability.

Mumps 6y ago

For those curious on the topic of Predictive Coding, I'd highly recommend reading two of they key source papers on it: 1. James CR Whittington and Rafal Bogacz. An approximation of the error backpropagation algorithmin a predictive coding network with local hebbian synaptic plasticity.Neural computation, 29(5):1229–1262, 2017. 2. Rafal Bogacz. A tutorial on the free-energy framework for modelling perception and learning.Journalof mathematical psychology, 76:198–211, 2017. (source: grad student working on PC networks. happy to chat if anyone wants)

igorkraw 6y ago
Can we discuss via mail? You can find my contact in my profile. I started out with neuromorphic research before I switched to something else and this paper basically was my original research plan (but I never would have arrived at actually deriving this on the old trajectory I was on I think) so I'd love to go into the nitty gritty details
l33tman 6y ago
Yeah I'd be happy to chat some time, also been doing some PC-related work! BTW the PC-stuff goes back to the 90's (at least), Rao and Ballard 1999 is usually referred to. The 2017 papers quoted above are specifically about comparing with backprop gradients.

jojo_was_a_man 6y ago

Can this apply also to Spiking neural network?(SNN)

jandrewrogers 6y ago

It is an interesting result, I worked on the design of efficient computational engines for predictive coding type models almost two decades ago. Intuitively it seemed like this should be the case but it was never demonstrated. Naive implementations were slow even at the time because obvious ways of prediction error minimization for these computational structures is fairly pathological in multiple dimensions. Somewhere in my dusty research archives is a new mechanism for doing close to optimal prediction error minimization at scale at a much higher efficiency such that it became practical. This might motivate me to revive that work; few people were interested at the time.