3 ms·

"Whenever these kind of papers come out I skim it looking for where they actually do backprop. Check the pseudo code of their algorithms. "Update using gradie

"Whenever these kind of papers come out I skim it looking for where they actually do backprop. Check the pseudo code of their algorithms. "Update using gradient based optimizations""

arrakark 2y ago
Same. If I had to guess it's just local gradients, not an end-to-end gradient.
f_devd 2y ago
I mean the only claim is no propagation, you always need a gradient of sorts to update parameters. Unless you just stumble upon the desired parameters. Even genetic algorithms effectively has gradients which are obfuscated through random projections.
- erikerikson 2y ago
  No you don't. See Hebbian learning (neurons that fire together wire together). Bonus: it is one of the biologically plausible options. Maybe you have a way of seeing it differently so that this looks like a gradient? Gradient keys my brain into a desired outcome expressed as an expectation function.
  - red75prime 2y ago
    > See Hebbian learning The one that is not used, because it's inherently unstable? Learning using locally accessible information is an interesting approach, but it needs to be more complex than "fire together, wire together". And then you might have propagation of information that allows to approximate gradients locally.
    - erikerikson 2y ago
      Is that what they're teaching now? Originally it was not used because it was believed it couldn't learn XOR (it can [just not as perceptrons were defined]). Is there anyone in particular whose work focuses on this that you know of?
      - ckcheng 2y ago
        Oja's rule dates back to 1982? It’s Hebbian and solves all stability problems. https://en.wikipedia.org/wiki/Oja's_rule https://en.wikipedia.org/wiki/Oja's_rule
  - yobbo 2y ago
    If there is a weight update, there is a gradient, and a loss objective. You might not write them down explicitly. I can't recall exactly what the Hebbian update is, but something tells me it minimises the "reconstruction loss", and effectively learns the PCA matrix.
    - orbifold 2y ago
      Not every vector field has a potential. So not every weight update can be written as a gradient.
      - yobbo 2y ago
        True.
    - erikerikson 2y ago
      > loss objective There is no prediction or desired output, certainly explicit. I was playing with those things in my work to try and understand how our brains cause the emergence of intelligence rather than solve some classification or related problem. What I managed to replicate was the learning of XOR by some nodes and further that multidimensional XORs up to the number of inputs could be learned. Perhaps you can say that PCAish is the implicit objective/result but I still reject that there is any conceptual notion of what a node "should" output even if iteratively applying the learning rule leads us there.
  - HarHarVeryFunny 2y ago
    Even with Hebbian learning, isn't there a synapse strength? If so, then you at least need a direction (+/-) if not a specific gradient value.
    - erikerikson 2y ago
      Yes there is a weight on every connection. At least when I was at it gradients were talked about in reference to the solution space (e.g. gradient descent). The implication is that there is some notion of what is "correct"for some neutron to have output and then we bend it to our will by updating the weight. In Hebbian learning there isn't a notion of correct activation, just a calculation over the local environment.
  - srean 2y ago
    Nope that update with the rank one update is exactly the projected gradient of the reconstruction loss. That's not the way it is usually taught. So Hebbian learning was an unfortunate example. Gradient descent is only one way of searching for a minima, so in that sense it is not necessary, for example, when one can analytically solve for the extrema of the loss. As an alternative one could do Monte Carlo search instead of gradient descent. For a convex loss that would be less efficient of course.
- bob1029 2y ago
  In genetic algorithms, any gradient found would be implied by way of the fitness function and would not be something to inherently pursue. There are no free lunches like with chain rule of calculus. GP is essentially isomorphic with beam search where the population is the beam. It is a fancy search algorithm. It is not "training" anything.
  - f_devd 2y ago
    True, genetic algorithms are only implied, but those implied gradients are used in the more successful evolutionary strategies. So while they might not look like it (because it's not used in a continuous descent) they still very much work like (although they represent a smoother function than) regular back-prop gradients when aggregated.
- gsf_emergency_2 2y ago
  GP glancing at the pseudo-code is certainly an efficient way to dismiss an article, but something tells me he missed the crucial sentence in the abstract: >"We believe this work takes a first step TOWARDS introducing a new family of GRADIENT-FREE learning methods" I.e. for the time being, authors can't convince themselves not to take advantage of efficient hw for taking gradients (*Checks that Oxford University is not under sanctions*)
scarmig 2y ago
Check out feedback alignment. You provide feedback with a random static linear transformation of the loss to earlier layers, and they eventually align with the feedback matrix to enable learning. It's certifiably insane that it works at all. And not even vaguely backprop, though if you really wanted to stretch the definition I guess you could say that the feedforward layers align to take advantage of a synthetic gradient in a way that approximates backprop.
isentrop1c 1y ago
Nicely stolen comment from reddit (https://www.reddit.com/r/MachineLearning/comments/1jsft3c/r_noprop_training_neural_networks_without/mlotmtx/ https://www.reddit.com/r/MachineLearning/comments/1jsft3c/r_...)

"Whenever these kind of papers come out I skim it looking for where they actually do backprop. Check the pseudo code of their algorithms. "Update using gradient based optimizations""

arrakark 2y ago

Same. If I had to guess it's just local gradients, not an end-to-end gradient.

f_devd 2y ago

I mean the only claim is no propagation, you always need a gradient of sorts to update parameters. Unless you just stumble upon the desired parameters. Even genetic algorithms effectively has gradients which are obfuscated through random projections.

erikerikson 2y ago
No you don't. See Hebbian learning (neurons that fire together wire together). Bonus: it is one of the biologically plausible options. Maybe you have a way of seeing it differently so that this looks like a gradient? Gradient keys my brain into a desired outcome expressed as an expectation function.
- red75prime 2y ago
  > See Hebbian learning The one that is not used, because it's inherently unstable? Learning using locally accessible information is an interesting approach, but it needs to be more complex than "fire together, wire together". And then you might have propagation of information that allows to approximate gradients locally.
  - erikerikson 2y ago
    Is that what they're teaching now? Originally it was not used because it was believed it couldn't learn XOR (it can [just not as perceptrons were defined]). Is there anyone in particular whose work focuses on this that you know of?
    - ckcheng 2y ago
      Oja's rule dates back to 1982? It’s Hebbian and solves all stability problems. https://en.wikipedia.org/wiki/Oja's_rule https://en.wikipedia.org/wiki/Oja's_rule
- yobbo 2y ago
  If there is a weight update, there is a gradient, and a loss objective. You might not write them down explicitly. I can't recall exactly what the Hebbian update is, but something tells me it minimises the "reconstruction loss", and effectively learns the PCA matrix.
  - orbifold 2y ago
    Not every vector field has a potential. So not every weight update can be written as a gradient.
    - yobbo 2y ago
      True.
  - erikerikson 2y ago
    > loss objective There is no prediction or desired output, certainly explicit. I was playing with those things in my work to try and understand how our brains cause the emergence of intelligence rather than solve some classification or related problem. What I managed to replicate was the learning of XOR by some nodes and further that multidimensional XORs up to the number of inputs could be learned. Perhaps you can say that PCAish is the implicit objective/result but I still reject that there is any conceptual notion of what a node "should" output even if iteratively applying the learning rule leads us there.
- HarHarVeryFunny 2y ago
  Even with Hebbian learning, isn't there a synapse strength? If so, then you at least need a direction (+/-) if not a specific gradient value.
  - erikerikson 2y ago
    Yes there is a weight on every connection. At least when I was at it gradients were talked about in reference to the solution space (e.g. gradient descent). The implication is that there is some notion of what is "correct"for some neutron to have output and then we bend it to our will by updating the weight. In Hebbian learning there isn't a notion of correct activation, just a calculation over the local environment.
- srean 2y ago
  Nope that update with the rank one update is exactly the projected gradient of the reconstruction loss. That's not the way it is usually taught. So Hebbian learning was an unfortunate example. Gradient descent is only one way of searching for a minima, so in that sense it is not necessary, for example, when one can analytically solve for the extrema of the loss. As an alternative one could do Monte Carlo search instead of gradient descent. For a convex loss that would be less efficient of course.
bob1029 2y ago
In genetic algorithms, any gradient found would be implied by way of the fitness function and would not be something to inherently pursue. There are no free lunches like with chain rule of calculus. GP is essentially isomorphic with beam search where the population is the beam. It is a fancy search algorithm. It is not "training" anything.
- f_devd 2y ago
  True, genetic algorithms are only implied, but those implied gradients are used in the more successful evolutionary strategies. So while they might not look like it (because it's not used in a continuous descent) they still very much work like (although they represent a smoother function than) regular back-prop gradients when aggregated.
gsf_emergency_2 2y ago
GP glancing at the pseudo-code is certainly an efficient way to dismiss an article, but something tells me he missed the crucial sentence in the abstract: >"We believe this work takes a first step TOWARDS introducing a new family of GRADIENT-FREE learning methods" I.e. for the time being, authors can't convince themselves not to take advantage of efficient hw for taking gradients (*Checks that Oxford University is not under sanctions*)

scarmig 2y ago

Check out feedback alignment. You provide feedback with a random static linear transformation of the loss to earlier layers, and they eventually align with the feedback matrix to enable learning. It's certifiably insane that it works at all. And not even vaguely backprop, though if you really wanted to stretch the definition I guess you could say that the feedforward layers align to take advantage of a synthetic gradient in a way that approximates backprop.

isentrop1c 1y ago

Nicely stolen comment from reddit (https://www.reddit.com/r/MachineLearning/comments/1jsft3c/r_noprop_training_neural_networks_without/mlotmtx/ https://www.reddit.com/r/MachineLearning/comments/1jsft3c/r_...)