26 ms·

the thing is about every week there is a paper published with groundbreaking claims, with this question in particular being very popular, trying to unify neuro

the thing is about every week there is a paper published with groundbreaking claims, with this question in particular being very popular, trying to unify neuroscience and deep learning in some way, in search for computational foundations of AI. Mostly this is driven by success of DL in certain industrial applications. Unfortunately most of these papers are heavy on theory but light on empirical evidence. If we follow the path of natural sciences, theory has to agree with evidence. Otherwise it's just another theory unconstrained by reality, or worse, pseudo-science.

autopoiesis 6y ago
The paper (arxiv:2103.04689) linked by eutropia above has some empirical evidence on the ML side, showing that performance of predictive coding is not so far off backprop. And there is no shortage of suggestions for how neural circuits might work around the strict requirements of backprop-like algorithms. cs702's original comment above is excessively hyperbolic: the compositional structure of Bayesian inversion is well known and is known to coincide structurally with the backward/forward structure of automatic differentiation. And there have been many papers before this one showing how predictive coding approximates backprop in other cases, so it is no surprise that it can do so on graphs, too. I agree with the ICLR reviewers that this paper is borderline and not in itself a major contribution. But that does not mean that this whole endeavour, of trying to find explicit mathematical connections between biological and artificial learning, is ill motivated.
- eli_gottlieb 6y ago
  >the compositional structure of Bayesian inversion is well known /u/tsmithe's results on that are well known, now? I can scarcely find anyone to collaborate with who understands them!
TaupeRanger 6y ago
Not only light on evidence, but essential practicality-free. There's no "there" there. Literally nothing useful will come from this.

autopoiesis 6y ago

The paper (arxiv:2103.04689) linked by eutropia above has some empirical evidence on the ML side, showing that performance of predictive coding is not so far off backprop. And there is no shortage of suggestions for how neural circuits might work around the strict requirements of backprop-like algorithms. cs702's original comment above is excessively hyperbolic: the compositional structure of Bayesian inversion is well known and is known to coincide structurally with the backward/forward structure of automatic differentiation. And there have been many papers before this one showing how predictive coding approximates backprop in other cases, so it is no surprise that it can do so on graphs, too. I agree with the ICLR reviewers that this paper is borderline and not in itself a major contribution. But that does not mean that this whole endeavour, of trying to find explicit mathematical connections between biological and artificial learning, is ill motivated.

eli_gottlieb 6y ago
>the compositional structure of Bayesian inversion is well known /u/tsmithe's results on that are well known, now? I can scarcely find anyone to collaborate with who understands them!

TaupeRanger 6y ago

Not only light on evidence, but essential practicality-free. There's no "there" there. Literally nothing useful will come from this.